Whitepaper: Backwards Compatibility — the promise, the cost, and the platform decision - rdkcentral/rdk-halif-aidl GitHub Wiki

• Type: Product Whitepaper
• Audience: Product and programme. Technical readers welcome; no engineering background assumed
• Purpose: To explain what our interface contracts promise, what that promise costs us, and the one decision it needs from product — where the platform itself is allowed to break

A frozen interface never breaks. That promise is the whole reason the vendor layer is cheap, and everything we want from a HAL rests on it holding.

Keeping the promise has an exact and unforgiving meaning: we may only add, and we may never remove. When we genuinely need to change our mind, we do not edit the interface — we publish a second one beside the first and let consumers move at their own pace. That works, and it costs us two of everything for as long as both are in use.

We have not made the permanent promise yet, and the window in which we can still shape an interface is closing. The intent is to freeze each one as early as it can responsibly be frozen, and to have every one of them frozen before we reach production — the last pre-freeze version simply becomes the first frozen version. Getting the shape right now costs a rebuild; the same correction after freeze costs a second interface, permanently.

The gap is one layer up. Every one of these rules governs a single interface. Nothing defines where the platform is allowed to break. Android does this deliberately at V12, V13, V14. We have no equivalent, and until we do, every genuine need for a clean break has nowhere to land. Defining what an RDK major release is permitted to change is a product decision, and this paper asks for it.

An interface is a contract between us and a silicon vendor. Once we freeze one, we are telling every vendor and every OEM: build against this, and it will keep working.

That promise is what makes the vendor layer cheap:

• A vendor implements once, rather than re-implementing for each consumer that comes along.
• An OEM upgrades middleware without renegotiating with silicon.
• A device already in the field keeps working when the software above it moves on.

Every benefit we claim for a HAL is downstream of the promise holding. Break it once and the vendor's estimate for the next component goes up, because they now price in the risk of doing the work twice.

"It will keep working" is stricter than most people expect.

We may only add. New capabilities are appended to an existing interface. That is the entire list of permitted changes.

We may never remove, rename, reorder or change the shape of anything. Not a call nobody uses. Not a field we regret. Not a name we got wrong on the first pass. The moment a contract is frozen, everything in it is permanent.

The reason is not tidiness, it is control. We do not choose who implemented the interface, and we do not choose when they next rebuild. A vendor who shipped against last year's contract is entitled to have it still mean what it meant. Android governs the same kind of interface with the same rule, for the same reason.

The consequence product should hear: a tidy-up is never available to us. A frozen interface only ever grows.

flowchart LR
    v1["Version 1<br/>A, B"]
    v2["Version 2<br/>A, B, C"]
    v3["Version 3<br/>A, B, C, D"]
    v1 -->|"add C"| v2
    v2 -->|"add D"| v3
    no1["Remove B"]
    no2["Rename A"]
    no3["Change what C means"]
    v2 -.->|"never"| no1
    v2 -.->|"never"| no2
    v2 -.->|"never"| no3

    classDef ok fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
    classDef bad fill:#ffebee,stroke:#c62828,color:#b71c1c,stroke-dasharray:4 3
    class v1,v2,v3 ok
    class no1,no2,no3 bad

A frozen interface only ever gains. Everything already in it stays, and keeps meaning what it meant.

We publish a second interface beside the first.

The original keeps working and keeps its consumers. The new one carries the redesign. Consumers move when their own plans allow, and nothing breaks on the way across. In time the old one falls out of use — but it stays published, because it must.

flowchart LR
    built["Vendor who built<br/>against the original"]
    newc["New consumer"]

    old["<b>Original interface</b><br/>unchanged, still published"]
    new["<b>Second interface</b><br/>carries the redesign"]

    built -->|"keeps working, unchanged"| old
    newc -->|"starts here"| new
    built -.->|"migrates on its own schedule"| new

    classDef keep fill:#e3f2fd,stroke:#1565c0,color:#0d47a1
    classDef fresh fill:#e8f5e9,stroke:#2e7d32,color:#1b5e20
    class old keep
    class new fresh

Nobody is forced to move, and nothing breaks while they decide. That is the whole benefit — and the reason we carry both for as long as it takes.

This is well understood and entirely manageable. What it is not is free.

The cost: for as long as both are in use we carry two interfaces, two implementations and two sets of tests — and we ask vendors to do the same.

That cost is the real argument for spending time on the shape of an interface before it is frozen rather than after. It is the same correction either way; only the price changes.

Every component is currently pre-freeze, which means breaking changes are still allowed. That is the only window in which they are, and it is not open indefinitely.

The intent is to freeze each interface as soon as it can responsibly be frozen, and to have every one of them frozen before we reach production.

Freezing is not a redesign. The last pre-freeze version becomes the first frozen version unchanged — we simply stop being allowed to alter it. There is no separate rewrite at the boundary, and nothing should be planned as though there were.

What we are waiting on is evidence, not time. An interface is ready when it has been reviewed and — the part that matters most — implemented on more than one silicon platform, because the second and third platform are where the assumptions baked in by the first become visible. Once it has survived that, holding it open buys nothing and costs everyone the ability to rely on it.

A change made now costs a rebuild. The same change made after freezing costs a second interface, permanently. That is the entire argument for spending the effort now, while it is still only a rebuild.

flowchart LR
    pre["<b>Pre-freeze</b><br/>we are here<br/><br/>breaking changes allowed<br/>a correction costs a rebuild"]
    ready["<b>Ready</b><br/>reviewed, and running on<br/>a second and third silicon platform"]
    frozen["<b>Frozen</b><br/><br/>additions only<br/>a correction costs a second interface"]
    prod["<b>Production</b><br/><br/>every interface frozen<br/>by this point"]

    pre --> ready
    ready -->|"the last pre-freeze version becomes<br/>the first frozen version, unchanged"| frozen
    frozen --> prod

    classDef open fill:#fff8e1,stroke:#f9a825,color:#e65100
    classDef gate fill:#eceff1,stroke:#546e7a,color:#263238
    classDef shut fill:#e3f2fd,stroke:#1565c0,color:#0d47a1
    class pre open
    class ready gate
    class frozen,prod shut

The window is open now and shuts at production. Freezing is a promotion, not a redesign — the version we stop at is the version that ships.

Two things about this are easy to get wrong, and both matter for planning:

Freezing is per component, not a programme milestone. A component freezes when it is ready. It does not wait for the others, and we will run a mixed estate of frozen and unfrozen components until the last one lands. That is the intended state, not a transitional mess to be cleared.

"Everything frozen" is a milestone, not a gate. Nothing is held back waiting for it, and no plan should be built on it as a dependency.

Everything above governs a single interface. Nothing above says anything about the platform, and that is the gap.

Android does not only ever add. It ships V12, V13, V14 — and at each of those boundaries it breaks things on purpose. A deliberate, scheduled, announced break is how a platform absorbs a change too large to add: a new graphics baseline, a new security model, a capability that cannot be bolted onto the existing shape without disfiguring it.

We have the first half of that model and not the second. We have a firm rule that individual interfaces may not break, and no defined place where the platform is allowed to.

flowchart TB
    need["<b>A change too large to add</b><br/>a new graphics baseline<br/>a new security model"]

    subgraph platform["Platform release — where a deliberate break belongs"]
        gap["RDK9?  RDK10?<br/><b>not defined</b><br/>nothing to plan against"]
    end

    subgraph interfaces["Interfaces — may never break once frozen"]
        i1["Interface"]
        i2["Interface"]
        i3["Interface"]
    end

    need -.->|"has nowhere<br/>to land"| gap
    need ==>|"lands here<br/>instead"| i2

    classDef missing fill:#ffebee,stroke:#c62828,color:#b71c1c,stroke-dasharray:5 4
    classDef solid fill:#e3f2fd,stroke:#1565c0,color:#0d47a1
    classDef src fill:#fff8e1,stroke:#f9a825,color:#e65100
    class gap missing
    class i1,i2,i3 solid
    class need src
    style platform fill:#fafafa,stroke:#b0bec5,color:#37474f
    style interfaces fill:#fafafa,stroke:#b0bec5,color:#37474f

The dotted route is the one that should exist. Because it does not, the pressure takes the solid one — onto interfaces we have promised never to break.

The failure mode is predictable, and the timing makes it urgent. Today, while everything is still pre-freeze, a clean break is cheap — so the absence of a platform major costs us little. The moment interfaces begin freezing, which is the plan and is soon, that stops being true. From then on a genuine need to make a clean break has nowhere to land, and one of two things happens: it is absorbed as yet another addition to an interface already carrying too many, or it is argued about indefinitely, because there is no forum in which "yes, and it lands in the next major" is an available answer.

What is needed is a definition of what RDK9 is, and what RDK10 is — not dates, but a statement of what each major release is permitted to change.

If we decide that RDK10 takes a new graphics baseline, then it is understood from the outset that middleware and possibly the application layer change with it, and every team plans against that. The break becomes a product decision taken once, in the open, with a date attached — instead of a long series of quiet engineering compromises that nobody ever chose.

1. Agree that platform majors exist and are allowed to break. Without somewhere for pressure to go, the only place it can go is into interfaces we have promised not to break. This becomes pressing the moment interfaces start freezing, which is soon and by intent.
2. Decide what defines an RDK major — which classes of change earn one, and what a consumer should expect to re-do when one lands.
3. Recognise that the window for shaping interfaces closes at production. Every interface is expected to be frozen by then, and the last pre-freeze version becomes the first frozen one. Effort spent on shape now is the cheapest it will ever be; the same correction after freeze is a second interface, permanently.

Nothing here slows current delivery, and nothing here is new process for engineering. The rules governing individual interfaces are already written down, already followed, and already working.

What is missing is the layer above them — and that layer is a product decision, not an engineering one.