System Design APIs

API Compatibility & Contracts

Imagine renaming one JSON field on a popular mobile API on a Tuesday. Your web app picks up the change within hours, because you redeploy it yourself. Half the phones in the field still run last month's build, and you cannot force them to update. Those phones keep sending the old field name, the server no longer recognises it, and checkout starts failing with HTTP 400 ("bad request") errors for people who did nothing wrong.

That is the whole problem this page solves. An API outlives any single deploy of the service behind it, so the server that offers it (the provider) and everything that calls it (the consumers) must be able to ship on different schedules. is the promise that old callers keep working; turn that promise into an automated test; and migrations with windows are how you still change things without breaking anyone.

Safe evolution path

Ship new optional fields or endpoints; old clients simply ignore what they do not recognise.

Consumers publish what they rely on; the provider build fails before a breaking deploy.

Serve old and new shapes together, then remove the old only when traffic reads zero.

Announce the removal date with a Sunset header and gate it on measured adoption.

Try it: gate API rollout with a consumer-producer compatibility matrix

Upgrade the producer, tighten consumers, or remove a legacy field. The matrix marks breaking versus expand-only changes. Shadow traffic compares old and new responses before you shift production load.

Shadow traffic to new handler: 0%

Compatibility verdict

ship

Backward compatible: v1 consumer reads nickname.

Producer: v1
Consumer min: v1
Payload: nickname
Shadow: 0%

Backward compatible · producer v1 · consumer min v1 · fields nickname. Producer v1 emits nickname only. Consumers on v1 parse successfully.

Expand before contract. Dual-publish optional fields until telemetry shows every consumer upgraded.

Contracts in CI. Consumer-driven pact tests fail builds when the producer removes a field still required downstream.

What actually happens when clients lag

The Tuesday story above is not a corner case; it is the default whenever you do not control every caller. A web front end redeploys the moment you push. A mobile app only changes when each user installs the update, and a meaningful slice of users take months to do that. Partner integrations and batch jobs are worse: nobody may be watching them at all. So at any moment your API is being called by several different generations of client, and the server has to satisfy all of them.

An added optional response field is compatible only when supported clients tolerate unknown fields. A strict generated decoder can still reject it; a new enum value can break an exhaustive switch; changing a default can alter behavior without changing the schema at all. Test the actual supported client behavior. Renaming fields, requiring a formerly optional input, narrowing accepted values, and changing error semantics require a deliberate compatibility or versioning plan.

A is what makes that CI check possible: each consumer publishes examples of the requests it sends and the responses it depends on, and the provider's build fails if a change would violate them. This matters because your OpenAPI document (the machine-readable description of the API) is not the real contract. Regenerating it after a rename produces tidy docs while the old binaries in the field still hard-code the old name. A therefore needs a new major version or a period where both shapes are served, never a silent schema tweak.

When you genuinely must change a shape, is the migration pattern that keeps every generation working throughout (the playbook section below walks through it), and paired with a tells clients and gateways the removal date in a way machines can read.

Compatibility rules: which bucket is this change in?

Assess compatibility in both directions: can the new server accept old requests, and can an old client interpret new responses? Relaxing accepted request values can be safe for old callers while adding response enum values can break old readers. A field kept under the same name but changed from milliseconds to seconds is a breaking semantic change even when both versions still use an integer. Documentation, runtime behavior, and deployment tests must agree on those meanings.

Before choosing a strategy, you need a fast way to tell whether a change is harmless or a break. Run every schema or endpoint change through this checklist; anything in the right-hand column needs the full playbook, not a quick deploy.

Add an optional response field; old clients skip keys they were never taught to read.
Add a new endpoint, or a new enum value (an option in a fixed list) that has a safe default.
Relax input validation carefully, because accepting more today is very hard to take back later.

Remove or rename a field; old binaries have the exact name hard-coded and will fail.
Change a field's type or meaning, such as cents to dollars, even when the shape looks identical.
Make an input required without a version bump; old clients never send it and get rejected.

How to choose an evolution strategy

Once you know which bucket a change is in, three questions decide how to ship it.

1. Can this be additive? An optional field or a new endpoint almost always beats a rename, because old clients ignore what they do not understand and nothing has to be coordinated.

2. Who consumes this API? Mobile apps and external partners need long windows because you cannot make them upgrade. Internal services can move faster, provided a shared proves nobody still depends on the old shape.

3. How will you know removal is safe? Measure real traffic on the old path, publish a date, and make provider CI fail on contract breaks before deploy. A dashboard that merely looks empty is not evidence.

Rule of thumb

Never break clients silently. Prefer , use only for breaking moves, and treat a public API as a long-lived product with contracts and metrics rather than a file you edit. In simple terms: expand before you contract, test with consumers, deprecate with evidence.

From problem to mechanism

Here is the Tuesday scenario again, slowed down so you can see exactly where the naive approach fails and what the fix changes.

Problem. Mobile, web, and batch consumers deploy on different clocks. Product wants to rename userName to user_name "because it's cleaner."

Naive system. Ship the rename on Tuesday and regenerate the docs. There are no to object, no period where both field names are served, and no metric showing how many callers still use the old one. Everyone assumes app-store updates happen fast.

Break it. Old builds still send and parse userName, so checkout returns 400s for users who never opened the store. Worse, rollback cannot save you: the data path was already "contracted" to the new name, so re-deploying yesterday's server would break the new clients instead.

Mechanism. Prefer evolution: add the new name alongside the old rather than replacing it. When a real migration is unavoidable, follow (next section) and pair with a and a long mobile window. Make provider CI fail while any active consumer still expects the old shape.

The rule the mechanism enforces is simple to state: at every moment during the migration, every supported client can read what the server sends, and the server can read what every supported client sends. The production deep dive below states this invariant formally and shows how to prove it holds at each step.

Safe migration playbook

The mechanism above names ; this is what each of its three steps looks like in practice. The running example is a v1 API whose price is a locale-formatted string, while new clients need a structured {amount_minor, currency} object instead.

1. Expand

price + price_v2

Publish price_v2 alongside the supported old representation and derive both from one canonical amount and currency. A locale-formatted string such as “1,234” is ambiguous without its locale, currency, and decimal rules, so shadow comparison must use known historical context rather than guessing. Exercise zero-decimal currencies, rounding rules, negative adjustments, and integer range limits before exposing the new representation to a client cohort.

2. Migrate

dual-read/write + cohort flags

Migrate client cohorts gradually while the server accepts both supported request shapes and emits responses each cohort understands. Define precedence when a caller sends both fields and reject inconsistent values rather than silently choosing one. Keep the canonical storage representation authoritative so dual-write logic cannot create two independent prices. Monitor validation failures and semantic comparisons by client version before expanding adoption.

3. Contract

remove when traffic reads zero

Remove the old contract only after its supported-client policy and evidence permit it. Servers can observe incoming old request fields, but cannot directly observe which response fields a client reads unless the client reports capabilities or version information. Combine traffic data with consumer contracts, partner inventories, mobile-version support windows, and an announced retirement date. Preserve a migration or explicit unsupported-version response for clients outside the supported window.

Numbers to know

Roughly 180 days: how long some phones take to update, so plan mobile windows in months.
Adoption gate: old-field traffic under 0.01% for 30 straight days, measured on the wire, not from download share.
Rollback boundary: keep dual output until the longest cache TTL plus the mobile window has passed.

One counterexample to keep handy when someone says "OpenAPI allows the rename": generated docs are not runtime clients. The field names hard-coded in old binaries are the real contract, and no spec file changes them.

Consumer-driven contract flow

A consumer test records the request it will make and the parts of the response its code relies on. The provider verifies those contracts against controlled provider states, such as an order that exists or an account without permission. Publish verification results keyed to provider and consumer versions so a deployment gate checks compatible combinations. Testing only the newest consumer contract can miss an older mobile or partner client still in production.

Contract tests complement schema checks and end-to-end tests. They catch agreed request/response interactions but do not automatically validate performance, authorization of every object, or a semantic change omitted from all examples. Include representative error responses, optional and absent fields, pagination, and idempotent retries where consumers rely on them.

The playbook only works if something stops a breaking deploy before it reaches production. That is the job of a , and the flow has four steps.

Each consumer records the requests it sends and the response fields it actually reads, using a contract-testing tool such as Pact or a diff of its OpenAPI expectations.
The provider's CI pulls every published consumer contract and replays it against the new build, checking real serialisation rather than the documented schema.
A fails that build, and the only ways past are a new major version or a dual-publish window where both shapes are served.
Mobile and web clients declare the minimum API version they support, and the server announces retirements with a so gateways and monitoring can see the date too.

Failure timeline, and what still breaks after the fix

Putting the pieces together, here is how the Tuesday rename plays out minute by minute, followed by the signals that would have caught it, and the ways an improved design can still go wrong.

Timeline: breaking API rename

Tue 10:00  Provider renames userName → user_name; deploys to 100%
Tue 10:05  Web OK (redeployed). Mobile v3.2 still sends userName
Tue 10:12  Checkout 400 rate +18% on iOS cohort last updated 40 days ago
Tue 10:20  Feature flag cannot restore field: schema already contracted
Tue 11:00  Hotfix re-adds userName alias; dual-publish restored
Wed        Consumer contracts added to CI; expand-contract playbook mandated
+180d      userName traffic below the adoption gate (see Numbers to know); Sunset honoured; field removed

What to watch

Error rate split by client and API version, so a lagging cohort shows up immediately.
Deprecated field usage, and whether every remaining caller maps to a known owner.
Contract verification failures and shadow mismatch rate, which block cutover before users notice.
Sunset dates approaching while material old traffic remains, meaning the migration is not done.

Still fails when…

A shape-compatible but semantic break, like cents becoming dollars, passes contracts that only check field types.
A new enum value crashes old clients with exhaustive switches; ship tolerant readers before the value.
One enterprise account still on the old path at 0.1% traffic; removal bricks a whale customer.

Trade-offs

Many live versions isolate breaks but multiply docs, tests, and security fixes to maintain.
Strict contracts catch real bugs, but asserting whole payloads locks in coupling nobody intended.
Per-cohort feature flags add operational work, while big-bang mobile migrations gamble the whole user base.

Independent deployment contracts

What production needs to hold

A production design needs a way to recognize trouble and a rule it preserves while recovering. A trigger is the measurement or event that starts a response. An invariant is a statement that must remain true, such as never confirming the same reservation to two customers.

Trigger metric or condition

Formalize compatibility when providers and consumers deploy independently, mobile versions remain in the field for months, schemas cross a durable log, or “small” changes repeatedly cause integration incidents.

Correctness invariant

During an expand-contract migration, every supported consumer can read provider output and the provider can interpret every supported request. Removal occurs only after observed use is zero for the policy window and rollback no longer requires the old path.

Contract ownership and safe migration

Consumer contract

owned expectations

Provider CI

schema + behavior verify

Expand + shadow

add, dual, compare

Adoption telemetry

version and field use

Sunset + contract

rollback window closed

Migrate a mobile price field without breaking old apps

API v1 returns price as a locale-formatted string. New clients need {amount_minor, currency}. Some mobile clients update only after 180 days. Provider adds price_v2, keeps price, and shadows serialization plus checkout calculations before any removal.

Adoption gate

old-field traffic < 0.01% for 30 days, with named enterprise exceptions = 0

Download share is insufficient. Use privacy-safe client version or capability reporting and supported consumer contracts; the server cannot infer which response field a client read merely from sending both fields. The example threshold still requires an explicit policy for any remaining old clients.

Shadow comparison

canonical minor units must agree across both serializers for supported currency and locale pairs

Compare both representations with the same canonical amount and currency. Parsing a display string requires its locale, separators, and rounding rules; an unqualified numeric parse can silently change the amount. Shadow validation exposes these semantic differences before the new format is served.

Rollback boundary

dual output retained through max cache TTL + mobile safety window

Rollback means route/read old behavior while both representations remain correct. After destructive contract, rollback requires a forward fix or restored data path.

Decision

The provider publishes a deprecation and Sunset timeline, contacts lagging owners, uses a client cohort flag for cutover, and removes v1 only after telemetry and contracts agree. Error codes and rounding semantics are tested alongside field shape.

How the mechanism holds

Follow how each part protects the rule above. The mechanism explains what the system does; the reasoning explains why that action is sufficient under the stated assumptions. If an assumption changes, the guarantee may change too.

Consumer contract ownership

Consumers publish minimal expectations with owner, environment, and supported version. Provider CI verifies them against real serialization and semantics; a broker records deployment compatibility.

Why it holds: The provider cannot deploy a breaking fixture while an active consumer contract exists.

Expand-contract

Add new optional shape, deploy tolerant readers, dual-write/serve, backfill if needed, switch cohorts, observe, then remove. The sequence is a state machine with rollback gates, not a single release.

Why it holds: N and N−1 providers/consumers interoperate at every intermediate state.

Deprecation telemetry

Measure API/client/schema version, endpoint, reported client capability, rejection reason, and known owner. Measure response-field use only where client instrumentation actually reports it. Send deprecation/Sunset headers where applicable and provide migration docs.

Why it holds: Every remaining old call maps to an owner or an explicit unsupported-client decision.

Shadow and rollback

Mirror safe requests or execute new code without side effects; compare normalized response and domain outcome. Keep old code/data until rollback SLO expires and avoid doubling mutations.

Why it holds: A feature flag returns cohorts to old behavior without data corruption or duplicate side effects.

Failure timeline: an enum grows

Provider adds PENDING_REVIEW

An old mobile client has an exhaustive switch that crashes on unknown values.

Response: Consumer contract fixture exposes the behavior; release is blocked or mapped to UNKNOWN.

Fix

Clients become tolerant

New clients preserve unknown values and render a safe fallback.

Response: Ship client first and wait for measured supported adoption.

Canary

New enum is served

A tiny cohort receives the value; crash-free and semantic metrics are watched.

Response: Flag disables generation immediately if old-client impact appears.

Sunset

Legacy behavior retires

Minimum supported version and telemetry window are satisfied.

Response: Record the decision, remove compatibility branch, and retain contract fixtures.

Compatibility matrix: the wire format changes the rules

Change	JSON	Protobuf	Avro
Add optional field	usually safe if unknown fields ignored	safe with new field number	safe only with reader default for backward compatibility
Remove field	breaking for readers that require it	reserve number/name; do not reuse	reader/writer resolution needs default or compatible union
Rename field	breaking on the wire	wire-safe if number stays; JSON mapping may break	breaking unless alias/resolution is planned
Change numeric type	semantic/parser risk	limited wire compatibility is not semantic safety	promotion rules are directional; test reader/writer pairs
Add enum symbol	old exhaustive clients may fail	unknown enum handling varies by generated client	reader needs compatible default/strategy

Backward means a new reader can consume old data; forward means an old reader can consume new data. Registries may enforce backward, forward, or full compatibility—state the direction and transitive policy.

How to choose among alternatives

Version endpoint vs evolve

Choose when: Evolve additively for compatible changes; use a new major contract for changed meaning or removal that cannot be bridged.

Caution: Too many live versions multiply security fixes, docs, and test combinations.

Provider vs consumer schema

Choose when: Provider schemas describe capability; consumer contracts protect only behavior actually relied upon.

Caution: Consumer contracts can ossify accidental coupling if they assert entire payloads.

Mirror vs shadow compute

Choose when: Mirror read-only/idempotent traffic; run local shadow logic for unsafe mutations.

Caution: Scrub credentials/PII and isolate shadow capacity so experiments cannot overload production.

Running it day to day

Measurements tell you what is happening; alerts tell an operator when action is needed. A rollout plan limits how many users encounter a change before it is checked. A runbook records the steps to diagnose and recover from a known failure.

Measure

traffic/errors by contract, client, schema, and endpoint version
deprecated endpoint/field use and owner coverage
shadow mismatch, contract verification, and rollback cohort health

Alert

provider change fails active consumer contract
unknown enum/field parse errors rise
sunset date approaches with material old use

Roll out safely

publish schema and compatibility direction
expand, verify contracts, shadow, and canary cohorts
measure adoption, announce sunset, contract after rollback window

Runbook

disable new response/behavior via flag
identify impacted contract and client versions
restore compatible output, repair semantic data, and add fixture before retry

Practice scenario

Evolve a payment API and its event from integer cents to a Money object while web, mobile, and batch consumers deploy independently.

Questions worth asking next

How do JSON, Protobuf, and Avro differ?
Who owns deprecation?
How is shadow traffic safe for payments?

A solid answer sketch

Add Money as optional while retaining cents, deploy tolerant readers, dual-populate from one canonical amount, shadow comparisons without side effects, and gate cohorts. Provider CI verifies owned consumer contracts. Measure old-field use and contract only after the mobile horizon and rollback window.

Common mistakes

• assuming schema compatibility proves semantic compatibility
• reusing a removed Protobuf field number
• removing based on app-store adoption rather than actual production use

Authoritative references

Protocol Buffers: Updating a Message Type Apache Avro schema resolution RFC 8594: The Sunset HTTP Header Field Pact contract testing documentation

Last reviewed: 6 August 2026

Choosing and explaining API compatibility

A complete explanation connects the mechanism to a requirement. Describe what improves when you introduce it, which new failure cases become possible, and how you would know that the trade-off is worthwhile.

Explain the core idea

Prefer additive JSON changes and explicit API versioning for breaking moves.

Contract tests catch provider regressions before deploy.

Mobile clients need long deprecation windows: measure adoption before removal.

When it belongs

Multiple client types consume one BFF or public API.

Independent deploy cadence between services and apps.

GraphQL/REST field deprecation policy question.

How it fails

Server deploy breaks old app versions in the field.

Error format change breaks client parsers.

Implicit coupling via undocumented response fields.

Trade-offs to weigh

Version proliferation vs single evolving API.

Strict schemas catch bugs; slow iteration.

Feature flags add ops burden vs big-bang migration risk.

Common misunderstandings

Renaming field because OpenAPI allows it: clients hardcode names.

Removing endpoint with 0.1% traffic: enterprise still on it.

Skipping contract tests for internal APIs that mobile BFF mirrors.

Wrap-up

You have seen the failure, the mechanism, and the operational signals. Use these exercises to check the ideas stuck, then follow the references for the primary sources.

Exercises

Integer cents → Money object. Web, mobile, and batch consumers deploy independently. Sketch the expand-contract steps and who owns deprecation, then compare your answer with the practice question in the production deep dive above.
Rename vs alias. Product insists on renaming a public JSON field. Argue for an additive alias first, and name the metric that would gate hard removal.
Consumer contract scope. A consumer asserts the entire payload in Pact. How does that slow evolution, and what minimal expectations should they publish instead?

Topic-specific references

Pact: consumer-driven contracts : the tooling behind the contract flow above; providers verify what dependents actually rely on.
Fowler: Parallel Change (expand-contract) : the original write-up of add, migrate, remove without a big-bang cutover.
RFC 8594: Sunset HTTP Header Field : the standard for machine-readable deprecation timelines.

Key takeaway

Treat a public API as a long-lived product. Expand before you contract, let consumer contracts fail the build before a break ships, and retire old shapes with metrics and a Sunset header rather than with surprise 400s.

Practice and further reading

Use these exercises to connect the lesson to a design decision. State your assumptions, trace what happens to one request, and explain the rule that must remain true even during a failure. That rule is the design's correctness invariant.

Focus for this lesson: Apply format-specific compatibility rules, expand-contract, consumer ownership, deprecation telemetry, shadow traffic, rollback, and long-lived client constraints.

Explain the decision

Why is adding a required response field usually safer than adding a required request field?

Read the reasoning

Tolerant readers may ignore new response fields, but old clients cannot supply a newly required request field; semantics still need compatibility review.

Apply it to a production scenario

A mobile client version from two years ago still sends an old enum. What does the server do?

Read one approach

Measure supported versions, parse unknown/legacy values safely, keep a translation path or return an explicit upgrade error, and never reinterpret silently.

Work through the numbers

A deprecated field still appears in 0.4% of 50 million calls/day. How many daily calls block removal?

Read the worked calculation

200,000 calls/day; identify consumer/version owners before contract removal.

Reason through a failure

What must a protobuf change preserve?

Read the recovery reasoning

Field numbers are never reused, old readers tolerate additions, enum/default semantics remain safe, and incompatible meaning gets a versioned field/message.

Sources and further study

Continue with Cloud Computing Models.