Course · Term 30 of 30
Multi-Repo Orchestration
Every lesson in this course builds one system: a stablecoin reserve attestation registry that discovers issuers, reads the PDFs they publish, and refuses to publish a field it could not read. What it is and where this lesson sits.
Verdict: known. Landing one change across several repositories has been a
problem since there were several repositories. Android's repo tool is from
2008; git submodules are older; every large shop has an internal script called
something like land-everywhere.sh, and every one of them has the same bug.
The bug is that it is not atomic and cannot be made atomic. There is no transaction spanning four remotes. What you can do is make the inconsistent window short, ordered, and named — and the third is the part the scripts skip.
What agents add is not a technique. It is that the fan-out is now cheap, so you do it more often and across more repositories, which makes the window wider and the credential surface larger.
The receipt
npx tsx scripts/terms/demo-30.ts
Four repositories, one change, api fails to land:
schema landed merged
api failed the change did not land
worker landed merged
web blocked depends on api, which did not land
the progress bar version: "2 of 4 landed, 1 retrying".
the state of the world:
schema has the change, api does not
consistent: false
Those are two different reports of the same event and only the second is
useful at 3am. "2 of 4 landed" says the job is in progress. "schema has the
change and api does not" says what is currently true of production.
The other half is what is not an inconsistency:
schema fails instead:
landed 0, blocked 3, inconsistent pairs 0
Nothing landed, so nothing disagrees.
A failure at the root is the cheap one — everything is still coherent, just unchanged. Reporting it identically to a mid-sequence failure throws that away.
The three mechanisms
Order. order() topologically sorts the set. A dependency lands first
because a dependency carrying an unused capability is harmless, while a
dependent whose dependency is missing is broken. A cycle has no harmless
direction, so order() throws rather than picking one:
cannot order a landing: a -> b -> a is a cycle. No sequence lands these
consistently.
Scope. Each repo's worker gets its own credential (term 5). The temptation is one token with access to everything, because it is one secret to manage instead of N — and it also means any worker that goes wrong goes wrong everywhere:
worker for schema presented credential 'token:org-wide', which is not scoped to it
Accounting. consistent is computed from whether every repo landed,
never from whether any did. inconsistentPairs names the disagreements.
Break it first
mutation tests demo
consistency inferred from the landed count 1 failed | 12 passed exit 1
a cycle is given an arbitrary order 1 failed | 12 passed exit 1
dependents of a failure are tried anyway 1 failed | 12 passed exit 0
any credential is accepted 2 failed | 11 passed exit 1
every unlanded repo counts as inconsistent 1 failed | 12 passed exit 1
"Dependents of a failure are tried anyway" is the one the demo misses, and it
is the one a hurried script does: push everywhere, collect the failures,
retry. It works, mostly, and it means web gets pushed against an api that
does not have the change — so the failure count is higher, the reasons are
misleading, and the inconsistency is wider than it needed to be.
Where it overlaps
Line H is isolation, and each repo is a worktree with its own scope (term 13). Line C is delegation topology — this is term 22's orchestrator-workers with each worker holding a different remote, so the merge step is a landing and the completeness record becomes a consistency record. Line E is the trust boundary: N credentials, N blast radii, and the decision about how they are scoped is the one that matters after something goes wrong.
Sharp edges
-
No network and no remotes here. Landing is a supplied predicate, so this exercises the ordering, the blocking and the accounting — not the mechanics of pushing to four hosts, which is where the interesting operational failures actually live.
-
A rollback is another landing. Undoing a partial change set is the same problem in reverse, with the same non-atomicity, and it usually gets less design attention than the forward path.
-
The dependency graph is a claim about the code.
dependsOnis hand-written here and hand-written in most real setups too. When it is wrong the ordering is wrong, and the report will state its incorrect conclusion with total confidence.
A personal teaching project, in development. The registry it builds is a teaching artifact and is not an assurance opinion about any issuer. Not affiliated with or endorsed by any employer, or by any vendor whose tools it describes. Where a unit depends on a specific flag or path, it names the version it was verified against. All units.