Git Worktrees

The isolation primitive every later parallel term depends on. Term 11 gave each worker its own context; this gives each worker its own filesystem, and the second one is where the surprises live.

The failure it fixes

Extraction over thirty issuers is embarrassingly parallel, so the obvious move is N processes in one checkout. That works, right up until two of them want the same file.

What makes it dangerous is that it does not fail loudly. Two processes writing one scratch file do not deadlock or crash; the second write lands on top of the first and both processes carry on. The run that results has issuer A's extraction scored against issuer B's intermediate state. It completes. It emits a number. The number has a page citation attached, because the citation was copied from whichever record happened to be in the file at the time.

The reason this is worth a full unit rather than a warning is that the single-worker version is correct. Every path resolution against the repository root works perfectly with one agent, because with one agent the repository root and the working directory are the same place. The bug is latent in code that has been running fine for months, and it activates the day you add concurrency.

Break it first

Run the demo and watch run B:

$ npx tsx scripts/terms/demo-13.ts
creating 4 worktrees under .rr-worktrees
git sees 5 worktrees (1 main + 4)

--- run A: every worker writes only inside its own worktree
  escapes detected: 0
--- run B: one worker resolves a path against the repo root
  escapes detected: 1
  artifacts  <- receipts are written per run and would overwrite each other

The offending line in run B is not exotic:

const REPO_ROOT = ROOT;
fs.writeFileSync(path.join(REPO_ROOT, "artifacts", "shared-scratch.json"), ...);

That is how it gets written every time. Delete three of the four worktrees and it becomes correct again, which is why it survives review.

The mechanism

git worktree add gives a second working directory over the same object store. History and objects are shared, so it is cheap. The index and the files are not, so it is isolated.

src/worktree.ts wraps creation and teardown, and adds the part that actually bites: SHARED_PATHS, an explicit list of things that are common across every worktree, and detectEscapes, which diffs a before and after fingerprint of them. .git/index, artifacts/, corpus/ and the build cache are on the list, each with a recorded reason.

Build it

The commit is the module plus a demo that creates four real worktrees, runs both scenarios, and tears them down in a finally so a failure does not leave five directories behind.

The receipt

artifacts/13-worktrees.json, with both runs and the escape.

Seven tests, two of which are negative controls. Gutting detectEscapes to return [] fails exactly those two and leaves the other five passing:

x detectEscapes > fires when a shared path changes
x detectEscapes > reports one finding per shared path, not one per worker
Tests  2 failed | 5 passed (7)
demo:  FAIL: run B should have been detected as an escape

Where it overlaps

Line H is isolation and this is its filesystem rung; term 11 was its context rung. Line C is delegation topology: you cannot fan out safely until the workers cannot see each other's mess. Line D is execution surface, because a worktree is a different place for work to happen, and that is the same question term 18 asks about a run with no terminal attached.

Term 15, next in build order, puts a hook in front of exactly this. That pairing is interstitial II, and this unit is the half that makes it concrete.

Sharp edges

• Findings carry no worker attribution, deliberately. The first version stamped a worktree id on every finding, so one escaping write was reported four times naming four culprits, three of them invented. A snapshot diff of a shared path proves an escape happened and cannot prove who did it. Real attribution needs per-process tracing, which is a much larger option and is named in the module rather than faked.
• git worktree remove fails if the directory is dirty. remove() falls back to rm -rf plus git worktree prune, because a demo that leaves five worktrees behind on failure is worse than one that force-removes.
• Worktrees live outside the repo. .rr-worktrees is a sibling directory, not a subdirectory, so they never appear in the repo's own git status and cannot be committed by accident.
• Shared object store means shared branches. Two worktrees cannot check out the same branch. create() makes a unique rr/run-<id> branch per worktree for that reason, and deletes it on teardown.