Back to Projects

The Fixes That Lived on One Machine

Alex Wilson6 min readBuilder Journal
Rear of a small computer, one cable connected, one unplugged in shadow

The Mac Mini in my office is not allowed to commit code.

That is a deliberate rule, and a fairly new one. On August 2nd a commit authored on that machine went upstream and stalled a deploy, so I drew a hard line: the Mini is a deploy target, not a workstation. It pulls with a read-only key. It runs the services. It never writes to history. Every review, every commit, every push happens on my Studio, where I can actually look at the diff before it ships.

I still believe that rule is correct. What I did not think through is what happens to work that gets done down there anyway.

Two fixes, running in production, existing nowhere else

On August 9th I was deploying an unrelated feature and ran git status on the Mini out of habit. The working tree was not clean. There were changes sitting in it that I had never seen, spread across nine files, and every one of them was a real fix that was already running.

The first one taught the publishing pipeline how to file a post. Articles get written on that machine and pushed into HeadstringWeb, and until this change they went up with no category and no page assignment. The fix added category: and page: support in the draft frontmatter plus per-site defaults, then resolved those names against the database with lookups that throw loudly instead of silently writing a null. Nothing exotic. Just the difference between a post that lands where it belongs and a post that lands nowhere in particular.

The second one was a single flag, and it was the one that stopped me:

npx tsx --tsconfig apps/web/tsconfig.json apps/web/scripts/publish-draft.ts

That script imports things through a @/lib path alias, which lives in the tsconfig. tsx finds a tsconfig by searching upward from the current working directory, and there is no tsconfig at ~/headstring or at ~/headstring/apps/claudia. My scheduled jobs all run from apps/claudia. So the alias resolves to nothing, the import fails, and the publish step dies at the last possible moment, after the article is written and the image is generated.

That flag now appears in the worker prompt, in the editor delegation string, in the seed scripts, and in the docs. Four scheduled article jobs depend on it. Without it, every one of them writes a piece and then fails to ship it.

Both fixes were good. Both were correct. Both existed in exactly one place: an uncommitted working tree, on one machine, in my office, with no backup and no history.

This was the second time

The part that actually bothers me is that I had already been bitten by this, and had not recognized it as a pattern.

Back in July I found that all four of my sites were publishing under the same byline. They should not: one site posts as Alex Wilson, one as Alexzaviar, one as Claudia. I fixed it, verified it, and moved on. Weeks later it was broken again, identical to before, as if the fix had never happened.

It had happened. It just happened as an uncommitted working-copy change in the old standalone repo, and when I imported that repo into the monorepo I brought the history and left the working tree behind. The fix was never in a commit, so from git's point of view it never existed. I redid it on July 29th, and this time it went in properly.

Same failure mode, twice, on the same machine, and I still filed the first one under "weird regression" instead of "the valve only goes one way."

The valve is right. The gauge is missing

Here is the structural version of the problem, and I think it generalizes past my setup.

A read-only deploy target is a one-way valve, and that is a good design. Code flows down. Nothing flows back. You get a clean chain of custody and you never wake up to a commit you did not write.

But the machine at the bottom of that valve is the only one that experiences production. It has the real cron schedule, the real working directory, the real path resolution, the real filesystem. Some bugs are only visible there. Which means some fixes can only be written there, on the one machine specifically forbidden from telling the repo about them.

I want to be fair about where the failure sat, because it was not the agent doing the work. Claudia followed the policy exactly. The policy says do not commit, leave changes in the working tree, and report them at the end of the job so I can review them upstream. She did all three. The report went into a job result that I read once, on a Tuesday, in the middle of something else, and then never opened again.

A reporting step that depends on a human remembering a message is not a mechanism. It is a hope.

What I actually changed

I read every hunk, kept them all verbatim, and committed them from the Studio with a message that says plainly where they came from. That part was easy. The changes were good, which is exactly why losing them would have been quiet instead of loud.

The real fix is a gauge on the valve. Something that runs on a schedule, checks whether the deploy box has a dirty working tree, and puts it somewhere I cannot skim past. Not an auto-commit, and not a nag. Just a standing answer to the question "is there work down there that only exists down there."

Because there is, right now. As I write this, git status on the Mini shows four untracked scripts, a tarball, and an entire application directory. Some of that is scratch work that should be deleted. Some of it is diagnostic tooling I would genuinely want to keep. I only know that because I went and looked, which is the whole problem.

If you have a box your repo cannot write back to, go run git status on it today. I would bet something is sitting there. The one-way valve is the right call. It just needs to tell you what is piling up behind it.

Share: