I have a small model that gets fine-tuned overnight on a box in the other room. The interesting part of that project has turned out not to be the training. It is the dataset. I spent last week almost entirely inside a 137-row file, and I found five separate problems in it.
Only after the fifth one did I notice they were all the same problem.
In every case I had a description of the data (a tag, a filename, a line in my own notes) and I had the actual rows. I had been checking the description. The description was wrong.
What the sleep cycle is
Some background on that project, since this is the first post that goes this far into its internals.
The model is open weights, about four billion parameters, small enough to load into local memory and answer in a second or two. That size is the point. Every night it gets fine-tuned on material from that day's real work: facts, corrections, lessons, whatever the day actually produced. The loop is called the sleep cycle.
Turning a day into training material means writing pairs, a question and the answer I want back. Every pair goes into a review queue, and I read all of them and approve or reject each one by hand. Nothing reaches the weights that I have not said yes to.
That gate exists because of how this kind of training fails. It does not crash. It produces a model that states something slightly wrong, confidently, on every future run, and the cheapest place to catch that is before it is baked in. Unlearning a fact costs more than the night that taught it.
After training, the new model gets scored against a held out set it has never been shown. If it does not clear the bar the night is rejected, and the previous model stays deployed. Nights get rejected. The gate firing is a normal outcome, not a setback.
All five of the problems below are dataset problems, which is to say they are problems in what I chose to teach.
The tag counted half of what it claimed to
Part of the training set teaches the model to correct a wrong premise instead of going along with it. Those rows get annotated (disambiguation) when I write them, and I had been describing the previous night's set as "16 of 128 negation-framed, about 12.5 percent." That number came from counting the annotation.
So I counted the rows themselves instead, with a regex against the first few words of every answer:
NEG = re.compile(r"^\s*(no\b|not\b|nope|that('s| is) not|that isn't|neither\b|never\b)", re.I)
Twenty-four answers actually open with a negation. The union of the tagged and the untagged is 29 out of 128, which is 22.7 percent, not 12.5. Three facts had negation-opening answers carrying no tag at all.
That gap matters because the whole point of tracking the number is to keep the model from learning "start by disagreeing" as a general habit. I had been managing a proportion that was off by nearly half. The new draft caps it at one per fact and audits the answer text, never the annotation.
The rows were right and the shape was wrong
One of the facts covers a twenty year career in banking. Eleven rows taught it, all of them phrased in the past tense, two of them phrased as corrections.
The held out question that fact exists to answer is "I met someone who also worked in banking, what's my story?" The model came back with:
"Your story is no longer yours to tell."
Every fact in those eleven rows was correct. Nothing was misspelled, nothing contradicted anything. But eleven consistent past tense sentences do not teach a biography, they teach a mood. The model learned the shape "that is over" and never picked up the content.
The rewrite asserts the background as something present and recallable, and keeps exactly one row that handles the past tense case. Same facts, different grammar, and grammar turned out to be what was actually being learned. A row-level review would never have caught this. It only shows up when you look at eleven rows together and ask what they have in common besides their content.
The neighboring version of this: a relationship taught from one side does not bind. Every row describing how two components relate had the same component as the grammatical subject, and in live use the roles came back inverted. Both directions now get taught explicitly, one entity per question.
My fix contaminated the thing measuring the fix
There is a held out set that never appears in training. That is the only honest read I get on whether any of this works.
While rewriting rows I reworded one question about a website. Routine cleanup. Then I ran an embedding check across the whole draft, comparing each held out question against its nearest neighbor in the old file versus the new one, and that item had moved from a cosine similarity of 0.6632 to 0.7481.
I had leaked the test into the training set while fixing the training set. Not by copying anything, just by drifting a phrase close enough to matter. Swapping the example to a different site put it back to 0.6632 exactly.
The check is about thirty lines and runs in under a minute. It now runs on every edit to the dataset, not on some milestone. If you maintain a held out set and only verify separation when you build it, you are trusting that every future edit was harmless.
The file I staged was never read
This one is the plainest and the one I am most annoyed about.
The training mix includes a replay file that keeps the model anchored to its prior behavior. Before the last run I had built a cleaned version of that file, 411 rows with a formatting problem stripped out, and staged it under a clear name.
The pipeline script had the path hardcoded:
REPLAY=data/${NIGHT}-replay.jsonl
It read the old file. The cleaned copy sat on disk, untouched, while the run trained on 32 rows carrying exactly the characters I had removed. Nothing errored. Nothing logged a path. The run looked completely normal, and my notes said it had used the clean file, because I had put the clean file there.
Two changes. The variable takes an override (REPLAY=${REPLAY:-data/${NIGHT}-replay.jsonl}), and every run now prints the resolved path, the row count, and the sha256 of what it actually opened, marked either (canonical) or (OVERRIDE via $REPLAY env). There is also a DRYRUN=1 mode that resolves every path, prints them, and exits without running a stage.
Staging a file is not the same as using a file. A pipeline that never says which file it opened will let you believe anything you want about what it read.
The rule I came away with
Audit the rows, not the record of the rows.
Annotations, filenames, and the notes I wrote at the time are all summaries, and a summary is written once and then trusted forever while the thing it summarizes keeps changing underneath it. Every one of these five bugs lived in that gap, and not one of them threw an error. They all produced clean runs and plausible numbers.
The cheap defense is to make the pipeline say out loud what it actually touched: the path, the count, the hash. The expensive one is to keep re-deriving your stats from the content instead of from your own labels, every time, even when you are certain nothing moved.
Something moved. It usually has.