Jaggedness: what RL optimises, and what stalls
Karpathy's argument that models are advancing only where a reward can be computed, with a joke that hasn't changed in three or four years as the demonstration — and why that is the same boundary the harness cluster keeps running into from the other side.
From the NoPriors interview. The observation first: working with a current agent feels like talking to a brilliant systems programmer and a ten-year-old at the same time. His point is not that it is uneven — people are uneven — but that the combination is one you would never meet in a person. Human ability is far more correlated across domains than this.
The joke test
His demonstration is deliberately petty. Ask a frontier model for a joke and you get the one about scientists not trusting atoms, because atoms make everything up. It is the joke you would have got three or four years ago. Meanwhile the same model will work for hours on an agentic task and move mountains.
The joke has not improved because nothing is optimising it. Coding, maths and anything else with a checkable answer sit inside the reinforcement-learning loop; humour sits outside it, so it is frozen at whatever the pre-training distribution produced.
Asked directly whether competence at code generalises to everything else, he says no — maybe a little, "not a satisfying amount". This is worth recording plainly because the opposite is a common background assumption: that capability bought in verifiable domains arrives free everywhere else as a side effect. His answer is that it does not, and that the gains are clustered inside opaque models where you cannot see which parts got the attention.
Where the ridge falls
The formulation he uses: you are either on rails, inside the verifiable domains, where everything moves at the speed of light — or you are off them, and things meander.
What he names as off-rails and consequently weak:
- Working out what you actually meant. Nuance about intent.
- Knowing when to ask a clarifying question. He calls this out specifically, which is interesting, because it is the behaviour that would most reduce wasted work.
- Anything he describes as "softer".
Both interviewers volunteer the same complaint from the user side: the frustration is not that the agent fails, it is that it burns a lot of compute on something it should have recognised as an obvious dead end.
Why this is the same wall the harness cluster keeps hitting
Karpathy's constraint on Auto Research: taking the researcher out of the loop is that if you cannot evaluate it you cannot automate it. Jaggedness is that same line drawn from the training side rather than the harness side: the reason your loop only works on verifiable tasks is that a verifiable reward is also the only thing that made the model good at them.
That makes several scattered findings in this notebook one finding:
- The model is a poor QA agent out of the box — Anthropic's observation, and the reason the evaluator needed several rounds of prompt surgery before it graded sensibly. Judging taste is off-rails work. Nothing trained it.
- Concrete criteria beat quality questions. The owner's rule from The harness is a skill issue — name the colour scheme, check the spacing, check the buttons are clickable — is not just good rubric hygiene. It is a way of dragging a judgment onto the rails, where the model is competent.
- The evaluator that drives the artifact beats the one that reads a transcript. Same move again: replace a soft judgment with an observation.
/goal's tool-free evaluator (Keeping an agent running: goals, loops, hooks and schedules) is cheap precisely because it stays inside the verifiable slice — a test exit code in the transcript — and useless outside it.
The practical reading: when you are designing a loop, the question "can this be checked mechanically?" is not a nice-to-have constraint you might engineer around later. It is a prediction of whether the model is any good at the task at all.
The same ridge, seen from the buyer's chair
Four months later, DoorDash describes the pattern without naming it, from the position of a company paying for the models rather than reasoning about them. On non-coding work — accounting, analytics — the sequence they report is: the team says "yeah, it works okay"; the data gets scrubbed and handed to a lab, an RL environment is built around the task, and the models crush it; the same thing pointed at real enterprise data does not hold.
Step two is the verifiability operation. Building an RL environment around a task is dragging it onto the rails, and the gap between step two and step three measures how much of the real task the rails did not cover. Their own open question — harness gap, or something missing from the models' data distribution — is Karpathy's unresolved part restated by someone with a procurement decision attached to the answer. Followed through in Benchmarking your own agent spend.
Someone who expects the transfer to hold
The first direct contradiction of this page in the notebook, and it is worth stating as a disagreement rather than smoothing into a synthesis. Greenblatt is asked more or less Karpathy's question — does capability bought inside the rails arrive outside them — and answers the opposite way: in practice the transfer "seems okay". His evidence is the absence of a counter-example. He says it is hard to point to a domain that is genuinely hard to verify where the improvement from GPT-4 to a current frontier model has not been large.
Both of them are describing the same models, so the disagreement has to be about what counts. Karpathy's instrument is a ceiling: the atoms joke is unchanged, which shows a capability that has not moved at all. Greenblatt's is a slope: hard-to-verify work is much better than it was, even where it is still worse than a competent professional — a distinction he makes explicitly and does not try to hide behind. Those are compatible descriptions of one situation, and they license opposite forecasts, which is why the crux matters more than the observation.
Two things sharpen it rather than resolve it:
- His mechanism is not "it comes along for free." He expects transfer because you can train learning on the fly — many environments demanding adaptation under limited resources — and then spend the resulting skill outside the rails. That is a claim that the rails can be laid under a more general capability than any specific task, which is a different bet from the one this page rejects, and it is untested.
- He agrees about where the difficulty is, and disagrees about what it's made of. Asked for the least verifiable part of AI research, he says making calls on large experiments — few tries, no iteration — and what he expects models to lack is taste about in-the-weeds experiments rather than insight. That is the same ridge this page draws, with the far side described as thin rather than as another country. See Containerising AI research.
The version of the disagreement that could actually be settled is on this page already, in the section above: DoorDash builds an RL environment around a task, the models crush it, and the real data does not hold. Greenblatt's answer to that specific pattern is that environments built from production data close the gap — which is a prediction, about a thing several companies are now doing, that will be visible either way.
The unresolved part
He does not claim to know whether the remaining roughness is a model limitation or an operator one, and says so. That hedge is discussed in the interview note; it applies here more than anywhere else in the transcript, because jaggedness is the one part of his account that is an argument about the models rather than about how you drive them.
There is also a precedent for the observation that is older than the vocabulary. Efros and Torralba said in 2011 that computer vision was optimising a definition of success it had chosen itself, and the field's best model at the time had been fitted to its benchmark's quirks one false positive at a time — see Optimising for the benchmark. What that episode adds is a reason not to read this page as a claim about a permanent shape: the ridge moved, and what moved it was a harder measure rather than a better model.
Two things follow that the interview does not address. If capability is concentrated where rewards exist, then a benchmark suite is a map of what was optimised, not of what the model can do — and the gaps are unmeasured by construction. And the same argument predicts that jaggedness gets worse as the verifiable domains advance, not better, which is the opposite of what "the models keep improving" implies. Neither is tested in the transcript. The first one has since acquired a worked example: the scrubbed task that passes and the real one that does not is a benchmark reporting on the version of the job it could grade.
Linked from
- Agentic engineering: the work moves to the harnessThe emerging discipline around long-running coding agents — designing the scaffolding, feedback loops and environments that let an agent do reliable work, rather than writing the code yourself. Entry point for the harness-design cluster in this notebook.
- Auto Research: taking the researcher out of the loopKarpathy's name for an agent loop that improves a model without a human in it — an objective, a metric, boundaries, and go. What it did to a repo he had hand-tuned for two decades, the one constraint that decides where it applies, and the parallel version he has not solved.
- Benchmarking your own agent spendDoorDash's model spend rose about 20× in five months, so they built a benchmark over their own coding tasks to work out what it bought — and found that the models crush a scrubbed version of a task and then underperform on the real data. The first buyer-side view in this notebook, and the open question it leaves.
- DoorDash on agentic commerce and DOT (NoPriors)Notes on the NoPriors interview with DoorDash's two co-founders — a natural-language ordering surface that half the time sends people to a merchant they have never used, a delivery robot built to a use case rather than to a capability, and an internal benchmark built to work out what a 20× rise in model spend actually bought.
- Generator–evaluator loopsSplitting the agent that does the work from the agent that judges it — why self-evaluation fails, what makes an evaluator worth its cost, and how three independent teams arrived at the same shape from taste, from QA, and from production corrections.
- Learning from deploymentProduction traffic is becoming training data — tasks the model did badly on, turned into environments that match them exactly, with the rubric built from what the human actually wanted. Which is the loop this notebook already admires, run with the model as the artefact, and it changes where the reinforcement comes from.
- Optimising for the benchmarkComputer vision spent the 2000s carving its best models around the quirks of the datasets it was scored on, and hard negative mining was how it did it — training against your own error cases, one gradient step at a time. The episode is the oldest worked example in this notebook of a measure being optimised until it stops measuring.
- Reading notes: Sutskever's List (Heimann), ch. 1–2Richard Heimann's book reads the reconstructed reading list Ilya Sutskever gave John Carmack as an argument rather than a bibliography. Chapters 1 and 2 give the four-part worldview it claims to encode, and then spend a chapter showing that AlexNet invented almost nothing — which is the part this notebook cares about.
- Separating drafting from judgingSeven writers independently arrive at the same move this notebook's harness material arrives at — never let the thing that produces the work also judge it while it is being produced. The structure matches; the reason for it is the exact opposite, and that asymmetry is the interesting part.
- Skill Issue: Karpathy on code agents (NoPriors, 2026-03)Notes on the NoPriors interview where Karpathy says he hasn't typed a line of code since December, that every remaining failure feels like a skill issue rather than a capability ceiling, and that token throughput is the resource you should feel nervous about wasting. Plus the jaggedness caveat that undercuts the whole picture.
- The case for recursive self-improvement (Dwarkesh × Greenblatt, 2026-08)Ryan Greenblatt argues that AI research is verifiable enough to automate itself, that automating it buys four or five years of progress in one, and that what comes out the other end can transform the world without ever learning to play politics. His median for full automation is 2030–31 and his odds on takeover by 2040 are 35–40%.
- The claw layer: an agent that persists when you close the lidKarpathy's name for the layer above a coding session — something that keeps looping in its own sandbox, remembers more than a compacted context, and answers on one messaging channel. The home-automation example he built, and why it is a partial answer to two open questions in this notebook.
- The last hundred feet: building toward a use caseDoorDash's argument for why they had to build their own robot — the field builds a capability and then hunts for a problem — plus the edge cases nobody writes down at a desk, and the one incumbent data advantage in the interview that is actually load-bearing.
- Too dangerous to release: the GPT-2 precedentOpenAI withheld GPT-2 in February 2019 on the grounds of risks that a later post admitted had not materialised, and the justification pointed at future systems rather than at the model being withheld. The argument structure — act early because acting late is worse — is the one that later produced the board crisis, and it is unresolved in exactly the same way today.