Recent entries
A Latent Space writeup of early access to OpenAI's GPT-6 Astra, framed around the claim that it is cheap enough and capable enough to function as a junior AI Engineer — running 20-50 subagents in parallel, monitoring its own multi-day jobs, and building its own benchmarks. Read as a vendor demo rather than a measurement, and cross-checked against what the notebook already has on spend, throughput and unattended runs.
Domain-driven design's shared vocabulary, generated from a codebase and handed to a model as a working artefact. The reported effect is the odd part — not just better plans but shorter thinking — and it is cheap enough to test here.
A conference talk arguing that the spec-to-code loop degrades a codebase on every pass, that the reason is ordinary software entropy, and that five failure modes of AI coding each have a fix sitting in a twenty-year-old book. The first source in this notebook that locates the harness inside the code rather than around it.
Ousterhout's deep modules, repurposed as an agent-legibility property and as the line between what a human designs and what an agent is allowed to write unread. Why the boundary is the unit of delegation, and the one place this collides with the notebook's other evidence.
Planning note for a 50-minute talk in Prague — the business case for specialized small models, then the training harness live. Includes the evidence check that killed the original opening slogan, the six conditions under which fine-tuning is the right call, and the caveat about borrowing a frontier lab's factory as a template.
Why video generation is stuck at five-second clips — quadratic attention over 35,000 tokens for five seconds of 480p — and why the open/closed gap that has nearly closed for text is night-and-day for video. Plus the argument that text diffusion is being marketed as the wrong product.
Philip Kiely and Ali Taha on what actually happens between an open-weight checkpoint and a production endpoint — the four optimisations that stack to 2–4× on fixed hardware, why quantising more of a model can make it better, and the reliability failures that turn out to be a race condition in a kernel rather than anything about the weights.
Inference engineers now write training pipelines, RL rollouts bottleneck on inference speed, and Baseten reports running GPU kernels inside their inference engine that GLM-5.2 wrote for itself from its own profiling traces. Plus the argument that continual learning belongs in the KV cache rather than in the weights.
Before AlexNet the field's best argument against neural networks was that the data was doing the lifting — and Efros had the control experiment to prove it. AlexNet settled that case on the same data everyone else had. The reason to keep the episode is that the modern version of the argument is running now, with the control missing.
Computer vision spent the 2000s carving its best models around the quirks of the datasets it was scored on, and hard negative mining was how it did it — training against your own error cases, one gradient step at a time. The episode is the oldest worked example in this notebook of a measure being optimised until it stops measuring.
Richard Heimann's book reads the reconstructed reading list Ilya Sutskever gave John Carmack as an argument rather than a bibliography. Chapters 1 and 2 give the four-part worldview it claims to encode, and then spend a chapter showing that AlexNet invented almost nothing — which is the part this notebook cares about.
OpenAI withheld GPT-2 in February 2019 on the grounds of risks that a later post admitted had not materialised, and the justification pointed at future systems rather than at the model being withheld. The argument structure — act early because acting late is worse — is the one that later produced the board crisis, and it is unresolved in exactly the same way today.