FIG 5 Β· ATTENTIONrunning
latest03 oct 268 min#ai#llm
How a transformer reads
A walk through the parts of a transformer, from tokens to sampling, with a working figure for each step.
read it βA walk through the parts of a transformer, from tokens to sampling, with a working figure for each step.
read it βMistral Large 4 is the best model built outside the US and China and a big jump from Large 3. Independent tests put it about seven months behind the US labs and behind five Chinese open models, and a task on it costs about fifteen times more than on the small US models with the same score.
Lionel Messi made football look like art, and no spreadsheet can hold that. But the spreadsheet still says it, plainly. The best forward who ever played and the best playmaker who ever played are the same person.
Lean is a programming language in which a theorem is a type and a proof is a program, checked by a small kernel. This post covers how that works, where it can still go wrong, and why it lets language models write proofs nobody has to read.
On 6 October OpenAI published 722 machine-written math papers with claims on the Unique Games Conjecture, the colouring of the plane, a $5,000 ErdΕs problem and dozens of other open problems. This post goes through the claims, the parts that come with Lean proofs, and the first day of reactions.
DeepSeek V4.1 and GLM-5.3 compared on agentic work and coding in two weight classes, next to the closed models behind Claude Code and Codex, with benchmark scores by task type, speed, cost per solved task and reports from daily use.
A walk through the parts of a transformer, from tokens to sampling, with a working figure for each step.
A hard DP that's worth knowing, because it stacks three classics in one problem. Partition DP, prefix sums and an LCP table take it from O(nΒ³) to O(nΒ²).
What actually happens when a provider says your prompt is cached. The trick underneath is called KV caching, and it's a small idea with a big memory bill.
Agents loop, and every loop resends everything. Where the tokens actually go, how prefix caching claws most of it back, which benchmarks matter when you pick a model, and a router you write yourself.
What I check first when an Elasticsearch cluster gets slow. Shards, segments, refreshes, and the profiling result that changed how we fetch ids.
The site, twice. A one-font page I shipped in april, then a rebuild with figures you can poke, a small green critter and a status bar.
A thinking canvas. Notes, mini-apps and agents on one infinite board, and the AI reads the board before it acts.
A terminal for agentic coding. Tabs, splits and real shells, plus the panels that show what your agents actually did.
An agent that turns a written scenario into a product demo video: it drives a real browser, moves like a person, and cuts the footage.