Short technical notes on LLM inference and agent harness design. Each one works out the maths, states its assumptions, and tells you when the technique is not worth building — or the price not worth paying.
Run the same block of layers again and again. Build one, watch it reason hop by hop, then price it: a 3.5B looped model stores like 3.5B and runs like 53B.
Read → System One modelsJev, and the three open models chasing it, answer typed questions without writing a sentence. Race them, then train one in the page and watch it overfit.
Read → Inference pricingTypeSafe AI sells its decision model at a flat rate per input token. That price bounds how big the model can be, and stops covering cost on long documents.
Read → Inference enginesA small model guesses the next few tokens and the big one checks them in a single pass. 2–3× faster generation, with provably identical output.
Read → Speculative tool calling · Part 1When an agent writes code that calls tools, you can start the calls before generation finishes. Streaming overlap caps at 2×; running the calls concurrently does not.
Read → Speculative tool calling · Part 2Part 1 as a calculator. Give it your token rate and tool latency and it says which half, if either, is worth building.
Read →