The Inference Stack in 2026.
A field note on token economics, runtime systems, model architecture, and the stack changes behind public LLM API price compression.
Research Program / emerging
Distributed Runtimes
Runtime systems, state, scheduling, observability, and reliability for AI workloads.
Questions
- How should AI runtimes expose state and backpressure?
- Which scheduling decisions matter for cost and latency?
- How do distributed failures appear at the model interface?
Topic Links
Linked Artifacts
Speculative Decoding
A decoding-time acceleration technique that drafts tokens with a smaller model and verifies them with a larger model.