It’s Day 3 of building a reasoning model. Let’s code a text generation function, and optimize for performance during inference today ✨ This one change made my model 13x faster — and I only changed 2 lines of code. Every time an LLM generates a new word, it re-feeds the entire conversation back through the model — even though it already computed attention for every earlier word on the last step. The fix: a KV cache. Store what you already computed, only process the new token. 5 tok/sec → 29 tok/sec with caching. → 68 tok/sec with torch.compile stacked on top. Reasoning models generate hundreds of candidate chains of thought — every token saved here gets multiplied hundreds of times over. Save this if you’re building your own LLM 🔖 Day 4: teaching the model to actually reason → #buildareasoningmodel #llmfromscratch #kvcache #machinelearning #AIengineering