# @awomanindatascience on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/DdxhfzCziGc
- **Gondola URL:** https://gondola.cc/posts/70951318-awomanindatascience-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/911b0ef276.jpg
- **Posted:** 2026-09-27T02:22:20.000+00:00
- **Account Owner:** Chinar Arora (@awomanindatascience) — https://gondola.cc/awomanindatascience

## Caption

It’s Day 3 of building a reasoning model.
Let’s code a text generation function, and optimize for performance during inference today ✨
This one change made my model 13x faster — and I only changed 2 lines of code.
Every time an LLM generates a new word, it re-feeds the entire conversation back through the model — even though it already computed attention for every earlier word on the last step.
The fix: a KV cache. Store what you already computed, only process the new token.
5 tok/sec → 29 tok/sec with caching. → 68 tok/sec with torch.compile stacked on top.
Reasoning models generate hundreds of candidate chains of thought — every token saved here gets multiplied hundreds of times over. Save this if you’re building your own LLM 🔖
Day 4: teaching the model to actually reason →
#buildareasoningmodel #llmfromscratch #kvcache #machinelearning #AIengineering

## Stats

- **Views:** 0
- **Likes:** 42
- **Shares:** 0
- **Comments:** 2

## Tags

aiengineering, kvcache, machinelearning, buildareasoningmodel, llmfromscratch

---
Copyright (c) Gondola