# @awomanindatascience on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/DeDb2XXz3HR
- **Gondola URL:** https://gondola.cc/posts/71535736-awomanindatascience-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/7fcd841458.jpg
- **Posted:** 2026-10-04T01:18:34.000+00:00
- **Account Owner:** Chinar Arora (@awomanindatascience) — https://gondola.cc/awomanindatascience

## Caption

Day 4 of building reasoning model:  how do we evaluate reasoning models? 🧠
Right answer ≠ right reasoning. A model can stumble into the correct answer with broken logic and fail the next problem. Here’s how to grade the thinking, not just the answer:
1.  Verifiable rewards — math, code, logic puzzles. Write a checker, score automatically. Cheap, objective, no humans needed. (This is what DeepSeek R1 used.)
2.  Process vs outcome — outcome rewards check only the final answer; process rewards score each intermediate step. This is what catches right-answer-wrong-reasoning.
3.  LLM-as-judge — for open-ended work where no checker exists. Fast and scalable, but biased: it favors longer answers and its own style. Rule: the judge must be stronger than the student.
4.  Human eval — still the gold standard for taste and tone. Expensive and slow, so use it last, on a small sample, to validate your automated pipeline.
The playbook: verifiable where you can check, process rewards where steps matter, LLM judges for the open-ended, humans for the final word.
Building a math verifier for day 4 of my challenge. 
📖 Following Build a Reasoning Model (From Scratch) by Sebastian Raschka. 
Which one are you using on your models right now? Drop it below 👇
#MachineLearning #LLMs #ReasoningModels #aiengineering

## Stats

- **Views:** 1,288
- **Likes:** 11
- **Shares:** 0
- **Comments:** 0

## Tags

aiengineering, llms, reasoningmodels, machinelearning

---
Copyright (c) Gondola