# @awomanindatascience on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/DUhr_W0kw06
- **Gondola URL:** https://gondola.cc/posts/61229893-awomanindatascience-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/196c1fd62a.jpg
- **Posted:** 2026-02-09T06:05:54.000+00:00
- **Account Owner:** Chinar Arora (@awomanindatascience) — https://gondola.cc/awomanindatascience

## Caption

Hey Folks! Happy Seahawks day 💚💙
Still smiling because the Seahawks won the Super Bowl! 🏈 What a day to be in Seattle!

And on a personal note, another exciting win for my Build an LLM from Scratch journey.

This week’s milestone: from training small to leveraging GPT-2.

Here’s what I did:
• Started by training my own mini language model with a small vocabulary and limited dataset from a short story publicly available to understand the fundamentals -tokenization, training loops, loss behavior.
• Saved model checkpoints using PyTorch state_dict so I could resume training without starting over (training LLMs is expensive!).
• Then took the big leap: loaded pretrained GPT-2 weights instead of training from scratch.
• Mapped the OpenAI parameters into my custom architecture and moved the model to GPU for inference.
• Generated coherent text the best validation that the weights were loaded correctly.

Why this matters:
Training from scratch teaches you how models learn.
Loading pretrained weights shows you how real-world systems scale without spending millions on compute.

#deeplearning #generativeai #womenwhocode #largelanguagemodels #womenintech

## Stats

- **Views:** 5,701
- **Likes:** 320
- **Shares:** 0
- **Comments:** 5

## Tags

generativeai, womenintech, largelanguagemodels, deeplearning, womenwhocode

---
Copyright (c) Gondola