facebook pixel
Hello Folks! It’s Day 23 of Building an LLM from Scratch ✨ From past few weeks I have been diving into Fine tuning. Today I completed my code! It was all about instruction finetuning. Until now, my model could generate text but couldn’t reliably follow instructions. I used a dataset of ~1100 instruction–response pairs and formatted them in Alpaca style to ensure consistency. Then I tokenized the data, built custom batches, and masked padding so training focuses on meaningful tokens. Instead of training from scratch, I loaded a pretrained GPT-2 medium model and fine-tuned it for two epochs. The improvement was clear - it finally started following instructions. Biggest takeaway: pretraining teaches language, but finetuning teaches behavior. #deeplearning #largelanguagemodels #generativeai #womenwhocode #datascience

 5.9k

 208

 13

 5.9k

    Suggested Credits
    Tags, Events, and Projects