🌻It’s day 4/30 days of Building a LLM from scratch. Today I implemented a PyTorch data loader that generates input-target pairs required for training an LLM. 🌻And how exactly is this done? So given a text sample, the data loader extracts input blocks as subsamples that serve as input to the LLM and the LLMs prediction task during training is to predict the next word that follows the input block. 🌻During training I mask out all words that are past the target. 🌻The inputs and targets are returned as PyTorch tensors. The input tensor contains the text that the LLM sees and the target tensor includes the targets for the LLM to predict. #deeplearning #generativeai #30dayschallenge #womenwhocode #datascience #womenintech #largelanguagemodels