facebook pixel
Hello folks!! Its Day 15 of Building an LLM from scratch✨ Today, I implemented Layer Normalization with per-token, feature-wise normalization to stabilize activations and improve training convergence, using learnable scale and shift parameters and biased variance for GPT-style compatibility. I also integrated the transformer feed-forward network with a 4× hidden dimension expansion and GELU activations, enabling smoother gradient flow and stronger non-linear representations within each transformer block. #deeplearning #generativeai #womenwhocode #largelanguagemodels #

 55.3k

 1.2k

 18

 55.3k

    Suggested Credits
    Tags, Events, and Projects