Hello folks!! Its Day 15 of Building an LLM from scratch✨
Today, I implemented Layer Normalization with per-token, feature-wise normalization to stabilize activations and improve training convergence, using learnable scale and shift parameters and biased variance for GPT-style compatibility.
I also integrated the transformer feed-forward network with a 4× hidden dimension expansion and GELU activations, enabling smoother gradient flow and stronger non-linear representations within each transformer block.
#deeplearning #generativeai #womenwhocode #largelanguagemodels #