Hi Folks! For day 12 of Building a LLM from scratch I understand what is causal self- attention. Causal self-attention ensures an LLM never looks into the future. By masking attention weights above the diagonal, each token can attend only to itself and past tokens, making next-word prediction truly autoregressive. Instead of masking after softmax (which breaks probability normalization), we efficiently mask attention scores with -∞ before softmax. We also apply dropout on attention weights during training to reduce overfitting. Finally, this logic comes together in a compact, batch-aware CausalAttention module used in GPT-style models. I understood how to code a ClassAttention module which will be used in the next section of multi head attention. See you soon! ✌️ #deeplearning #generativeai #largelanguagemodels #womenintech