# @awomanindatascience on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/DTEsmL2kw3r
- **Gondola URL:** https://gondola.cc/posts/61230178-awomanindatascience-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/634883b27a.jpg
- **Posted:** 2026-01-04T03:19:31.000+00:00
- **Account Owner:** Chinar Arora (@awomanindatascience) — https://gondola.cc/awomanindatascience

## Caption

Day 10/30 Build a LLM from scratch 
Self-attention, broken down intuitively 🥸

Each word in a sentence is already a vector.
Self-attention lets a word look at every other word, measure relevance using similarity (dot products), normalize those scores with softmax, and create a context vector : a richer representation that captures meaning from the entire sentence.

This is a simplified, weight-free version of self-attention, built purely for intuition.
Real Transformers extend this idea with learnable weights- but the core idea stays the same:
context is everything. 

#deeplearning #generativeai #30dayschallenge #womenwhocode #largelanguagemodels

## Stats

- **Views:** 2,397
- **Likes:** 69
- **Shares:** 0
- **Comments:** 6

## Tags

largelanguagemodels, deeplearning, 30dayschallenge, womenwhocode, generativeai

---
Copyright (c) Gondola