# @awomanindatascience on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/C3MIzJAReOU
- **Gondola URL:** https://gondola.cc/posts/61230239-awomanindatascience-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/eaf52afcac.jpg
- **Posted:** 2024-02-11T02:00:12.000+00:00
- **Account Owner:** Chinar Arora (@awomanindatascience) — https://gondola.cc/awomanindatascience

## Caption

🍃Day 23 was all about learning the evolution of language models and coding RNNs from scratch in Python. 
From N gram language models —> Word2vec —> Recurrent neural network (RNN)—> Long short term memory networks(LSTM) —> Transformers, we have come a long way.

🍃But what is language modeling? 🤔
Language modeling serves as the backbone of natural language processing, predicting the next word in a sequence to enable meaningful communication. From autocomplete suggestions to virtual assistants, its applications are deeply woven into our daily interactions. This evolution traces back to the pre-deep learning era, where n-gram models laid the foundation for understanding and predicting sequences of words.

🍃N-gram language models, ranging from bigrams to trigrams, relied on a probability-based approach. These models predicted the likelihood of the next word based on historical n-grams, a fixed window of preceding words.

🍃The transition from probability-based models to neural networks marked a paradigm shift in language modeling. Neural networks introduced a more dynamic approach to understanding and predicting sequences, moving beyond the limitations of fixed windows.

🍃To address the sequential nature of language, Recurrent Neural Networks (RNNs) were introduced. RNNs allowed information to persist through the network, enabling them to consider the entire context of a sequence, not just a fixed window.

🍃While RNNs were a significant step forward, they faced challenges such as vanishing gradients. Long Short-Term Memory networks (LSTMs) were developed to address this issue by allowing the network to selectively retain and forget information, making them more effective at capturing long-range dependencies.

🍃The need for models capable of processing arbitrary amounts of context prompted the development of the Transformer architecture. Traditional sequential models struggled with capturing long-range dependencies efficiently.
More on this tomorrow!

#deeplearning #machinelearning #naturallanguageprocessing #100daysofcode #100daysofcodechallenge #womenwhocode #womenintech #codinggirls #languagemodel #llm

## Stats

- **Views:** 904
- **Likes:** 23
- **Shares:** 0
- **Comments:** 0

## Tags

100daysofcodechallenge, womenwhocode, naturallanguageprocessing, 100daysofcode, womenintech, codinggirls, llm, languagemodel, deeplearning, machinelearning

---
Copyright (c) Gondola