11K subscribers • 1 video on Hallway
Intermediate. Assumes working knowledge.
540K views6 years ago4 recommendations
AI and machine learningNeural networks and deep learningTransformers and LLMs
Leo Dirac explains the transition from RNNs and LSTMs to transformer-based models in NLP, detailing multi-headed self-attention and positional encoding.
Leo Dirac (@leopd) talks about how LSTM models for Natural Language Processing (NLP) have been practically replaced by transformer-based models. Basic background on NLP, and a brief history of supervised learning techniques on documents, from bag of words, through vanilla RNNs and LSTM. Then there's a technical deep dive into how Transformers work with multi-headed self-attention, and positional encoding. Includes sample code for applying these ideas to real-world projects.What people say. Highly recommended as a clear, visual explanation of how attention mechanisms and transformers work compared to older RNN architectures.
leopdon Hacker News • Recommends
On “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”
FL33TW00Don Hacker News • Recommends
On “Rethinking Attention with Performers”
peterfireflyon Hacker News • Recommends
On “The Transformer Family”
qigtyofhpon Hacker News • Recommends
On “Understanding and Coding the Self-Attention Mechanism of Large Language Models”
abetuskon Hacker News • Mentions
On “We aren't close to creating a rapidly self-improving AI”