Skip to content
Hallway
0:00 / 0:00

#transformers#attentionmechanism#languagemodels

AI Language Models & Transformers - Computerphile

Computerphile

2.6M subscribers • 70 videos on Hallway

YouTube
  • Recommended by 3 people: 3 on Hacker News.
  • Presented by Rob Miles, AI safety researcher
  • May be out of date. Uploaded 7 years ago, in an area where tools change fast.

Beginner. No background in the topic needed.

353K views7 years ago3 recommendations

AI and machine learningTransformers and LLMsNeural networks and deep learning

Rob Miles explains how language models generate text and how the transformer architecture improves upon recurrent neural networks.

Plausible text generation has been around for a couple of years, but how does it work - and what's next? Rob Miles on Language Models and Transformers. More from Rob Miles: bit.ly/Rob_Miles_YouTube AI YouTube Comments: youtu.be/XyMdpcAPnZc Thanks to Nottingham Hackspace for providing the filming location: bit.ly/notthack facebook.com/computerphile twitter.com/computer_phile This video was filmed and edited by Sean Riley. Computer Science at the University of Nottingham: bit.ly/nottscomputer Computerphile is a sister project to Brady Haran's Numberphile. More at bradyharan.com

Why it's here

3 people recommend this

What people say. Recommended as an excellent, accessible introduction to transformers and the attention mechanism.

cercatrovaon Hacker News • Recommends

On “A demo of GPT-3's ability to understand long instructions”

This video by Computerphile is a great overview of transformers and how we got to this point 0. Basically the networks we used before, recurrent neural networks, "forgot" prior information so they're not good at long tasks. The transformer architecture however does not forget (or at least as easily). 0 link to this video
View on Hacker News

FranklinMailloton Hacker News • Recommends

On “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”

Those Computerphile videos0 by Rob Miles helped me understand transformers. He specifically references the "Attention is all you need" paper. And for a deeper dive, Andrej Kharpaty has this hands-on video1 where he builds a transformer from scratch. You can check-out his other videos on NLP as well they are all excellent. 0 link to this video , link 1 link
View on Hacker News

mhaston Hacker News • Recommends

On “Modern-Day Oracles or Bullshit Machines? How to thrive in a ChatGPT world”

…parameters you combine with the input tokens to get the next token the system outputs.) For a more fundamental introduction to what all these systems do there are a number of Computerphile videos which are quite informative. Unfortunately I can't find a good playlist of them all but here's one of the early ones. (Robert Miles is in many of them.) link to this video…
View on Hacker News