Skip to content
0:00 / 0:00

#transformer#neuralnetworks#gradientdescent

Large Language Models from scratch

Graphics in 5 Minutes

33K subscribers • 1 video on Hallway

YouTube
  • Recommended by 3 people: 3 on Hacker News.
  • Presented by Steve Seitz

Beginner. No background in the topic needed.

375K views4 years ago3 recommendations

AI and machine learningComputer scienceTransformers and LLMsNeural networks and deep learningMath for machine learning

Steve Seitz explains the fundamentals of large language models, covering conditional probabilities, neural networks, gradient descent, and backpropagation.

How do language models like chatGPT and Palm work? A short cartoon that explains transformers and the tech behind LLMs. Part 2: youtu.be/YDiSFS-yHwk - intro - autocomplete - search query completion - language modeling with probabilities - time series and graphs - text generation - conditional probabilities - trigrams - universal function approximation - neural networks - gradient descent - back propagation - network capacity

Why it's here

3 people recommend this

What people say. Recommended as a clear, non-technical introduction to how LLMs and neural networks work without dumbing down the content.

sainezon Hacker News • Recommends

On “A Completely Non-Technical Explanation of Deep Learning”

If anyone is looking for a quick overview of how LLMs are built, I highly recommend this video by Steve Seitz: link to this video . It does an excellent job of taking you from 0 to a decent understanding without dumbing down the content or abusing analogies.

peter_d_shermanon Hacker News • Recommends

On “Large Language Models from scratch (2022)”

Consider the following frame/time from the video: link to this video At this point in time in the video, we see the following 3 things: 1) Fourier Series 2) Taylor Series 3) Neural Network The remainder of the video goes on to show how Neural Networks can approximate functions -- much like the Fourier Series and Taylor Series can... Now, I don't hold an advanced Ph.D. degree in Math, and prior to watching this video, I had never made the connection between Fourier Series, Taylor Series, and Neural Networks... I knew about each of those independently -- but prior to watching the video, I never made the connection that they can be used to approximate one another. And that's the one of the key words -- ap…

runetechon Hacker News • Recommends

On “Ask HN: Can someone ELI5 Transformers and the “Attention is all we need” paper”

Not sure I can give an ELI5 explanation, but for me this the closest to what you are after in two short videos on the topic. At least they helped me get up to speed fast. Hope they will do the same for you ;-) Large Language Models from scratch link to this video Large Language Models: Part 2 link

mistermannon Hacker News • Mentions

On “Globalization Is Dead and No One Is Listening”

> What is so unserious, disingenuous, and illegitimate about the “system”? Some examples (I could write indefinitely): link (section "Bank Error In Our Favor" is particularly offensive, to me) link (this sort of thing (which is fairly common) is offensive to me, as is our culture's complacency towards it) How it is (in part) perpetuated, in my estimation: - I’m sure whatever the system is it isn’t perfect. But here I am using the “system” and living my life. rationalization - I see so many strange comments on the internet that sound like “red-pilled” nonsense. training-based heuristic pattern matching to ~bad/dumb/wrong - Could it be better and is there room for improvement? Absolute…

mistermannon Hacker News • Mentions

On “The Starship or the Canoe: Where will our future adaptations be? (2017)”

(Note: I am a different Human than the one you are replying to - apologies for interjecting.) > "Physically preventable" is a moving target. So are rockets and artillery shells, but check this out: link Humans can be very clever and generous ($60,000 USD per interceptor missile, hoo baby!! Is "democracy" not the greatest thing ever!!!???) when they set their minds to something. Unfortunately, delusion, untruthfulness, deceit, misdirection, etc are also somethings , and apparently very desirable somethings, seemingly irresistible to some. > The more resources are invested in preventing currently preventable deaths, the more farther away the target moves, because more and more causes of deat…