Grant Sanderson explains the mathematics of cost functions and gradient descent in neural network training.
Cost functions and training for neural networks.
Help fund future projects: patreon.com/3blue1brown
Special thanks to these supporters: 3b1b.co/nn2-thanks
Written/interactive form of this series: 3blue1brown.com/topics/neural-networks
This video was supported by Amplify Partners.
For any early-stage ML startup founders, Amplify Partners would love to hear from you via 3blue1brown@amplifypartners.com
To learn more, I highly recommend the book by Michael Nielsen
neuralnetworksanddeeplearning.com/
The book walks through the code behind the example in these videos, which you can find here:
github.com/mnielsen/neural-networks-and-deep-learning
MNIST database:
yann.lecun.com/exdb/mnist/
Also check out Chris Olah's blog:
colah.github.io/
His post on Neural networks and topology is particular beautiful, but honestly all of the stuff there is great.
And if you like that, you'll *love* the publications at distill:
distill.pub/
For more videos, Welch Labs also has some great series on machine learning:
youtu.be/i8D90DkCLhIyoutu.be/bxe2T-V8XRs
"But I've already voraciously consumed Nielsen's, Olah's and Welch's works", I hear you say. Well well, look at you then. That being the case, I might recommend that you continue on with the book "Deep Learning" by Goodfellow, Bengio, and Courville.
Thanks to Lisha Li (@lishali88) for her contributions at the end, and for letting me pick her brain so much about the material. Here are the articles she referenced at the end:
arxiv.org/abs/1611.03530arxiv.org/abs/1706.05394arxiv.org/abs/1412.0233
Music by Vincent Rubinetti:
vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
Thanks to these viewers for their contributions to translations
Звуковая дорожка на русском языке: Влад Бурмистров.
Hebrew: Omer Tuchfeld
Italian: @teobucci
-------------------
Video timeline
- Introduction
- Recap
- Using training data
- Cost functions
- Gradient descent
- More on gradient vectors
- Gradient descent recap
- Analyzing the network
- Learning more
- Lisha Li interview
- Closing thoughts
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click the bell to receive notifications (if you're into that).
If you are new to this channel and want to see more, a good place to start is this playlist: 3b1b.co/recommended
Various social media stuffs:
Website: 3blue1brown.com
Twitter: twitter.com/3Blue1Brown
Patreon: patreon.com/3blue1brown
Facebook: facebook.com/3blue1brown
Reddit: reddit.com/r/3Blue1Brown
Why it's here
4 people recommend this
What people say. Highly recommended as one of the clearest visual explanations of neural network fundamentals and gradient descent.
h
hal9000xpon Hacker News • Recommends
On “Ask HN: Best way to get started with AI?”
…principles behind it. So math is a must. A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus. Just recently he published 4 videos about deep neural networks: linklink to this videolinklink So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc…
On “What is backpropagation and what is it doing? [video]”
This was very timely for me and for anyone else learning, here are the first few videos of the series: linklink to this videolink (Original video) link
If you want an extensive answer, I recommend the neural network playlist on 3Blue1Brown: link For a quick answer, watch this segment: link to this video
On “LLMs Aren't Thinking, They're Just Counting Votes”
…different from most human person's experience with basic education ( link ). Though it is obvious that humans can do more than that too. And while "voting" that may be a close-enough metaphor for the multi-layer perceptron layers, it is not what is happening in the attention layers. I suggest checking out 3blue1brown's videos on gradient descent link to this video and "How might LLMs store facts" link versus how attention works link…
On “What is backpropagation and what is it doing? [video]”
A bit of a side-note, but I think it is an interesting piece of marketing that Amplify Partners decided to sponsor1 the previous video in this series. I wonder (and hope) we'll see more VCs sponsoring open educational content relevant to their focus. 1link to this video
