Grant Sanderson explains the structure and mathematics of a basic neural network using handwritten digit recognition.
What are the neurons, why are there layers, and what is the math underlying it?
Help fund future projects: patreon.com/3blue1brown
Written/interactive form of this series: 3blue1brown.com/topics/neural-networks
Additional funding for this project was provided by Amplify Partners
For those who want to learn more, I highly recommend the book by Michael Nielsen that introduces neural networks and deep learning: goo.gl/Zmczdy
There are two neat things about this book. First, it's available for free, so consider joining me in making a donation to Nielsen if you get something out of it. And second, it's centered around walking through some code and data, which you can download yourself, and which covers the same example that I introduced in this video. Yay for active learning!
github.com/mnielsen/neural-networks-and-deep-learning
I also highly recommend Chris Olah's blog: colah.github.io/
For more videos, Welch Labs also has some great series on machine learning:
youtu.be/i8D90DkCLhIyoutu.be/bxe2T-V8XRs
For those of you looking to go *even* deeper, check out the text "Deep Learning" by Goodfellow, Bengio, and Courville.
Also, the publication Distill is just utterly beautiful: distill.pub/
Lion photo by Kevin Pluck
Звуковая дорожка на русском языке: Влад Бурмистров.
Thanks to these viewers for their contributions to translations
German: @fpgro
Hebrew: Omer Tuchfeld
Hungarian: Máté Kaszap
Italian: @teobucci, Teo Bucci
-----------------
Timeline:
- Introduction example
- Series preview
- What are neurons?
- Introducing layers
- Why layers?
- Edge detection example
- Counting weights and biases
- How learning relates
- Notation and linear algebra
- Recap
- Some final words
- ReLU vs Sigmoid
Correction - The final index on the bias vector should be "k"
------------------
Animations largely made using manim, a scrappy open source python library. github.com/3b1b/manim
If you want to check it out, I feel compelled to warn you that it's not the most well-documented tool, and has many other quirks you might expect in a library someone wrote with only their own use in mind.
Music by Vincent Rubinetti.
Download the music on Bandcamp:
vincerubinetti.bandcamp.com/album/the-music-of-3blue1brown
Stream the music on Spotify:
open.spotify.com/album/1dVyjwS8FBqXhRunaG5W5u
If you want to contribute translated subtitles or to help review those that have already been made by others and need approval, you can click the gear icon in the video and go to subtitles/cc, then "add subtitles/cc". I really appreciate those who do this, as it helps make the lessons accessible to more people.
------------------
3blue1brown is a channel about animating math, in all senses of the word animate. And you know the drill with YouTube, if you want to stay posted on new videos, subscribe, and click th
Why it's here
17 people recommend this
What people say. Recommended as an exceptionally clear, highly visual introduction to the core mathematical principles and intuition behind neural networks.
Moments people linked to
h
hal9000xpon Hacker News • Recommends
On “Ask HN: Best way to get started with AI?”
…of core principles behind it. So math is a must. A few months ago, I stumbled upon very amazing YouTube Channel 3Blue1Brown which explains math in very accessible way and at the same time I got feeling that I finally started understanding core ideas behind linear algebra and calculus. Just recently he published 4 videos about deep neural networks: link to this videolinklinklink So my fear of ML was gone away and I'm very excited to explore whole new world for neural networks and other things like support vector machines etc…
On “What was the best CS, SE or Math video you watched in 2017?”
link to this video 3Blue1Brown’s playlist on neural networks. Like other 3Blue1Brown videos, the visualisations and explanations of the math/concepts are so amazing, they give you the intuition behind the neural networks naturally.
On “Five Hundred and Seven Mechanical Movements (1908)”
3blue1brown videos are probably the best equivalent. Maybe a little basic? But really, all his math videos are just ridiculously good quality. link to this video
On “State-of-the-art open-source chatbot, Vicuna-13B, just released model weights”
…to bob is 0.5, then the value sent from Alice to Bob is 0.35. This value (and the values from all the other incoming connections) are summed at added to the neuron's negative bias. I highly recommend checking out 3blue1brown series on how neural nets, gradient descent, and the dot product (implemented as a matrix multiplication) all tie together: link to this video…
On “But what is a GPT? Visual intro to Transformers [video]”
For me 3brown1blue series: link to this video was an excellent introduction that made Andrej's videos understandable. Then I did 3 first chapters of fastai book, but found it too high level, while I was interested in how things works under the hood. Going through Andrej's makemore tutorials required quite a lot of time but it's definitely worth it. I used free tier of Google Colab until the last one. Pausing the video a lot after he explains what he plans to do and trying to do it by myself was a very rewarding way to learn, with a lot of "aha" moments.
On “What is backpropagation and what is it doing? [video]”
This was very timely for me and for anyone else learning, here are the first few videos of the series: link to this videolinklink (Original video) link
If you want an extensive answer, I recommend the neural network playlist on 3Blue1Brown: link to this video For a quick answer, watch this segment: link
I think FastAI lesson 3 in "Practical Deep Learning for Coders", has one of the most intuitive buildups of gradient descent and loss that I've seen. * Lecture 1 Book Chapter 2 It doesn't go into the math but I don't think that's a bad thing for beginners. If you want mathematical, 3blue1brown has a great series of videos 3 on the topic. 1link2link3link to this video * I've been messing around with this stuff since 2016 and have done a few different courses like the original Andrew Ng course and more.
On “The future of generative AI is niche, not generalized”
…rately described as references Uh, no, that's not at all how the training process works. The training process very roughly mirrors evolution/natural selection except more more direct and faster. This is a good introduction to the concept of how a computer could learn: link This is a more technical look at how exactly it works in more modern AIs: link to this video (it's a series) > If it encounters a problem it hasn't been programmed for, it cannot reason a new solution. But an LLM can? It's not as good at it as a human but it can Relevant: link (if you don’t want to click it it says “referring to AI models as "just math" or "matrix multiplication" is as uselessly reductive as referring to tigers a…
Studying C or Python is studying C or Python. Rather, study AI. Pick up python along the way, skip C. link to this videolink There's tons of book, but I recommend this since it's hands on "Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow" linklink download some local LLMs and play with them along the way. Good luck.
On “Better and Faster Large Language Models via Multi-Token Prediction”
>> YouTube has always been a mixed bag As an outsider but avid reader of this stuff linked from HN, I would recommend the channel 3blue1brown. He's got several NN and AI related videos, and the couple I've seen were pretty good. link to this video
On “Deep Learning Is Not So Mysterious or Different”
If anyone wants to delve into machine learning, one of the superb resources I have found is, Stanfords "Probability for computer scientists"( link ). It delves into theoretical underpinnings of probability theory and ML, IMO better than any other course I have seen. (Yeah, Andrew Ng is legendary, but his course demands some mathematical familarity with linear algebra topics) And of course, for deep learning, 3b1b is great for getting some visual introduction ( link to this video ).
