Skip to content
Hallway
0:00 / 0:00

#neuralnetworks#backpropagation#activationfunctions

CS231n Winter 2016: Lecture 5: Neural Networks Part 2

Andrej Karpathy

50K subscribers • 2 videos on Hallway

YouTube
  • Recommended by 1 person: 1 on Stack Overflow and its sister sites.
  • Presented by Andrej Karpathy, Instructor at Stanford
  • May be out of date. Uploaded 10 years ago, in an area where tools change fast.

Intermediate. Assumes working knowledge.

200K views10 years ago1 recommendation

AI and machine learningComputer scienceNeural networks and deep learningMath for machine learning

Andrej Karpathy covers neural network design, including activation functions, data preprocessing, weight initialization, and batch normalization.

Stanford Winter Quarter 2016 class: CS231n: Convolutional Neural Networks for Visual Recognition. Lecture 5. Get in touch on Twitter @cs231n, or on Reddit /r/cs231n.

Why it's here

1 person recommends this

What people say. Recommended as an excellent explanation of why neural networks require zero-mean input preprocessing to avoid restricted weight updates.

Ngoc Anh Huynhon Stack Overflow • Recommends

On “Neural Network flatlines after one epoch”

…early. Suppose that your problem is learnable at all, there are many reasons for the for this behavior. These are at the top of my head: Improper preprocessing of input: Neural network prefers input with zero mean. E.g., if the input is all positive, it will restrict the weights to be updated in the same direction, which may not be desirable (link to this video). Therefore, you may want to subtract the mean from all the images (e.g., subtract 127.5 from each of the 3 channels). Scaling to make unit standard deviation in each channel may also be helpful. Generalization ability of the network: The network is not complicated or deep enough for the task. This is very easy to check. You can tra…
11View the answer