Skip to content
Hallway
0:00 / 0:00

#gradientchecking#neuralnetworks#backpropagation

Neural Networks Demystified [Part 5: Numerical Gradient Checking]

Welch Labs

913K subscribers • 3 videos on Hallway

YouTube
  • Recommended by 1 person: 1 on Stack Overflow and its sister sites.
  • Presented by Stephen Welch
  • Linked in the descriptionstephencwelch/Neural-Networks-Demystified
  • May be out of date. Uploaded 11 years ago, in an area where tools change fast.

Intermediate. Assumes working knowledge.

191K views11 years ago1 recommendation

AI and machine learningComputer scienceNeural networks and deep learningMath for machine learningAlgorithms

Stephen Welch explains how to perform numerical gradient checking to verify the correctness of backpropagation computations.

When building complex systems like neural networks, checking portions of your work can save hours of headache. Here we'll check our gradient computations. Supporting code: github.com/stephencwelch/Neural-Networks-Demystified Link to excellent Stanford tutorial: ufldl.stanford.edu/wiki/index.php/UFLDL_Tutorial In this series, we will build and train a complete Artificial Neural Network in python. New videos every other friday. Part 1: Data + Architecture Part 2: Forward Propagation Part 3: Gradient Descent Part 4: Backpropagation Part 5: Numerical Gradient Checking Part 6: Training Part 7: Overfitting, Testing, and Regularization @stephencwelch

Why it's here

1 person recommends this

What people say. Recommended as a reference for implementing and debugging neural network backpropagation.

Robinon Data Science Stack Exchange • Recommends

I've been struggling with Neural Networks for a while now. I get the math behind backpropagation. Still as reference I'm using the formulas from link to this video. The Network learns XOR: Prediction After Training: 0.0003508415406266712 Expected: 0.0 But basically doesn't learn anything on the MNIST dataset: Error after n trainings examples: - 0 Total Net Error: 4.3739634316135225 - 10000 Total Net Error: 0.4876292680858326 - 20000 Total Net Error: 0.39989816082272944 - 30000 Total Net Error: 0.49507443066631834 - 40000 Total Net Error: 0.5483594859079792 - 50000 Total Net Error: 0.5135921029479789 - 59000 Total Net Error: 0.4686434346776871 Prediction Expected - [0.0…
5View the answer