10.18: Neural Networks: Backpropagation Part 5 – The Nature of Code

In this video, I implement the formulas for “gradient descent” and adjust the bias in the train() function of my “toy” JavaScript neural network library. I also test the library with a simple XOR dataset.

This video is part of Chapter 10 of The Nature of Code (

This video is also part of session 4 of my Spring 2017 ITP “Intelligence and Learning” course (

Support this channel on Patreon:
To buy Coding Train merchandise:
To donate to the Processing Foundation:

Send me your questions and coding challenges!:

The Coding Train website:

Links discussed in this video:
The Coding Train on Amazon:
Stochastic Gradient Descent on Wikipedia:

Videos mentioned in this video:
My Neural Networks series:
3Blue1Brown Neural Networks playlist:

Source Code for the all Video Lessons:


The Nature of Code playlist:
For More Coding Challenges:
For More Intelligence and Learning:






17 responses to “10.18: Neural Networks: Backpropagation Part 5 – The Nature of Code”

  1. Éliottulio Camaradonge Avatar

    Actually, XOR is just abs(i1 – i2) :
    abs(1 – 1) = 0
    abs(1 – 0) = 1
    abs(0 – 1) = 1
    abs(0 – 0) = 0

    I wonder if you could use this to generalise it to floats between 0 and 1…
    And then show it using grayscale value inside a square !

  2. alex z Avatar


  3. Laurie LaChanteuse Avatar

    12:47 can you explain why randomizing the order is important here ? what happened if you trained your data in same order at each iteration ?

  4. Wild Animal Channel Avatar

    Might be better if the outputs are from the set {-1,1} instead of {0,1} since your using a sigmoid? Just my guess. No sorry. I got confused with tanh. Can you do a tutorial for the relu function?

  5. Isfhan Ahmed Avatar

    you are real scientist man

  6. Dragonboy60 Avatar

    isnt this uploaded today?

  7. Isfhan Ahmed Avatar

    i tell your story to my kids

  8. Andreas Åhman Avatar

    Finally got a hair cut and trim? Looking good my man!!

  9. It's Eddie! Avatar

    I've gotten exactly the same results as you did in C#! Very interesting series.

  10. Ioni Nielavitzky Avatar

    I am so happy you finally got it!!! This is a great series!

  11. Ioni Nielavitzky Avatar

    Hi! Why are this videos hidden?

  12. Michał Kucharski Avatar

    Po takiej ilości czasu i kodu zobaczyć coś bliskiego 0 i 1. Brawo! 😀

Leave a Reply

Your email address will not be published. Required fields are marked *