Lesson 2 of 5

Training networks

Watch, read the card, then practise. Loss functions · Gradient descent · Backpropagation

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Loss functions
The loss measures how wrong the predictions are. Training tries to make it smaller.
Remember: Number → MSE. Category → cross-entropy.
Gradient descent
Gradient descent nudges each weight downhill on the loss: weight = weight − learning rate × gradient.
Remember: If the loss explodes, lower the learning rate first.
Backpropagation
Backpropagation works out every weight's gradient by passing the error backwards through the layers with the chain rule.
Remember: Forward makes predictions; backward makes gradients.

Try the example in Code Lab

import math
print(round(-math.log(0.9), 2), round(-math.log(0.1), 2))

Lesson 2 of 5

Watch, read the card, then practise. Loss functions · Gradient descent · Backpropagation

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Loss functions
The loss measures how wrong the predictions are. Training tries to make it smaller.
Remember: Number → MSE. Category → cross-entropy.
Gradient descent
Gradient descent nudges each weight downhill on the loss: weight = weight − learning rate × gradient.
Remember: If the loss explodes, lower the learning rate first.
Backpropagation
Backpropagation works out every weight's gradient by passing the error backwards through the layers with the chain rule.
Remember: Forward makes predictions; backward makes gradients.

import math
print(round(-math.log(0.9), 2), round(-math.log(0.1), 2))