Lesson 5 of 5

Training tips

Watch, read the card, then practise. Dropout and regularisation · Batch norm and learning rate

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Dropout and regularisation
Dropout randomly switches off some neurons during training, so the network can't rely on any one of them.
Remember: Use dropout when the training score is much higher than validation.
Batch norm and learning rate
Batch normalisation keeps each layer's outputs in a steady range, and a good learning-rate schedule makes training fast and stable.
Remember: The learning rate is the first setting to tune.

Try the example in Code Lab

import numpy as np
x = np.array([5.0, 6.0, 7.0, 8.0])
mask = np.array([1, 0, 1, 0])
print((x * mask / 0.5).tolist())   # scaled so the average stays the same

← Deep learning

Lesson 5 of 5

Training tips

Watch, read the card, then practise. Dropout and regularisation · Batch norm and learning rate

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Practise this topic

Dropout and regularisation
Dropout randomly switches off some neurons during training, so the network can't rely on any one of them.
Remember: Use dropout when the training score is much higher than validation.
Batch norm and learning rate
Batch normalisation keeps each layer's outputs in a steady range, and a good learning-rate schedule makes training fast and stable.
Remember: The learning rate is the first setting to tune.

Try the example in Code Lab

import numpy as np
x = np.array([5.0, 6.0, 7.0, 8.0])
mask = np.array([1, 0, 1, 0])
print((x * mask / 0.5).tolist())   # scaled so the average stays the same