Trees and ensembles

Watch, read the card, then practise. Decision trees · Random forests · Gradient boosting

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Decision trees
A decision tree predicts by asking yes/no questions about the features until it reaches an answer (a leaf).
Remember: Limit max_depth, or the tree will memorise the training data.
Random forests
A random forest trains many decision trees on random samples and combines their votes.
Remember: Many varied trees beat one deep tree.
Gradient boosting
Gradient boosting adds trees one at a time, each one fixing the errors left by the trees before it.
Remember: Use early stopping on a validation set to pick the number of trees.

Try the example in Code Lab

shares = [0.8, 0.2]          # 8 'yes', 2 'no'
print(round(1 - sum(s * s for s in shares), 2))

Lesson 1 of 4

Watch, read the card, then practise. Decision trees · Random forests · Gradient boosting

Video coming soon

The video for this lesson is being recorded.

Meanwhile, the notes below and the 60-second cards cover the same ideas.

Decision trees
A decision tree predicts by asking yes/no questions about the features until it reaches an answer (a leaf).
Remember: Limit max_depth, or the tree will memorise the training data.
Random forests
A random forest trains many decision trees on random samples and combines their votes.
Remember: Many varied trees beat one deep tree.
Gradient boosting
Gradient boosting adds trees one at a time, each one fixing the errors left by the trees before it.
Remember: Use early stopping on a validation set to pick the number of trees.

shares = [0.8, 0.2]          # 8 'yes', 2 'no'
print(round(1 - sum(s * s for s in shares), 2))