Decision trees

A decision tree predicts by asking yes/no questions about the features until it reaches an answer (a leaf).

• Each split picks the question that best separates the classes (lowest Gini impurity or entropy).
• Gini = 1 − sum of squared class shares; 0 means the node is pure.
• Trees don't need feature scaling and are easy to explain.

shares = [0.8, 0.2]          # 8 'yes', 2 'no'
print(round(1 - sum(s * s for s in shares), 2))

0.32

Remember: Limit max_depth, or the tree will memorise the training data.

Common mistake: Letting a tree grow fully, then wondering why the test score is much lower.