# Decision trees, splits and ensembles

> A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

Course: [Machine learning methods and validation](https://studybrix.com/en/courses/machine-learning-methods/index.md)
Level: university
Prerequisites: Classification and basic probability.

## Notes

### Tree model

A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

### Impurity

For class probabilities pj, Gini impurity is 1−sum pj². A split is evaluated using the sizes and impurity of its child groups.

### Complexity

Deep trees can fit small idiosyncratic groups. Depth limits, minimum leaf sizes and pruning can control complexity.

### Ensembles

Combining trees can reduce some errors, but correlated mistakes and data defects can remain. Ensemble size does not replace proper evaluation.

## Practice questions

### 1. A binary node has class proportions 0.5 and 0.5. Find Gini impurity.

Answer: 0.5

Explanation: 1−0.25−0.25=0.5.

### 2. A node contains only one class. Find its Gini impurity.

Answer: 0

Explanation: 1−1²=0.

### 3. What is a leaf in a decision tree?

A. A guaranteed causal rule
B. A terminal prediction region
C. A random test-set label
D. Always the first input column

Answer: A terminal prediction region

Explanation: Leaves end a branch and supply predictions.

### 4. Which can limit tree complexity?

A. Deleting all validation data
B. Increasing filename length
C. Allowing every record its own leaf without validation
D. A minimum number of examples per leaf

Answer: A minimum number of examples per leaf

Explanation: A minimum leaf size discourages unsupported fine partitions.

### 5. Child impurities are 0.2 and 0.4 with equal numbers of examples. Find weighted average impurity.

Answer: 0.3

Explanation: Equal weights give (0.2+0.4)/2.

### 6. Why should child impurity be weighted by child size?

Sample answer / marking guidance: A split can make a tiny pure child while leaving most data poorly separated. Weighting reflects the fraction of examples affected.

Explanation: Credit representativeness of the overall split.

### 7. Why can many models still share the same blind spot?

Sample answer / marking guidance: They may share training data, features, labels or assumptions, creating correlated errors rather than independent evidence.

Explanation: Credit common causes.

### 8. Does a tree split establish a causal relationship?

A. Only if depth exceeds ten
B. Only if the feature is numeric
C. No
D. Yes, every split does

Answer: No

Explanation: A predictive partition is not a causal experiment.

## Further reading

- https://developers.google.com/machine-learning/crash-course
- https://cs229.stanford.edu/

Original AI-assisted educational draft; not independently teacher-reviewed. References support further study, not copied exercises, affiliation or curriculum accreditation.