StudyBrixFree worksheets, study guides and courses

Artificial intelligence · lesson 1 of 4

Decision trees, splits and ensembles

A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

University8 practice questionsNotes + worked examples
Take the quizStudy guideRevision sheet

Learn the ideas

Prerequisites: Classification and basic probability.

Tree model

A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

Impurity

For class probabilities pj, Gini impurity is 1−sum pj². A split is evaluated using the sizes and impurity of its child groups.

Complexity

Deep trees can fit small idiosyncratic groups. Depth limits, minimum leaf sizes and pruning can control complexity.

Ensembles

Combining trees can reduce some errors, but correlated mistakes and data defects can remain. Ensemble size does not replace proper evaluation.

Worked examples

Example 1

A binary node has class proportions 0.5 and 0.5. Find Gini impurity.

1−0.25−0.25=0.5.

0.5

Example 2

A node contains only one class. Find its Gini impurity.

1−1²=0.

0

Check your understanding

8 questions

Decimals or fractions are accepted for numeric answers. Proofs and written responses use sample answers and require human marking. This is practice, not a protected examination.

1 A binary node has class proportions 0.5 and 0.5. Find Gini impurity.

Show answer and explanation

0.5

1−0.25−0.25=0.5.

2 A node contains only one class. Find its Gini impurity.

Show answer and explanation

0

1−1²=0.

3 What is a leaf in a decision tree?

Show answer and explanation

A terminal prediction region

Leaves end a branch and supply predictions.

4 Which can limit tree complexity?

Show answer and explanation

A minimum number of examples per leaf

A minimum leaf size discourages unsupported fine partitions.

5 Child impurities are 0.2 and 0.4 with equal numbers of examples. Find weighted average impurity.

Show answer and explanation

0.3

Equal weights give (0.2+0.4)/2.

6 Why should child impurity be weighted by child size?

Sample answer and marking guidance

A split can make a tiny pure child while leaving most data poorly separated. Weighting reflects the fraction of examples affected.

Credit representativeness of the overall split.

7 Why can many models still share the same blind spot?

Sample answer and marking guidance

They may share training data, features, labels or assumptions, creating correlated errors rather than independent evidence.

Credit common causes.

8 Does a tree split establish a causal relationship?

Show answer and explanation

No

A predictive partition is not a causal experiment.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.