StudyBrixFree worksheets, study guides and courses

Artificial intelligence · University

Decision trees, splits and ensembles

A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

Make a worksheet

Prerequisites

Classification and basic probability.

Study notes

Tree model

A decision tree partitions feature space using a sequence of tests. Leaf predictions summarize the training examples reaching them.

Impurity

For class probabilities pj, Gini impurity is 1−sum pj². A split is evaluated using the sizes and impurity of its child groups.

Complexity

Deep trees can fit small idiosyncratic groups. Depth limits, minimum leaf sizes and pruning can control complexity.

Ensembles

Combining trees can reduce some errors, but correlated mistakes and data defects can remain. Ensemble size does not replace proper evaluation.

Worked examples

A binary node has class proportions 0.5 and 0.5. Find Gini impurity.

1−0.25−0.25=0.5.

0.5

A node contains only one class. Find its Gini impurity.

1−1²=0.

0

Check your understanding

  1. What is a leaf in a decision tree?

    1. A guaranteed causal rule
    2. A terminal prediction region
    3. A random test-set label
    4. Always the first input column

  2. Which can limit tree complexity?

    1. Deleting all validation data
    2. Increasing filename length
    3. Allowing every record its own leaf without validation
    4. A minimum number of examples per leaf

  3. Child impurities are 0.2 and 0.4 with equal numbers of examples. Find weighted average impurity.

  4. Why should child impurity be weighted by child size?

Answers and marking guidance

Answers and marking notes
  1. A terminal prediction region

    Leaves end a branch and supply predictions.

  2. A minimum number of examples per leaf

    A minimum leaf size discourages unsupported fine partitions.

  3. 0.3

    Equal weights give (0.2+0.4)/2.

  4. A split can make a tiny pure child while leaving most data poorly separated. Weighting reflects the fraction of examples affected.

    Credit representativeness of the overall split.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.

AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.