StudyBrixFree worksheets, study guides and courses

Artificial intelligence · Grades 9–12

Training, validation and honest testing

The model fits its parameters on training data. Success there does not alone demonstrate generalization to new cases.

Make a worksheet

Prerequisites

Fractions, tables and basic prediction tasks.

Study notes

Training

The model fits its parameters on training data. Success there does not alone demonstrate generalization to new cases.

Validation

Validation data supports choices such as model type, thresholds and hyperparameters. Repeated choices can overfit the validation process.

Testing

A held-out test set assesses the selected procedure. Do not repeatedly tune against it while continuing to call it untouched.

Metrics

Choose metrics that reflect important mistakes. Accuracy alone can hide failures on rare classes or specific groups.

Worked examples

Which dataset primarily fits model parameters?

Parameter fitting uses the training examples.

Training data

What is a usual purpose of validation data?

Validation supports model selection, not independent final assessment after repeated tuning.

Choose settings and compare candidate models

Check your understanding

  1. A classifier gets 90 of 120 test examples correct. Find accuracy as a fraction.

  2. What happens if test performance repeatedly guides tuning?

    1. The test is no longer an untouched evaluation
    2. Its size doubles automatically
    3. The test gets more independent
    4. All leakage disappears

  3. Why can 99% accuracy be unhelpful when only 1% of cases are positive?

  4. A baseline makes 20 errors and a new model 15 on the same 100 cases. How many fewer errors are observed?

Answers and marking guidance

Answers and marking notes
  1. 0.75

    90/120=0.75.

  2. The test is no longer an untouched evaluation

    Decisions based on test results leak information into selection.

  3. Predicting every case as negative would get 99% accuracy while missing all positives. Additional metrics and error costs matter.

    Credit the majority-class baseline and missed positives.

  4. 5

    20−15=5; statistical confidence requires further analysis.

Further reading and review references

These are reference links, not copied exercises, institutional endorsement or proof of independent review.

AI-assisted preview; independent educational review required. Level labels describe prerequisites, not accreditation or official curriculum alignment.