Which dataset primarily fits model parameters?
- A blank table
- Training data
- A random website title
- An untouched final test set only
Answer and explanation
Training data
Parameter fitting uses the training examples.
Subjects
Model evaluation
Choose questions individually, practise online, or create a worksheet with matching answers. Counts include generated variants, not only authored originals.
Make a worksheet1,195 question instances
Sample exercises
What is a usual purpose of validation data?
- Secretly add test labels to features
- Replace all training data with metadata
- Guarantee no future failures
- Choose settings and compare candidate models
Answer and explanation
Choose settings and compare candidate models
Validation supports model selection, not independent final assessment after repeated tuning.
A classifier gets 90 of 120 test examples correct. Find accuracy as a fraction.
Answer and explanation
0.75
90/120=0.75.
What happens if test performance repeatedly guides tuning?
- The test is no longer an untouched evaluation
- Its size doubles automatically
- The test gets more independent
- All leakage disappears
Answer and explanation
The test is no longer an untouched evaluation
Decisions based on test results leak information into selection.
Why can 99% accuracy be unhelpful when only 1% of cases are positive?
Answer and explanation
Predicting every case as negative would get 99% accuracy while missing all positives. Additional metrics and error costs matter.
Credit the majority-class baseline and missed positives.
A baseline makes 20 errors and a new model 15 on the same 100 cases. How many fewer errors are observed?
Answer and explanation
5
20−15=5; statistical confidence requires further analysis.
Study notes
Training, validation and honest testing
The model fits its parameters on training data. Success there does not alone demonstrate generalization to new cases.
Grades 9–12Training, validation and honest testing · revision notes
The model fits its parameters on training data. Success there does not alone demonstrate generalization to new cases.
Grades 9–12Cross-validation, grouped data and time-aware splits
A k-fold procedure rotates held-out folds to estimate performance for a specified training process. Splits and all fitted preprocessing must be controlled.
UniversityCross-validation, grouped data and time-aware splits · revision notes
A k-fold procedure rotates held-out folds to estimate performance for a specified training process. Splits and all fitted preprocessing must be controlled.
UniversityPrecision from true and false positives
The denominator counts predicted positives. This generator always gives a nonzero denominator.
Grades 9–12Precision from true and false positives · formula reference
The denominator counts predicted positives. This generator always gives a nonzero denominator.
Grades 9–12Recall from true positives and false negatives
The denominator counts actual positives, not predicted positives.
Grades 9–12Recall from true positives and false negatives · formula reference
The denominator counts actual positives, not predicted positives.
Grades 9–12Positive-class F1 score
This is a class-specific score, not a multiclass macro average or a cost-sensitive objective.
UniversityPositive-class F1 score · formula reference
This is a class-specific score, not a multiclass macro average or a cost-sensitive objective.
UniversityRelated courses
Artificial intelligence4 lessons
AI foundations and responsible use
Understand learning tasks, datasets, honest evaluation, privacy and human responsibility. No paid AI tool is required.
32 quiz questions · School (K–12)
Artificial intelligence4 lessons
Mathematics of machine learning
Work through regression, gradients, classification and generalization using paper-based examples.
32 quiz questions · University
Artificial intelligence4 lessons
Machine learning methods and validation
Compare trees, clustering, PCA and validation without confusing training performance with evidence of generalization.
32 quiz questions · University
Artificial intelligence4 lessons
Neural networks, attention and embeddings
Build from artificial neurons and backpropagation to attention and embedding geometry.
32 quiz questions · University, Advanced university
Artificial intelligence4 lessons
Generative AI, RAG and practical evaluation
Practise clear prompting, retrieval-grounded workflows, evidence checking and permission-aware deployment.
32 quiz questions · School (K–12), University