# PCA, variance and dimensionality reduction > Principal component analysis finds orthogonal directions of high variance in centred data under its standard formulation. Course: [Machine learning methods and validation](https://studybrix.com/en/courses/machine-learning-methods/index.md) Level: university Prerequisites: Linear algebra, covariance and orthogonal projections. ## Notes ### Purpose Principal component analysis finds orthogonal directions of high variance in centred data under its standard formulation. ### Computation Principal directions can be obtained from covariance eigenvectors or a singular-value decomposition of centred data. ### Variance Explained-variance ratios summarize variance captured, not predictive accuracy or importance to every task. ### Scale PCA is sensitive to units and scaling. High variance can reflect nuisance variation, and low-variance directions can still be useful for prediction. ## Practice questions ### 1. Covariance eigenvalues are 6,3,1. Find the explained-variance ratio of the first component. Answer: 0.6 Explanation: 6/(6+3+1)=0.6. ### 2. For eigenvalues 6,3,1, find the first two components' cumulative ratio. Answer: 0.9 Explanation: (6+3)/10=0.9. ### 3. What does standard PCA do before computing covariance-based directions? A. Always convert all values to zero B. Use test labels to orient axes C. Centre each feature D. Sort features alphabetically as the only step Answer: Centre each feature Explanation: Subtracting feature means is part of the standard centred formulation. ### 4. Are explained-variance ratios identical to classification accuracy? A. Only if ratios sum to one B. Yes, exactly C. Only for every two-dimensional dataset D. No Answer: No Explanation: They measure retained variance, not supervised predictive success. ### 5. A feature has values 2,4,6. Find the centred value corresponding to 6. Answer: 2 Explanation: The mean is 4, so 6−4=2. ### 6. Why can an important predictive signal lie in a low-variance direction? Sample answer / marking guidance: Class separation can depend on a subtle feature even when other unrelated variation is much larger. PCA does not use labels in its basic form. Explanation: Credit unsupervised variance versus predictive relevance. ### 7. How are distinct principal directions chosen in the standard Euclidean formulation? A. Always parallel B. Orthogonally C. Always identically D. By arbitrary file order only Answer: Orthogonally Explanation: PCA components form orthogonal directions under the standard metric. ### 8. Why should PCA be fit on training data rather than the combined train and test data in a strict evaluation? Sample answer / marking guidance: Means, scales and directions are fitted preprocessing parameters. Using test inputs can leak information about the evaluation distribution into training. Explanation: Credit fitted preprocessing and split discipline. ## Further reading - https://developers.google.com/machine-learning/crash-course - https://cs229.stanford.edu/ Original AI-assisted educational draft; not independently teacher-reviewed. References support further study, not copied exercises, affiliation or curriculum accreditation.