Classification Models Explained for Analysts
After Regression Models Explained for Analysts, the next analyst interview question is different: instead of predicting a continuous value, how do you evaluate fraud / default versus legit / not fraud decisions? Classification models matter because the core decision is not just which algorithm to use, but which metric to optimise based on the business cost of false positives versus false negatives.
- A confusion matrix separates True Positive (TP), False Positive (FP), False Negative (FN), and True Negative (TN) using fraud / default examples.
- Accuracy = (TP + TN) / (TP + TN + FP + FN), but overall correctness is misleading with class imbalance.
- Precision = TP / (TP + FP), and should be optimised when FP is costly, such as wrongly flagging legit transactions.
- Recall (Sensitivity) = TP / (TP + FN), and should be optimised when FN is costly, such as missing fraud or cancer.
- F1 Score = 2 × (Precision × Recall) / (Precision + Recall), a harmonic mean used to balance when both FP and FN costs matter.
- AUC-ROC means Area Under ROC Curve - model's ability to discriminate; 0.5=random, 0.7-0.8=fair, 0.8-0.9=good, >0.9=excellent.
- Common classification algorithms include Logistic Regression, Decision Tree, Random Forest, XGBoost / LightGBM, SVM, and K-Nearest Neighbours.
The Big Picture: Classification Evaluation for Analysts
Classification evaluation starts with the confusion matrix and then moves to metric selection. The analyst's practical decision is to decide whether false positives are more costly, false negatives are more costly, or both costs matter.
Optimise Precision when FP is costly, optimise Recall when FN is costly, and use F1 Score to balance when both FP and FN costs matter.
Confusion Matrix - Key Metric Formulas
The confusion matrix gives the base counts behind every classification metric. Accuracy, Precision, Recall, F1 Score, and AUC-ROC answer different evaluation questions, so the right metric depends on the business context.
Choosing the Right Classification Algorithm
After choosing the evaluation metric, the analyst selects an algorithm based on how it works, its pros, its cons, and the business use case. The table below compares the core classification models commonly used in analytics ML problems.
How Analysts Connect Metrics to Business Cost
Accuracy measures overall correctness, but it can be misleading with class imbalance. In fraud, 99.9% accuracy can be achieved by predicting everything as not fraud, which is why precision, recall, F1 Score, and AUC-ROC matter in classification evaluation.
Precision asks: of all predicted positives, what fraction are correct? This is the metric to optimise when false positives are costly, such as wrongly flagging legit transactions.
Recall, also called Sensitivity, asks: of all actual positives, what fraction did we catch? This is the metric to optimise when false negatives are costly, such as missing fraud or cancer.
F1 Score is the harmonic mean of Precision and Recall. It is used to balance when both false positive and false negative costs matter, especially with imbalanced classes.
Structuring a Classification Models Explained for Analysts Interview Answer
"You are building a fraud or default classification model. Which algorithm would you choose, and how would you evaluate it when false positives versus false negatives have different business costs?"
Do not default to Accuracy. In classification interviews, the stronger answer is to link Precision, Recall, F1 Score, and AUC-ROC to the business cost of false positives and false negatives.
The most frequent error is using Accuracy as the main metric when classes are imbalanced. Overall correctness is misleading with class imbalance, because fraud can show 99.9% accuracy by predicting everything as not fraud, while still missing the cases the model was meant to catch.
Conclusion
Classification models should be evaluated through the confusion matrix first, then matched to the right metric and algorithm. The final takeaway is simple: choose Precision, Recall, F1 Score, Accuracy, or AUC-ROC based on what the business can least afford to get wrong.