Model & Statistical Metrics Reference Sheet for Analytics Interviews
If a fraud model blocks too many genuine UPI payments, is it “accurate” or expensive? The uncomfortable truth: a model can score 98% accuracy and still be useless if it misses the business error that matters.
- Choose the metric from the decision risk: false positives, false negatives, ranking quality, forecast error or statistical uncertainty.
- Classification metrics: Accuracy = correct predictions / total; Precision = TP / (TP + FP); Recall = TP / (TP + FN); F1 = harmonic mean of precision and recall.
- Regression metrics: MAE is average absolute error, RMSE penalises large errors more, MAPE gives percentage error, and R² explains variance captured.
- Statistical metrics: p-value tests evidence against a null hypothesis; confidence interval gives a plausible range for a population parameter.
- AUC-ROC measures ranking ability across thresholds; it is useful when probability ordering matters more than one fixed cut-off.
- Never quote a metric alone: always add context, baseline, class balance, cost of error and business action.
The Big Picture: Metrics Are Decision Filters
A metric is not a decoration on a dashboard. It is a filter that tells a manager whether to approve a model, change a threshold, launch an experiment, or investigate a process. Start with the business decision, then pick the statistical lens.
Here are the six measures you should be able to write and interpret quickly.
Core Explanation: Match the Metric to the Problem Type
Most analytics answers become simple when you first identify the task: classification, regression, ranking, or statistical inference.
1. Classification Metrics: When the Output Is a Class
A classification model predicts categories: fraud or not fraud, churn or not churn, lead converts or does not convert. The heart of classification metrics is the confusion matrix.
Worked Example: Why Accuracy Can Mislead
Suppose a payment fraud model reviews 1,000 transactions. Actual fraud cases are 50. The model flags 40 transactions as fraud. Of these, 30 are truly fraud.
The model looks excellent at 97% accuracy, but it misses 40% of fraud cases. For fraud detection, recall and the cost of false positives matter more than accuracy alone.
2. Regression Metrics: When the Output Is a Number
A regression model predicts a continuous value: demand, revenue, delivery time, credit loss, salary, or price. Here, the question is not “right or wrong” but “how far off?”
3. Statistical Metrics: When You Need Evidence, Not Prediction
Statistical metrics help answer whether an observed pattern is likely to be real or just noise. They are common in A/B testing, campaign measurement, pricing experiments and operations improvement.
The Metric Choice Matrix
Use this 2x2 to avoid the most common trap: using the metric you remember instead of the metric the decision needs.
Definitions You Can Say in One Breath
- Accuracy: the proportion of all predictions that are correct.
- Precision: the proportion of predicted positives that are truly positive.
- Recall: the proportion of actual positives correctly identified by the model.
- F1 score: the harmonic mean of precision and recall.
- p-value: the probability of results at least this extreme if the null hypothesis is true.
- Confidence interval: a range of plausible values for an unknown population parameter.
For an Indian payment gateway, a fraud model should not be judged only by accuracy because genuine transactions vastly outnumber fraudulent ones. The primary driver is the cost asymmetry between missed fraud and wrongly blocked customers, supported by RBI compliance expectations, merchant experience and real-time payment latency. So what: the metric must balance risk control with payment success, not merely maximise correctness.
Razorpay: Fraud Scoring in Indian Digital Payments
Razorpay shows why model metrics in payments must balance fraud prevention, approval rates, merchant trust and real-time decisioning.
Situation: Indian digital payments run at massive velocity across UPI, cards, wallets and net banking. A payment gateway has to help merchants accept legitimate payments smoothly while reducing fraud, chargebacks and suspicious transactions. In this environment, a model that simply labels transactions as “safe” or “risky” is not enough; the threshold decides customer experience and merchant revenue.
The move: Razorpay and similar payment infrastructure players use risk scoring, rule engines and transaction monitoring to detect suspicious behaviour. The primary driver is real-time risk classification. Supporting drivers include merchant category patterns, transaction velocity signals, device and behavioural indicators, compliance workflows and human review for ambiguous cases.
Outcome or lesson: The best evaluation is not one metric. Precision matters because false positives block genuine customers. Recall matters because missed fraud creates losses and disputes. Latency matters because payment decisions must happen almost instantly. The strategic lesson is simple: in high-volume fintech, model performance is a portfolio of business metrics, not a single ML score.
The complete answer is not “Razorpay uses AI for fraud.” A strong answer says: the primary challenge is asymmetric error cost, supported by real-time infrastructure, merchant-level patterns, compliance constraints and threshold management.
How AI Changes Model & Statistical Metrics
AI does not remove the need for metrics; it makes metric discipline more important because models are now easier to build, deploy and misuse.
For 2026 interviews, add one sentence on model monitoring: “After deployment, I would track both model metrics like AUC or RMSE and business metrics like conversion, loss rate, latency and complaint rate, because model quality can drift as customer behaviour changes.”
Interview Relevance
“A churn model has 92% accuracy. Would you call it a good model? Which other metrics would you check?”
If you remember only one line, say this: “Accuracy tells me overall correctness; precision and recall tell me whether the model is making the right kind of mistakes for the business decision.”
Common Mistake
The mistake: calling a model good because one metric is high, especially accuracy. Why it costs candidates: it shows you can calculate metrics but cannot manage business trade-offs. One-line fix: always compare the metric with baseline, class balance, error cost and the action the model will trigger.
What to Revise Next
Once metrics are clear, revise how to set fair targets for them and how benchmarks change by sector. Move next to: