When NOT to Use ML: The Pragmatic Analyst's Guide
After Model Evaluation & Selection: How to Choose the Right Model, the sharper analyst question is whether a model should be built at all. Machine learning (ML) is powerful but adds complexity, requires large data, is harder to explain, takes time to build, and often underperforms simpler methods on small/clean datasets. Interviewers will test this judgment because the right first response to a modelling request is to question whether ML is needed at all.
- Knowing when NOT to use ML is the analyst's most important skill.
- ML is powerful but adds complexity, requires large data, is harder to explain, takes time to build, and often underperforms simpler methods on small/clean datasets.
- If a dataset has < 500 rows, ML models overfit on small data, cannot generalise, and the train/test split is too small to be meaningful.
- If business rules already suffice, adding ML complexity where a simple IF-THEN rule achieves the same result is not explainable.
- If 100% explainability is needed, most ML models, especially ensemble/deep learning, are black boxes.
- Use the Analyst's Hierarchy of Solutions in order, from business rules / domain knowledge to Deep Learning / GenAI only when above options truly fail and data volume justifies.
Why Analyst Judgment Comes Before ML
The analyst's most important skill is knowing when NOT to use ML. Interviewers will test this: "A manager asks you to build a machine learning model to predict sales. What's your first response?" - The right answer is to question whether ML is needed at all.
The pragmatic lens is simple: use ML only after simpler, faster, more explainable methods fail or the data truly justifies the complexity.
ML is powerful but adds complexity, requires large data, is harder to explain, takes time to build, and often underperforms simpler methods on small/clean datasets.
The Analyst's Hierarchy of Solutions
Use the Analyst's Hierarchy of Solutions in order. It keeps the answer practical, explainable, and aligned with the amount and quality of data available.
Why Simpler Baselines Often Win
In a famous MIT and University of Chicago study, researchers found that simple linear regression beat expensive ML models in 61% of real business forecasting problems - because real business data is often too small, too noisy, or too subject to structural breaks for ML to outperform statistical baselines.
This is why the first pass should typically be SQL aggregation + descriptive stats, followed by interpretable methods such as linear/logistic regression when the problem requires modelling.
Structuring a When NOT to Use ML Interview Answer
"A manager asks you to build a machine learning model to predict sales. What's your first response?"
The #1 way candidates get this wrong is jumping directly to model choice. The stronger answer is to first test whether ML is needed at all.
The most frequent error is treating a manager's request to build a machine learning model as an instruction to start modelling. It costs points because interviewers test pragmatic thinking: ML adds complexity, requires large data, is harder to explain, takes time to build, and often underperforms simpler methods on small/clean datasets.
Conclusion
Knowing when NOT to use ML is as important as knowing how to use it. The strongest analyst answer is pragmatic: start with business rules, SQL aggregation, descriptive stats, and interpretable models, and move to complex ML only when simpler methods fail and the data truly justifies it.