Data Cleaning Fundamentals for Interviews: Missing Values, Outliers and Duplicates

The same sales dashboard can tell two completely different stories before and after cleaning: one shows a blockbuster month, the other reveals duplicate orders, blank pincodes and one impossible transaction doing all the drama. Data cleaning is the quiet step that decides whether analysis becomes insight or confidently presented nonsense.

The Big Picture: Cleaning Is Decision-Making, Not Housekeeping

Think of data cleaning as a controlled conversion of raw operational data into analysis-ready data. The goal is not to make the dataset look neat; the goal is to make sure every transformation is defensible, reproducible and aligned with the business question.

The Three Data Defects You Must Recognise

Most cleaning conversations reduce to three defects: missing values, outliers and duplicates. Each has a different risk. Missing values can bias your sample, outliers can distort averages and models, and duplicates can inflate counts, revenue, customers or conversions.

A Five-Step Cleaning Process You Can Apply in Any Dataset

Use this structure when you receive a raw dataset, whether it is a customer file, campaign report, loan dataset or operations dump.

Missing Values: Diagnose Before You Impute

A missing value is not automatically a mistake. A blank field can mean “not applicable”, “customer refused”, “system failed”, “not captured yet” or “unknown”. Those meanings require different treatments.

There are three useful missingness patterns:

• Missing completely at random: the absence is unrelated to observed or unobserved data. Example: a random upload failure.
• Missing at random: missingness is related to observed variables. Example: income is more often missing for self-employed applicants.
• Missing not at random: missingness is related to the missing value itself. Example: high-income customers may avoid revealing income.

Common treatments: delete rows only when the missing share is small and unbiased; use mean or median imputation for simple numeric fields; use mode imputation for categorical fields; add a missingness flag when absence itself may carry information; or collect the data again if the field is business-critical.

Outliers: Separate Error, Rare Event and Business Signal

An outlier is an observation that lies unusually far from the rest of the data. But “unusual” is not the same as “wrong”. A ₹0 order value may be a system error; a very large order may be a corporate buyer; a sudden delivery delay may reveal a route disruption.

Worked example - IQR method: Suppose order values are ₹100, ₹120, ₹130, ₹140, ₹150, ₹160, ₹180 and ₹1,500. Q1 is ₹125, Q3 is ₹170, so IQR = ₹45. The upper fence is Q3 + 1.5 × IQR = ₹170 + ₹67.5 = ₹237.5. The ₹1,500 order is statistically flagged, but you still verify it before removing it. It may be a bulk order, not an error.

Duplicates: Entity Resolution, Not Just Deleting Rows

A duplicate can be simple or surprisingly hard. Exact duplicates have the same values across rows. Fuzzy duplicates refer to the same real-world entity but appear differently: “Amit K.”, “Amit Kumar”, two phone numbers, one email typo, or two product listings for the same item.

Good duplicate handling requires a survivorship rule: if two records refer to the same entity, which value survives? The latest address may be better for delivery, but the verified PAN may be better for KYC. This is where business context matters.

Data Quality Metrics to Track

If you claim the dataset is clean, prove it with metrics. These measures are especially useful when explaining your cleaning work to a business stakeholder or interviewer.

Definitions You Should Be Able to Say Clearly

Meesho: Cleaning Marketplace Data Before It Reaches Search, Recommendations and Decisions

Situation: Meesho operates in Indian e-commerce with a large base of sellers, value-conscious buyers and highly varied product catalogues across fashion, home and lifestyle categories. In such a marketplace, raw data can become messy quickly: product titles may be inconsistent, sizes may be missing, duplicate listings may appear, and customer addresses may vary across languages, pincodes and local naming conventions.

The move: The primary cleaning driver is catalogue and data governance at ingestion: standardising product attributes before they flow into search, recommendations, pricing and operations dashboards. Supporting drivers include mandatory category-specific fields, fuzzy matching to identify duplicate listings, validation rules for pincodes and product attributes, human review for ambiguous cases, and feedback loops from returns, ratings and search behaviour.

Outcome or lesson: Cleaner data improves discoverability, reduces misleading analytics and makes downstream machine learning more reliable. The important lesson for interviews is that data cleaning is not one command in Python; it is a system of rules, validation and business ownership.

Takeaway: A shallow answer says “remove nulls and duplicates.” A strong answer says “build rules that preserve the real marketplace signal while preventing bad data from polluting decisions.”

How AI Changes Data Cleaning Fundamentals

AI is making data cleaning faster, but not judgment-free. In 2026, the best analysts use AI to detect patterns and propose rules, while humans still decide what the data means.

• AI-assisted anomaly detection: Machine learning can flag unusual transactions, delivery times, claims, clicks or customer behaviour that simple thresholds may miss. The analyst still classifies each pattern as error, rare event or signal.
• Entity resolution at scale: AI helps match fuzzy duplicates across names, addresses, phone formats and product titles. This is especially useful in customer master data, marketplace catalogues and banking KYC workflows.
• LLM-generated cleaning rules: Tools can read column descriptions and suggest checks like date consistency, allowed categories, pincode length or negative-value warnings. The risk is hallucinated rules, so every rule needs business validation.

Interview Relevance

What to Revise Next

Once you understand cleaning, move to the first-pass analysis that comes immediately after it. Then revise the mistakes that make otherwise smart analysts lose credibility.