Hypothesis Testing Explained: H0, H1, p-value and Significance
After Probability Essentials for Analytics Interviews, the next question is how analysts use probability to make decisions from sample data. Hypothesis testing gives that decision flow: define H₀ and H₁, choose the right statistical test, compare p-value with significance level, and make an evidence-based business conclusion. In interviews, this matters because examples like Zomato average delivery time or Flipkart checkout bounce rate test whether a result is likely real or just noise.
- Hypothesis testing flow has 6 steps: Define H₀ & H₁, choose test, set α, calculate test statistic, find p-value, compare p vs α & decide.
- Null Hypothesis (H₀): Default assumption of no effect or no difference; rejected if p < α.
- H₁ is the alternative: the competing claim against the null hypothesis.
- p-value: P(observing data this extreme | H₀ true); does NOT equal P(H₀ is true).
- α is the significance level: usually 0.05, and Type I Error probability = α.
- Decision rule: Reject H₀ if p < α; otherwise fail to reject H₀.
- Test choice depends on the situation: z-test, t-test, paired t-test, chi-square test, or ANOVA (F-test).
Big Picture: Hypothesis Testing Flow
Hypothesis testing is a structured analyst decision flow. The core movement is from a business question to H₀ and H₁, then to the right test, the significance level α, the test statistic, the p-value, and finally a decision to reject or fail to reject.
p-value = P(data this extreme | H₀ true). Reject H₀ if p < α (typically 0.05).
H₀, H₁, p-value and Significance
Null Hypothesis (H₀) is the default assumption of no effect or no difference; rejected if p < α. H₁ is the alternative hypothesis, the competing claim used against the null.
p-value is the probability of observing your result or something more extreme if the null hypothesis were true. A simple explanation is: "If the new feature had no effect at all, there's only a 3% chance we'd see a difference this large between groups just by random chance. Since 3% is below our threshold (5%), we conclude the feature likely does have a real effect."
The significance level α is usually 0.05. Type I Error is a false positive: rejecting H₀ when it is actually true; probability = α (significance level).
Choosing the Right Test
Test choice comes from the type of question, sample size, whether population σ is known, number of groups, and the assumptions behind the data. For example, a z-test fits a large sample with known population σ, while a t-test fits a small sample with unknown σ.
For business cases, this means matching the statistical test to the decision. Testing if Zomato avg delivery time differs from 35 min (n=500) points to a z-test. Asking whether a new Flipkart checkout page is reducing bounce rate vs benchmark points to a t-test (1-sample).
Structuring a Hypothesis Testing Explained Interview Answer
"How would you test if Zomato average delivery time differs from 35 min, and how would you decide using p-value and α?"
p-value is NOT the probability that the null hypothesis is true. It is NOT the probability the result is due to chance. It is NOT the probability that the effect is practically important.
The common mistake is choosing a test without checking assumptions. For example, if the interviewer gives non-normal data and you apply t-test, you'll be penalised - always check distribution first.
Conclusion
Hypothesis testing turns sample data into a decision by defining H₀ and H₁, choosing the right test, setting α, finding the p-value, and deciding whether to reject or fail to reject H₀. In analyst interviews, the strongest answers connect the business question to the correct test and explain the decision rule clearly.