Significance Tests | Notion

Hypothesis Tests - What to use & When

For Means (Continuous Data)

Scenario	Test	Notes
One group vs known value	One‑sample t‑test	Normal-ish data
Two independent groups	Independent t‑test	Equal variances? If not → Welch
Paired measurements	Paired t‑test	Before/after, matched pairs
>2 groups	ANOVA	If significant → post‑hoc tests
Non‑normal data	Mann–Whitney U, Wilcoxon signed‑rank	For independent / paired

For Proportions (Categorical Data)

Scenario	Test	Notes
Two proportions	Chi‑square test	Expected counts ≥ 5
Small sample proportions	Fisher’s exact test	Gold standard for small n
>2 categories	Chi‑square test of independence	Contingency tables

For Counts / Rates

Scenario	Test	Notes
Comparing incidence rates	Poisson test	Rare events
Overdispersed counts	Negative binomial regression	Variance > mean

For Survival / Time‑to‑Event

Scenario	Test	Notes
Compare survival curves	Log‑rank test	Non‑parametric
Adjusted survival	Cox proportional hazards model	Assumes proportional hazards

For Correlation & Association

Scenario	Test	Notes
Linear relationship	Pearson correlation	Continuous, normal
Monotonic relationship	Spearman correlation	Non‑normal or ordinal
Predicting continuous outcome	Linear regression	Check residuals
Predicting binary outcome	Logistic regression	Odds ratios

🧭 3. Quick Decision Tree

1. What type of data?

Continuous → t‑tests, ANOVA, regression
Categorical → χ², Fisher
Counts → Poisson, negative binomial
Time‑to‑event → Log‑rank, Cox

2. How many groups?

1 group → One‑sample tests
2 groups → Independent/paired tests