Choosing a Test

Are your population distributions normal, or, are there more than 40 observations in each sample? If so, choose the Two Sample t-Test. If not, read on:

Do you care about the magnitude of the difference? Consider the Permutation Test. Check the distribution and number of outliers. If the data is skewed, use the median variant, and if there are outliers on both tails, consider using trimmed means.

If you’re instead just looking to determine if one sample is greater than the other, use the Wilcoxon Rank-Sum, particularly with skewed distributions or heavy outliers. If we’re interested in generating a confidence interval for the difference, we can use the Mann-Whitney test. It works similarly to the Wilcoxon Rank-Sum, and results in the same p-value.

Power

Include a discussion about power here

---

All content is licensed under CC-BY-NC .