Chapter 2 Paired Permutation Test

We can use the permutation test if we violate the normality assumption of the paired t-test. However, as we’re still using an average of differences, our test statistic is still subject to outliers.

2.1 How It Works

By convention, differences are defined as \(\text{treatment} - \text{control}\), or \(\text{after} - \text{before}\).

Pre	9.1	6.2	4.2	5.9
Post	7.3	4.8	4.1	4.7
Diff	-1.8	-1.4	-0.1	-1.2

We’ll define our test statistic \(D_{obs}\) as the average of differences: \[ \bar{D}=\frac{1}{n}\sum_{i=1}^{n} D_{i} \] With that formula, we’ll get \(D_{obs}=\) -1.125.

How likely is it that we see \(D_{obs}=\)-1.125 by random chance?

Under the null hypothesis, we’d expect that if we were to randomly switch around (permute) the observations within the pair, we’d see the same test statistic.

Since we have \(n\) pairs, there are \(2^n=16\) possible arrangements where we swap around the values across the treatment and control group. For each permutation, we’ll find \(\bar{D}^*\), the mean of differences for that particular permutation.

More visually:

       Obs 1 Obs 2 Obs 3 Obs 4 Obs 5
before    31    38    46    54    43
after     39    49    55    57    44

can be permuted within pairs, with one permutation looking like:

        Obs * Obs 2 Obs * Obs 4 Obs 5
before*    39    38    55    54    43
after*     31    49    46    57    44

but not across pairs, because we’re using a matched pairs design.

So, our p-value is then just the fraction of permutations that have a test statistic \(D\) as or more extreme than what was observed \(D_{obs}\).

Formal Definitions

2.2 Code

#add code

#add code 2