13.3 Mediation: Sobel’s test and bootstrapping
So the causal steps approach doesn’t really work for mediation… What now? Well, there are two more contemporary methods of testing mediations. While the latter (the bootstrap method) is the most recommended, we demonstrate both for completion’s sake.
13.3.1 Running mediations in R
There are many packages available to run mediations in R, including the well-known package mediation package. One of the most popular software tools for running mediations (and moderations), particularly for SPSS users, is the PROCESS macro by Andrew Hayes. PROCESS is essentially a set of code that can run a series of what we call conditional process models - including mediation and moderation.
Note that due to copyright reasons, the R version of process isn’t available on this book’s corresponding GitHub repo; however, it can be downloaded for free from the official site using the link above. Detailed documentation is available in Hayes (2022). Loading PROCESS is not quite like loading a package in R as it isn’t available in package form. To load it, you must use the source() function in base R, which will run another R script. Place the process.R file somewhere you can find it, and pass the file path to source(). It will take a little bit to load depending on your computer’s specs, but if it has loaded correctly then you should see the following:
##
## ********************* PROCESS for R Version 4.3.1 *********************
##
## Written by Andrew F. Hayes, Ph.D. www.afhayes.com
## Documentation available in Hayes (2022). www.guilford.com/p/hayes3
##
## ***********************************************************************
##
## PROCESS is now ready for use.
## Copyright 2020-2023 by Andrew F. Hayes ALL RIGHTS RESERVED
## Workshop schedule at http://haskayne.ucalgary.ca/CCRAM
## Key to the PROCESS macro is the process() function, which is basically a does-it-all function that will run both mediations and moderations.
13.3.2 Sobel’s test
The first, and perhaps most common method, goes by a couple of names: Sobel’s test or the delta method are perhaps the two most well known. Jamovi calls this the Standard method. Sobel’s test is a direct test of the significance of the ab pathway in a mediation.
The principle behind Sobel’s test is actually relatively simple: it is analogous to a t-test, in that a t-statistic is calculated by dividing the estimate of ab by its estimated standard error:
\[ t_{ab} = \frac{ab}{SE_{ab}} \]
From there, a p-value can be calculated by using this t-statistic and comparing it against a null t-distribution, much like you would with a t-test or a regression slope.
Note that Sobel’s test assumes normality.
To run a mediation, we use the process() function as follows:
- data specifies the dataset.
- x, y and m specify the predictor, outcome and mediator respectively.
- model = 4 specifies that we want to run a mediation. (PROCESS comes with 20-odd different model types!)
- normal = 1 and boot = 0 specifies that we only want a Sobel test for the indirect effect.
med_sobel <- process(
data = med_data,
y = "happiness",
x = "grade",
m = "self_esteem",
model = 4,
normal = 1,
boot = 0
)##
## ********************* PROCESS for R Version 4.3.1 *********************
##
## Written by Andrew F. Hayes, Ph.D. www.afhayes.com
## Documentation available in Hayes (2022). www.guilford.com/p/hayes3
##
## ***********************************************************************
##
## Model : 4
## Y : happiness
## X : grade
## M : self_esteem
##
## Sample size: 100
##
##
## ***********************************************************************
## Outcome Variable: self_esteem
##
## Model Summary:
## R R-sq MSE F df1 df2 p
## 0.5144 0.2646 2.6868 35.2586 1.0000 98.0000 0.0000
##
## Model:
## coeff se t p LLCI ULCI
## constant 1.4995 0.5892 2.5450 0.0125 0.3303 2.6688
## grade 0.5610 0.0945 5.9379 0.0000 0.3735 0.7485
##
## ***********************************************************************
## Outcome Variable: happiness
##
## Model Summary:
## R R-sq MSE F df1 df2 p
## 0.6107 0.3730 2.6613 28.8524 2.0000 97.0000 0.0000
##
## Model:
## coeff se t p LLCI ULCI
## constant 1.9043 0.6055 3.1452 0.0022 0.7026 3.1059
## grade 0.0396 0.1096 0.3612 0.7187 -0.1780 0.2572
## self_esteem 0.6355 0.1005 6.3212 0.0000 0.4360 0.8350
##
## **************** DIRECT AND INDIRECT EFFECTS OF X ON Y ****************
##
## Direct effect of X on Y:
## effect se t p LLCI ULCI
## 0.0396 0.1096 0.3612 0.7187 -0.1780 0.2572
##
## Indirect effect(s) of X on Y:
## Effect
## self_esteem 0.3565
##
## Normal theory test for indirect effect(s):
## Effect se Z p
## self_esteem 0.3565 0.0829 4.2994 0.0000
##
## ******************** ANALYSIS NOTES AND ERRORS ************************
##
## Level of confidence for all confidence intervals in output: 95Here is an output of our mediation using Sobel’s test. You can see that it gives a p-value of not just the individual paths, but of the indirect and direct paths overall. Our indirect effect is significant (z = 4.299, p < .001), while our direct effect is not (z = .367, p = .719) - suggesting a full mediation, as we saw before.
Note that the output helpfully adds a note stating that the “Normal theory test” was used, which corresponds to Sobel’s test.
Sobel’s test is probably the most common method used, at least in most recent psychological literature. However, it too has a noticeable problem in that it only works well in large samples. This is for a couple of reasons, but primarily that the distribution of ab is only normal at very large sample sizes. Therefore, using Sobel’s test in smaller sample sizes is likely to be skewed, which leads to incorrect standard errors.
13.3.3 Bootstrapping
To overcome the problem described above, another approach is to bootstrap the standard errors. This is where we continually resample a large number of times from our original dataset and calculate ab for each sampled dataset. Once we do this enough times, we can build an empirical distribution of possible ab values, and then use that to calculate an empirical p-value for each effect.
The advantage of bootstrapping is that we can calculate more robust confidence intervals around our estimates without needing to assume normality - instead, these are directly derived from our data (in a sense). To that effect, here is an output from a bootstrapped mediation (specifically, a bias-corrected bootstrapping method). A couple of extra notes here:
bc = 1means we want bias-corrected bootstrap intervals.boot = 1000means that we want to bootstrap/resample 1000 times. While this is ok for now (and relatively fast), depending on the data you may want to up this to 10000 or higher.seed = 2024sets a seed for this run. Because bootstrapping involves random sampling, every run will give slightly different results (which isn’t good for replicability/transparency). Setting a seed locks R into generating the same values on every run.
med_bootstrap <- process(
data = med_data,
y = "happiness",
x = "grade",
m = "self_esteem",
model = 4,
bc = 1,
boot = 1000,
seed = 2024
)##
## ********************* PROCESS for R Version 4.3.1 *********************
##
## Written by Andrew F. Hayes, Ph.D. www.afhayes.com
## Documentation available in Hayes (2022). www.guilford.com/p/hayes3
##
## ***********************************************************************
##
## Model : 4
## Y : happiness
## X : grade
## M : self_esteem
##
## Sample size: 100
##
## Custom seed: 2024
##
##
## ***********************************************************************
## Outcome Variable: self_esteem
##
## Model Summary:
## R R-sq MSE F df1 df2 p
## 0.5144 0.2646 2.6868 35.2586 1.0000 98.0000 0.0000
##
## Model:
## coeff se t p LLCI ULCI
## constant 1.4995 0.5892 2.5450 0.0125 0.3303 2.6688
## grade 0.5610 0.0945 5.9379 0.0000 0.3735 0.7485
##
## ***********************************************************************
## Outcome Variable: happiness
##
## Model Summary:
## R R-sq MSE F df1 df2 p
## 0.6107 0.3730 2.6613 28.8524 2.0000 97.0000 0.0000
##
## Model:
## coeff se t p LLCI ULCI
## constant 1.9043 0.6055 3.1452 0.0022 0.7026 3.1059
## grade 0.0396 0.1096 0.3612 0.7187 -0.1780 0.2572
## self_esteem 0.6355 0.1005 6.3212 0.0000 0.4360 0.8350
##
## ***********************************************************************
## Bootstrapping progress:
## | | | 0% | | | 1% | |> | 1% | |> | 2% | |>> | 2% | |>> | 3% | |>> | 4% | |>>> | 4% | |>>> | 5% | |>>> | 6% | |>>>> | 6% | |>>>> | 7% | |>>>>> | 7% | |>>>>> | 8% | |>>>>> | 9% | |>>>>>> | 9% | |>>>>>> | 10% | |>>>>>>> | 10% | |>>>>>>> | 11% | |>>>>>>> | 12% | |>>>>>>>> | 12% | |>>>>>>>> | 13% | |>>>>>>>> | 14% | |>>>>>>>>> | 14% | |>>>>>>>>> | 15% | |>>>>>>>>>> | 15% | |>>>>>>>>>> | 16% | |>>>>>>>>>> | 17% | |>>>>>>>>>>> | 17% | |>>>>>>>>>>> | 18% | |>>>>>>>>>>>> | 19% | |>>>>>>>>>>>> | 20% | |>>>>>>>>>>>>> | 20% | |>>>>>>>>>>>>> | 21% | |>>>>>>>>>>>>> | 22% | |>>>>>>>>>>>>>> | 22% | |>>>>>>>>>>>>>> | 23% | |>>>>>>>>>>>>>>> | 23% | |>>>>>>>>>>>>>>> | 24% | |>>>>>>>>>>>>>>> | 25% | |>>>>>>>>>>>>>>>> | 25% | |>>>>>>>>>>>>>>>> | 26% | |>>>>>>>>>>>>>>>> | 27% | |>>>>>>>>>>>>>>>>> | 27% | |>>>>>>>>>>>>>>>>> | 28% | |>>>>>>>>>>>>>>>>>> | 28% | |>>>>>>>>>>>>>>>>>> | 29% | |>>>>>>>>>>>>>>>>>> | 30% | |>>>>>>>>>>>>>>>>>>> | 30% | |>>>>>>>>>>>>>>>>>>> | 31% | |>>>>>>>>>>>>>>>>>>>> | 32% | |>>>>>>>>>>>>>>>>>>>> | 33% | |>>>>>>>>>>>>>>>>>>>>> | 33% | |>>>>>>>>>>>>>>>>>>>>> | 34% | |>>>>>>>>>>>>>>>>>>>>> | 35% | |>>>>>>>>>>>>>>>>>>>>>> | 35% | |>>>>>>>>>>>>>>>>>>>>>> | 36% | |>>>>>>>>>>>>>>>>>>>>>>> | 36% | |>>>>>>>>>>>>>>>>>>>>>>> | 37% | |>>>>>>>>>>>>>>>>>>>>>>> | 38% | |>>>>>>>>>>>>>>>>>>>>>>>> | 38% | |>>>>>>>>>>>>>>>>>>>>>>>> | 39% | |>>>>>>>>>>>>>>>>>>>>>>>> | 40% | |>>>>>>>>>>>>>>>>>>>>>>>>> | 40% | |>>>>>>>>>>>>>>>>>>>>>>>>> | 41% | |>>>>>>>>>>>>>>>>>>>>>>>>>> | 41% | |>>>>>>>>>>>>>>>>>>>>>>>>>> | 42% | |>>>>>>>>>>>>>>>>>>>>>>>>>> | 43% | |>>>>>>>>>>>>>>>>>>>>>>>>>>> | 43% | |>>>>>>>>>>>>>>>>>>>>>>>>>>> | 44% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 44% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 45% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 46% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 46% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 47% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 48% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 48% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 49% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 49% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 50% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 51% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 51% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 52% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 52% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 53% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 54% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 54% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 55% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 56% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 56% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 57% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 57% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 58% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 59% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 59% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 60% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 60% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 61% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 62% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 62% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 63% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 64% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 64% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 65% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 65% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 66% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 67% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 67% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 68% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 69% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 70% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 70% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 71% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 72% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 72% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 73% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 73% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 74% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 75% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 75% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 76% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 77% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 77% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 78% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 78% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 79% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 80% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 80% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 81% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 82% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 83% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 83% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 84% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 85% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 85% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 86% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 86% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 87% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 88% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 88% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 89% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 90% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 90% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 91% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 91% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 92% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 93% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 93% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 94% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 94% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 95% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 96% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 96% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 97% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 98% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 98% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> | 99% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 99% | |>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>| 100%
##
## **************** DIRECT AND INDIRECT EFFECTS OF X ON Y ****************
##
## Direct effect of X on Y:
## effect se t p LLCI ULCI
## 0.0396 0.1096 0.3612 0.7187 -0.1780 0.2572
##
## Indirect effect(s) of X on Y:
## Effect BootSE BootLLCI BootULCI
## self_esteem 0.3565 0.0794 0.2237 0.5378
##
## ******************** ANALYSIS NOTES AND ERRORS ************************
##
## Level of confidence for all confidence intervals in output: 95
##
## Number of bootstraps for bias-corrected bootstrap confidence intervals: 1000Note that what changes here is the SE and confidence interval around each effect’s estimate. We can see that the 95% confidence interval [.224, .538] is different to what we observed in the Sobel test version of this analysis. These are likely to be more robust, and thus we generally will want to prefer bootstrapping when running mediations.