4.3 Types of error

Nothing in life is perfect, and that applies to the inferential statistics that we do. Every sample will differ slightly from one to another due to inherent sampling error; in a similar way, whenever we do an inferential test about a population from a sample, we always run the risk of making an error in the decisions that we make.

4.3.1 Statistical error

Imagine an old man is seeing his local GP complaining of a headache. Upon examination, the doctor concludes that the old man is pregnant.

A pregnant woman in her last trimester then comes in to the same GP. Despite her numerous pregnancy-related complaints, the doctor concludes that she is not pregnant.

Both of these scenarios (while hopefully very unlikely!) are obviously forms of errors on the doctor’s part. In the first scenario, the doctor has accepted a diagnosis that is very clearly wrong. In the second scenario, the doctor has rejected the correct diagnosis.

The same kind of logic applies directly to quantitative research. We want to be sure that when we observe a result, that result is actually likely. We therefore want to minimise the possibility of errors like above.

We can draw a table to illustrate the possible outcomes when we perform a given hypothesis test:

	Accept the null	Reject the null
The null is true	Correctly accept the null	Type I error
The alternative is true	Type II error	Correctly reject the null

4.3.2 Type I error and alpha

A moment ago we talked about alpha (\(\alpha\)), or the significance level, from the previous sections about hypothesis testing and the p-value. Alpha is the same here as it is there - it is the probability of making a Type I error - that we incorrectly reject the null hypothesis when the null hypothesis is actually true. Essentially, alpha is the rate of Type I error we’re willing to accept whenever we do a hypothesis test.

We generally set alpha as p < .05 out of convention - i.e. most of the time, we’re willing to accept a 5% chance of a Type I error rate. However, we can set alpha to anything we want. Sometimes, we may set it lower (more on this in a few weeks’ time).