Hypothesis Testing: Framework and Errors

Hypothesis testing is the statistical procedure used to decide, on the basis of sample evidence, whether a claim (or assumption) about a population is likely to be true. Every test begins with two competing statements:

• The null hypothesis ($H_0$) — the claim of "no difference" or "no effect" that is assumed true unless the sample evidence strongly contradicts it (e.g., $H_0: \mu = 500$ — the mean fill-weight of packets is 500 gm, as specified).
• The alternative hypothesis ($H_1$) — the statement accepted if $H_0$ is rejected (e.g., $H_1: \mu \neq 500$).

Because the decision is based on a sample rather than the whole population, two kinds of mistakes are possible:

• A Type I error is rejecting a null hypothesis that is actually true (a "false alarm" — e.g., concluding a good production batch is defective).
• A Type II error is failing to reject a null hypothesis that is actually false (a "missed detection" — e.g., passing a genuinely defective batch as good).

The probability of a Type I error that the analyst is willing to accept is called the level of significance, denoted $\alpha$ — commonly 5% or 1%. A test can be:

• Two-tailed, when $H_1$ simply states the parameter is different from the claimed value ($H_1: \mu \neq \mu_0$) — rejection can happen on either side of the distribution, so the critical region is split between both tails.
• One-tailed, when $H_1$ states the parameter is specifically greater than or specifically less than the claimed value ($H_1: \mu > \mu_0$ or $H_1: \mu < \mu_0$) — the entire critical region sits in one tail. …