Sampling Error and Non-Sampling Error

Whenever conclusions about a population are drawn from a sample rather than a census, some amount of error is unavoidable. Statisticians distinguish two broad kinds:

Sampling error is the difference between a sample statistic (e.g., the sample mean $\bar{x}$) and the true population parameter (e.g., the population mean $\mu$) that arises only because a sample, and not the whole population, was examined. It exists even when every step of the survey is carried out perfectly, simply because different random samples of the same size, drawn from the same population, will not all give exactly the same statistic. Sampling error:

• Is present in every sample survey, but absent in a census (there is no sampling involved).
• Decreases as the sample size $n$ increases — a larger sample tracks the population more closely; in the limit, when $n$ equals the population size, sampling error becomes zero.
• Can be measured and controlled using probability theory (this is exactly what the standard error, covered next, quantifies).

Non-sampling error covers every other source of error — it can arise in a census just as easily as in a sample survey, and, unlike sampling error, it generally does not shrink as the sample size grows (in fact it can grow, since a bigger survey is harder to supervise closely). Common sources include:

• Faulty questionnaire design — ambiguous or leading questions.
• Non-response — selected respondents refuse to answer or cannot be contacted.
• Interviewer bias — the investigator's own views colour how a question is asked or a response is recorded.
• Recording and processing errors — mistakes in writing down, coding, or tabulating responses. …