Variance and Standard Deviation (Ungrouped Data)

Variance is another way of averaging the deviations of the observations from the mean while avoiding the identical-cancellation problem noted in the previous section — but instead of taking the absolute value of each deviation, variance takes its square.

Why square the deviations, rather than use the absolute value again? Squaring, like taking the absolute value, destroys the sign of each deviation (since $(-3)^2=3^2=9$), so squared deviations never cancel to zero on averaging, exactly like absolute deviations. But the square has two further advantages that make it the preferred choice throughout higher statistics and probability: (i) it is a smooth, algebraically manipulable function (unlike $|\cdot|$, which has a sharp corner at zero and is awkward to differentiate or combine algebraically), which becomes essential once statistics is connected to calculus and probability theory later in the course; and (ii) squaring gives proportionately more weight to larger deviations — an observation twice as far from the mean contributes four times as much to the variance, which is often a desirable property when large deviations are considered especially significant.

Variance, for ungrouped data. If $x_1,x_2,\ldots,x_n$ has mean $\bar x$, the variance (often denoted $\sigma^2$ or $\text{Var}(X)$) is

Standard deviation. Because variance is expressed in the square of the original unit (e.g. marks$^2$, or kg$^2$), it is not directly comparable to the data itself. Taking the positive square root restores the original unit, giving the standard deviation:

Standard deviation is the single most widely used measure of dispersion in statistics.

A useful computational shortcut. Expanding the square in the variance formula gives an equivalent, often faster, way to compute it directly from $\sum x_i$ and $\sum x_i^2$, without first subtracting the mean from every single observation:

$\sigma^2 = \dfrac1n\sum(x_i-\bar x)^2 = \dfrac1n\sum\big(x_i^2-2\bar x\, x_i+\bar x^2\big) = \dfrac1n\sum x_i^2 - 2\bar x\cdot\dfrac1n\sum x_i + \bar x^2 = \dfrac1n\sum x_i^2 - 2\bar x^2+\bar x^2,$ …