Normal distribution

The normal distribution, also known as the Gaussian distribution, is a theoretical continuous distribution of a random variable - and is mathematically defined by several formulae. For non-mathematicians, a qualitative description of its properties may be more useful.

The normal distribution was so named because it was thought to be the natural or normal distribution for any continuous variable to follow. We now know that, in biology at least, that is not necessarily the case. But in statistics the distribution remains extremely important because it more-or-less describes the random variation of sample means - and many statistics that behave as means.

{Fig. 1}

Whilst many people visualise a normal distribution as a 'bell-shaped' curve, for a critical appraisal you need to define its properties much more clearly and quantitatively. Let us begin by stating the properties of the distribution.

• Any truly normal distribution has a maximum of infinity and a minimum of minus infinity - and, having an infinite range, is therefore unbounded.

• If you randomly select a value from a normal distribution, that value can be any number between minus and plus infinity. Moreover, because the probability obtaining a predefined value of a is vanishingly small, the probability of obtaining the same observation twice is effectively zero. As a result, every observation of a normal population is unique. In other words, because no two observations can be the same, so a truly normal population has no ties. Consequently, the normal distribution is smooth and continuous.

  In order to have these properties, a completely normal population must be infinitely large. From this it follows that, although a sample of a normal population might have almost any distribution, if a sample contains a finite number of observations it cannot be perfectly normal!

• The normal distribution is completely symmetrical so the mean and median are identical. So, although the mean and the median of a sample of that population may be unequal, on average (given an infinite number of samples of that population) they would be the same as each other - and the same as the population from which they were drawn - provided their selection was unbiased.

• Lastly, if a population is normal, its distribution is defined by two - and only two - values. The location of the population described by the population mean, and the dispersion of the population, described by the population standard deviation. These are known as the population parameters, and methods which assume your observations are normal are known as parametric.

Let us consider how varying these population parameters affects the appearance of the distribution.

The population's location

Because the normal distribution is smooth and symmetrical, the mean, median, and mode of any normal population are identical. The graph below shows the distribution of 3 normal populations, whose only difference is their location.

{Fig. 2}

The population mean is usually defined as the mean of all the values in that population - or μ. More explicitly, if we call our population X, μ_x would be the population mean. In contrast, the mean of a sample of that population, is only an estimate of the 'true' population mean, and the two are only the same on average.

Another way of defining the population mean is in terms of the average result of randomly sampling that population. For example, if x is an observation of population X then, if we took an infinitely large sample its mean would be Σx/∞ - which causes a few headaches! To avoid this dilemma, we say that, if we repeatedly sample a population, we would expect the average value of x, E(x) to be identical to the population mean, μ_x. In other words, μ_x = E(x) = E() - or various other mathematical formulae, depending upon the context.

If the probability of observing a value was unrelated to the value being observed the distribution would be uniform. However, if you sample a normal population at random, the most commonly observed values are closest to the population mean.

The population's dispersion

With a normal population, for mathematical reasons, the dispersion is usually defined in terms of the root mean squared deviation of its observations about their population mean - in other words the population standard deviation, σ. The graph below shows the distribution of 3 normal populations, whose only difference is their dispersion.

{Fig. 3}

By convention, the standard deviation of a population called Y is generally represented by the Greek letter s - in other words σ_y - or just σ. The standard deviation of a sample of that population may be written as s_y, or just s.

Aside from their mean and standard deviation, every normal population is identical. Therefore, if you rescale normal populations to allow for these two parameters, every normal population is completely identical. The commonest way to rescale a normal population is to subtract the mean from each observation, and divide by the standard deviation. This will produce a standard normal population, which has a mean of zero and a standard deviation of one. This is mathematically the simplest of all - and, because is so useful, the standard normal distribution has its own special symbols and terminology.

Central limit

As we said above, the main reason why the normal distribution is so important in statistics is that many sample statistics, including the mean, tend towards a normal distribution, irrespective of the population distribution. The way in which a statistic's normal tendency depends upon sample size is described by what is known as central limit theorem. Non-mathematically, there are three factors which determine how large a sample you need in order to assume a statistic is approximately normal.

• Which statistic you are using - although you may not have much choice in this matter.
• How normal is the population that is sampled - which, to some extent, depends upon what sort of variable you are dealing with.
• How 'approximate', or unrealistic, an answer you and your critics are prepared to accept.

Another reason the normal distribution is so popular is because its properties are well known - at least to mathematicians. In particular, there are various formulae which estimate the proportion of a normal population in a defined interval. These are known as probability functions.

Related topics :

Skewness and kurtosis

Chauvenet's criterion for identifying outliers

The log normal distribution

Gaussian smoothing