Normal distribution | Properties, proofs, exercises (2024)

by Marco Taboga, PhD

The normal distribution is a continuous probability distribution that plays a central role in probability theory and statistics.

It is often called Gaussian distribution, in honor of Carl Friedrich Gauss (1777-1855), an eminent German mathematician who gave important contributions towards a better understanding of the normal distribution.

Normal distribution | Properties, proofs, exercises (1)

Table of contents

  1. Why is it so important?

  2. Main characteristics

  3. The standard normal distribution

    1. Definition

    2. Expected value

    3. Variance

    4. Moment generating function

    5. Characteristic function

    6. Distribution function

  4. The normal distribution in general

    1. Definition

    2. Relation between standard and non-standard normal distribution

    3. Expected value

    4. Variance

    5. Moment generating function

    6. Characteristic function

    7. Distribution function

  5. Density plots

    1. Plot 1 - Changing the mean

    2. Plot 2 - Changing the standard deviation

  6. More details

  7. Solved exercises

    1. Exercise 1

    2. Exercise 2

    3. Exercise 3

Why is it so important?

The normal distribution is extremely important because:

Main characteristics

Sometimes it is also referred to as "bell-shaped distribution" because the graph of its probability density function resembles the shape of a bell.

Normal distribution | Properties, proofs, exercises (2)

As you can see from the above plot, the density of a normal distribution has two main characteristics:

  • it is symmetric around the mean (indicated by the vertical line); as a consequence, deviations from the mean having the same magnitude, but different signs, have the same probability;

  • it is concentrated around the mean; it becomes smaller by moving from the center to the left or to the right of the distribution (the so called "tails" of the distribution); this means that the further a value is from the center of the distribution, the less probable it is to observe that value.

The remainder of this lecture gives a formal presentation of the main characteristics of the normal distribution.

First, we deal with the special case in which the distribution has zero mean and unit variance. Then, we present the general case, in which mean and variance can take any value.

The standard normal distribution

The adjective "standard" indicates the special case in which the mean is equal to zero and the variance is equal to one.

Definition

Standard normal random variables are characterized as follows.

Definition Let Normal distribution | Properties, proofs, exercises (3) be a continuous random variable. Let its support be the whole set of real numbers:Normal distribution | Properties, proofs, exercises (4)We say that Normal distribution | Properties, proofs, exercises (5) has a standard normal distribution if and only if its probability density function isNormal distribution | Properties, proofs, exercises (6)

The following is a proof that Normal distribution | Properties, proofs, exercises (7) is indeed a legitimate probability density function:

Proof

The function Normal distribution | Properties, proofs, exercises (8) is a legitimate probability density function if it is non-negative and if its integral over the support equals 1. The former property is obvious, while the latter can be proved as follows:Normal distribution | Properties, proofs, exercises (9)

Expected value

The expected value of a standard normal random variable Normal distribution | Properties, proofs, exercises (10) isNormal distribution | Properties, proofs, exercises (11)

Proof

It can be derived as follows:Normal distribution | Properties, proofs, exercises (12)

Variance

The variance of a standard normal random variable Normal distribution | Properties, proofs, exercises (13) isNormal distribution | Properties, proofs, exercises (14)

Proof

It can be proved with the usual variance formula (Normal distribution | Properties, proofs, exercises (15)):Normal distribution | Properties, proofs, exercises (16)

Moment generating function

The moment generating function of a standard normal random variable Normal distribution | Properties, proofs, exercises (17) is defined for any Normal distribution | Properties, proofs, exercises (18):Normal distribution | Properties, proofs, exercises (19)

Proof

It is derived by using the definition of moment generating function:Normal distribution | Properties, proofs, exercises (20)The integral above is well-defined and finite for any Normal distribution | Properties, proofs, exercises (21). Thus, the moment generating function of Normal distribution | Properties, proofs, exercises (22) exists for any Normal distribution | Properties, proofs, exercises (23).

Characteristic function

The characteristic function of a standard normal random variable Normal distribution | Properties, proofs, exercises (24) isNormal distribution | Properties, proofs, exercises (25)

Proof

By the definition of characteristic function, we haveNormal distribution | Properties, proofs, exercises (26)Now, take the derivative with respect to Normal distribution | Properties, proofs, exercises (27) of the characteristic function:Normal distribution | Properties, proofs, exercises (28)By putting together the previous two results, we obtainNormal distribution | Properties, proofs, exercises (29)The only function that satisfies this ordinary differential equation (subject to the condition Normal distribution | Properties, proofs, exercises (30)) isNormal distribution | Properties, proofs, exercises (31)

Distribution function

There is no simple formula for the distribution function Normal distribution | Properties, proofs, exercises (32) of a standard normal random variable Normal distribution | Properties, proofs, exercises (33) because the integralNormal distribution | Properties, proofs, exercises (34)cannot be expressed in terms of elementary functions. Therefore, it is usually necessary to resort to special tables or computer algorithms to compute the values of Normal distribution | Properties, proofs, exercises (35). The lecture entitled Normal distribution values discusses these alternatives in detail.

The normal distribution in general

While in the previous section we restricted our attention to the special case of zero mean and unit variance, we now deal with the general case.

Definition

The normal distribution with mean Normal distribution | Properties, proofs, exercises (36) and variance Normal distribution | Properties, proofs, exercises (37) is characterized as follows.

Definition Let Normal distribution | Properties, proofs, exercises (38) be a continuous random variable. Let its support be the whole set of real numbers:Normal distribution | Properties, proofs, exercises (39)Let Normal distribution | Properties, proofs, exercises (40) and Normal distribution | Properties, proofs, exercises (41). We say that Normal distribution | Properties, proofs, exercises (42) has a normal distribution with mean Normal distribution | Properties, proofs, exercises (43) and variance Normal distribution | Properties, proofs, exercises (44) if and only if its probability density function isNormal distribution | Properties, proofs, exercises (45)

We often indicate the fact that Normal distribution | Properties, proofs, exercises (46) has a normal distribution with mean Normal distribution | Properties, proofs, exercises (47) and variance Normal distribution | Properties, proofs, exercises (48) byNormal distribution | Properties, proofs, exercises (49)

To better understand how the shape of the distribution depends on its parameters, you can have a look at the density plots at the bottom of this page.

Relation between standard and non-standard normal distribution

The following proposition provides the link between the standard and the general case.

Proposition If Normal distribution | Properties, proofs, exercises (50) has a normal distribution with mean Normal distribution | Properties, proofs, exercises (51) and variance Normal distribution | Properties, proofs, exercises (52), thenNormal distribution | Properties, proofs, exercises (53)where Normal distribution | Properties, proofs, exercises (54) is a random variable having a standard normal distribution.

Proof

This can be easily proved using the formula for the density of a function of a continuous variable (Normal distribution | Properties, proofs, exercises (55) is a strictly increasing function of Normal distribution | Properties, proofs, exercises (56), since Normal distribution | Properties, proofs, exercises (57) is strictly positive):Normal distribution | Properties, proofs, exercises (58)

Thus, a normal distribution is standard when Normal distribution | Properties, proofs, exercises (59) and Normal distribution | Properties, proofs, exercises (60).

Expected value

The expected value of a normal random variable Normal distribution | Properties, proofs, exercises (61) isNormal distribution | Properties, proofs, exercises (62)

Proof

The proof is a straightforward application of the fact that Normal distribution | Properties, proofs, exercises (63) can we written as a linear function of a standard normal variable:Normal distribution | Properties, proofs, exercises (64)

Variance

The variance of a normal random variable Normal distribution | Properties, proofs, exercises (65) isNormal distribution | Properties, proofs, exercises (66)

Proof

It can be derived as follows:Normal distribution | Properties, proofs, exercises (67)

Moment generating function

The moment generating function of a normal random variable Normal distribution | Properties, proofs, exercises (68) is defined for any Normal distribution | Properties, proofs, exercises (69):Normal distribution | Properties, proofs, exercises (70)

Proof

The mgf is derived as follows:Normal distribution | Properties, proofs, exercises (71)It is defined for any Normal distribution | Properties, proofs, exercises (72) because the moment generating function of Normal distribution | Properties, proofs, exercises (73) is defined for any Normal distribution | Properties, proofs, exercises (74).

Characteristic function

The characteristic function of a normal random variable Normal distribution | Properties, proofs, exercises (75) isNormal distribution | Properties, proofs, exercises (76)

Proof

The derivation is similar to the derivation of the moment generating function:Normal distribution | Properties, proofs, exercises (77)

Distribution function

The distribution function Normal distribution | Properties, proofs, exercises (78) of a normal random variable Normal distribution | Properties, proofs, exercises (79) can be written asNormal distribution | Properties, proofs, exercises (80)where Normal distribution | Properties, proofs, exercises (81) is the distribution function of a standard normal random variable Normal distribution | Properties, proofs, exercises (82) (see above). The lecture entitled Normal distribution values provides a proof of this formula and discusses it in detail.

Density plots

This section shows the plots of the densities of some normal random variables. These plots help us to understand how the shape of the distribution changes by changing its parameters.

Plot 1 - Changing the mean

The following plot contains the graphs of two normal probability density functions:

  • the first graph (red line) is the probability density function of a normal random variable with mean Normal distribution | Properties, proofs, exercises (83) and standard deviation Normal distribution | Properties, proofs, exercises (84);

  • the second graph (blue line) is the probability density function of a normal random variable with mean Normal distribution | Properties, proofs, exercises (85) and standard deviation Normal distribution | Properties, proofs, exercises (86).

By changing the mean from Normal distribution | Properties, proofs, exercises (87) to Normal distribution | Properties, proofs, exercises (88), the shape of the graph does not change, but the graph is translated to the right (its location changes).

Normal distribution | Properties, proofs, exercises (89)

Plot 2 - Changing the standard deviation

The following plot shows two graphs:

  • the first graph (red line) is the probability density function of a normal random variable with mean Normal distribution | Properties, proofs, exercises (90) and standard deviation Normal distribution | Properties, proofs, exercises (91);

  • the second graph (blue line) is the probability density function of a normal random variable with mean Normal distribution | Properties, proofs, exercises (92) and standard deviation Normal distribution | Properties, proofs, exercises (93).

By increasing the standard deviation from Normal distribution | Properties, proofs, exercises (94) to Normal distribution | Properties, proofs, exercises (95), the location of the graph does not change (it remains centered at Normal distribution | Properties, proofs, exercises (96)), but the shape of the graph changes (there is less density in the center and more density in the tails).

Normal distribution | Properties, proofs, exercises (97)

More details

The following lectures contain more material about the normal distribution.

Normal distribution values

How to tackle the numerical computation of the distribution function

Multivariate normal distribution

A multivariate generalization of the normal distribution, frequently encountered in statistics

Quadratic forms involving normal variables

Discusses the distribution of quadratic forms involving normal random variables

Linear combinations of normal variables

Discusses the important fact that normality is preserved by linear combinations

Solved exercises

Below you can find some exercises with explained solutions.

Exercise 1

Let Normal distribution | Properties, proofs, exercises (98) be a normal random variable with mean Normal distribution | Properties, proofs, exercises (99) and variance Normal distribution | Properties, proofs, exercises (100). Compute the following probability:Normal distribution | Properties, proofs, exercises (101)

Solution

First of all, we need to express the above probability in terms of the distribution function of Normal distribution | Properties, proofs, exercises (102):Normal distribution | Properties, proofs, exercises (103)

Then, we need to express the distribution function of Normal distribution | Properties, proofs, exercises (104) in terms of the distribution function of a standard normal random variable Normal distribution | Properties, proofs, exercises (105):Normal distribution | Properties, proofs, exercises (106)

Therefore, the above probability can be expressed asNormal distribution | Properties, proofs, exercises (107)where we have used the fact that Normal distribution | Properties, proofs, exercises (108), which has been presented in the lecture entitled Normal distribution values.

Exercise 2

Let Normal distribution | Properties, proofs, exercises (109) be a random variable having a normal distribution with mean Normal distribution | Properties, proofs, exercises (110) and variance Normal distribution | Properties, proofs, exercises (111). Compute the following probability:Normal distribution | Properties, proofs, exercises (112)

Solution

We need to use the same technique used in the previous exercise (express the probability in terms of the distribution function of a standard normal random variable):Normal distribution | Properties, proofs, exercises (113)where we have found the value Normal distribution | Properties, proofs, exercises (114) in a normal distribution table.

Exercise 3

Suppose the random variable Normal distribution | Properties, proofs, exercises (115) has a normal distribution with mean Normal distribution | Properties, proofs, exercises (116) and variance Normal distribution | Properties, proofs, exercises (117). Define the random variable Normal distribution | Properties, proofs, exercises (118) as follows:Normal distribution | Properties, proofs, exercises (119)Compute the expected value of Normal distribution | Properties, proofs, exercises (120).

Solution

Remember that the moment generating function of Normal distribution | Properties, proofs, exercises (121) isNormal distribution | Properties, proofs, exercises (122)Therefore, using the linearity of the expected value, we obtainNormal distribution | Properties, proofs, exercises (123)

How to cite

Please cite as:

Taboga, Marco (2021). "Normal distribution", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/probability-distributions/normal-distribution.

Normal distribution | Properties, proofs, exercises (2024)

References

Top Articles
Latest Posts
Article information

Author: Van Hayes

Last Updated:

Views: 5888

Rating: 4.6 / 5 (66 voted)

Reviews: 81% of readers found this page helpful

Author information

Name: Van Hayes

Birthday: 1994-06-07

Address: 2004 Kling Rapid, New Destiny, MT 64658-2367

Phone: +512425013758

Job: National Farming Director

Hobby: Reading, Polo, Genealogy, amateur radio, Scouting, Stand-up comedy, Cryptography

Introduction: My name is Van Hayes, I am a thankful, friendly, smiling, calm, powerful, fine, enthusiastic person who loves writing and wants to share my knowledge and understanding with you.