Expectation and variance

A random variable can take many values, so a full distribution table is often more detail than we need. Sometimes we just want one number that says what to expect 'on average': if a game costs $2 to play and pays out unpredictably, is it worth playing? Expectation answers that.

But an average alone can mislead. Two investments can both average a 5% return, while one barely wobbles and the other swings wildly. Variance is the standard second number: it measures how far outcomes typically sit from the mean. Together, expectation and variance compress a distribution into a useful two-number summary.

Expectation: the long-run average

The expectation (or expected value) of a discrete random variable $X$ is the weighted average of its possible values, where each value is weighted by its probability:

$$E[X]=\sum _{x}x\cdot P(X=x).$$

The word 'expectation' does not mean the outcome you expect to see once. A die never lands on 3.5, yet $E[X]=3.5$. The right picture is the long-run average: if you repeat the experiment many times and average the results, the average settles toward $E[X]$. This is why expectation is the natural tool for decisions repeated over time, like pricing insurance or evaluating a game strategy.

Example 1. Expected value of a die roll A fair six-sided die shows $X\in \{1,2,3,4,5,6\}$, each with probability $1/6$. Find $E[X]$.

Multiply each outcome by its probability and add:

$$E[X]=1\cdot \frac{1}{6}+2\cdot \frac{1}{6}+3\cdot \frac{1}{6}+4\cdot \frac{1}{6}+5\cdot \frac{1}{6}+6\cdot \frac{1}{6}.$$

$$E[X]=\frac{1+2+3+4+5+6}{6}=\frac{21}{6}=\mathrm{3.5.}$$

No single roll gives 3.5, but the average of many rolls approaches it.

Variance: spread around the mean

Variance measures how far $X$ typically is from its mean. It is defined as the average squared distance from the mean:

$$\mathrm{Var}(X)=E\left[(X-E[X]{)}^2\right]=E[X^2]-(E[X]{)}^2.$$

The second form is usually easier for computation: find the average of $X^2$, then subtract the square of the average. The units are squared (dollars${}^2$, cm${}^2$), so for interpretation we often take the square root, called the standard deviation $\sigma =\sqrt{\mathrm{Var}(X)}$, which is back in the original units.

Why square the distances? Squaring makes all deviations positive so they cannot cancel out, and it penalizes large deviations heavily, which is usually what we care about when we talk about risk.

Example 2. Variance of a small bet A game pays $X=0$ with probability $0.5$ and $X=4$ with probability $0.5$. Find $E[X]$ and $\mathrm{Var}(X)$.

First the mean:

$$E[X]=0\cdot 0.5+4\cdot 0.5=2.$$

Then the average of the squares:

$$E[X^2]=0^2\cdot 0.5+4^2\cdot 0.5=8.$$

Now the shortcut formula:

$$\mathrm{Var}(X)=E[X^2]-(E[X]{)}^2=8-2^2=4.$$

Check by the definition: $(0-2{)}^2\cdot 0.5+(4-2{)}^2\cdot 0.5=2+2=4$. Both routes agree. The standard deviation is $\sqrt{4}=2$.

How expectation and variance transform

If you rescale a random variable, its summary statistics rescale in simple, predictable ways. For constants $a$ and $b$:

$$E[aX+b]=aE[X]+b,\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X).$$

Adding $b$ shifts every outcome by the same amount, so the average shifts by $b$ but the spread does not change at all. Multiplying by $a$ stretches distances by $|a|$, and since variance measures squared distances, it picks up a factor $a^2$. These rules let you convert units (say, Celsius to Fahrenheit) without recomputing everything from the distribution.

Example 3. Converting a temperature distribution A temperature in Celsius, $C$, has $E[C]=20$ and $\mathrm{Var}(C)=4$. Let $F=1.8C+32$ be the temperature in Fahrenheit. Find $E[F]$ and $\mathrm{Var}(F)$.

Apply the linearity rule for expectation:

$$E[F]=1.8\cdot E[C]+32=1.8\cdot 20+32=68.$$

Apply the scaling rule for variance; the $+32$ contributes nothing:

$$\mathrm{Var}(F)=1.8^2\cdot \mathrm{Var}(C)=3.24\cdot 4=\mathrm{12.96.}$$

The standard deviation scales by $|1.8|$: ${\sigma }_F=1.8\cdot 2=3.6$.

A complete worked example

Putting it all together: read the distribution, compute $E[X]$, compute $E[X^2]$, then subtract. Always confirm the probabilities sum to 1 before you start, and sanity-check that the mean lies inside the range of values.

Example 4. Full computation from a distribution table A random variable $X$ has $P(X=1)=0.2$, $P(X=2)=0.5$, $P(X=3)=0.3$. Find $E[X]$ and $\mathrm{Var}(X)$.

Check the probabilities sum to 1: $0.2+0.5+0.3=1$. Good.

Compute the mean:

$$E[X]=1\cdot 0.2+2\cdot 0.5+3\cdot 0.3=0.2+1.0+0.9=\mathrm{2.1.}$$

Compute the mean of the squares:

$$E[X^2]=1^2\cdot 0.2+2^2\cdot 0.5+3^2\cdot 0.3=0.2+2.0+2.7=\mathrm{4.9.}$$

Subtract the squared mean:

$$\mathrm{Var}(X)=4.9-(2.1{)}^2=4.9-4.41=\mathrm{0.49.}$$

Check with the definition: $(1-2.1{)}^2\cdot 0.2+(2-2.1{)}^2\cdot 0.5+(3-2.1{)}^2\cdot 0.3=0.242+0.005+0.243=0.49$. The standard deviation is $\sqrt{0.49}=0.7$.

Example 5. Is the game worth playing? A raffle ticket costs $2. With probability $0.1$ you win a $15 prize; otherwise you win nothing. Let $X$ be your net profit per ticket. Find $E[X]$ and interpret it.

Net profit is $15-2=13$ if you win, and $-2$ if you lose:

$$E[X]=13\cdot 0.1+(-2)\cdot 0.9=1.3-1.8=-\mathrm{0.5.}$$

On average you lose 50 cents per ticket. For the spread, compute $E[X^2]$:

$$E[X^2]=13^2\cdot 0.1+(-2{)}^2\cdot 0.9=16.9+3.6=\mathrm{20.5.}$$

$$\mathrm{Var}(X)=20.5-(-0.5{)}^2=20.5-0.25=\mathrm{20.25.}$$

The variance is large relative to the mean, so individual outcomes (winning $13$ or losing $2$) differ a lot from the average loss of $0.5$.

Common mistakes

• Computing $E[X^2]$ as $(E[X]{)}^2$; these are different, and the gap between them is exactly the variance.
• Forgetting the $+b$ term contributes nothing to variance, writing $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)+b$; the correct rule is $\mathrm{Var}(aX+b)=a^2\mathrm{Var}(X)$.
• Averaging the outcomes without weights, e.g. $(1+2+3)/3=2$ when the probabilities are $0.2,0.5,0.3$; the correct mean is $1(0.2)+2(0.5)+3(0.3)=2.1$.
• Reporting variance in the original units; variance is in squared units, so take $\sqrt{\mathrm{Var}(X)}$ for the standard deviation.
• Using probabilities that do not sum to 1, which makes every downstream statistic meaningless; always verify ${\sum }_{x}P(X=x)=1$ first.

Practice

1. A random variable $X$ takes values $2$ and $5$, each with probability $0.5$. Find $E[X]$.
2. A random variable $X$ takes values $1,2,3$ with probabilities $0.2,0.5,0.3$ respectively. Find $\mathrm{Var}(X)$.
3. A fair six-sided die is rolled and the result is $X$. Find the standard deviation of $X$, i.e. $\sqrt{\mathrm{Var}(X)}$. Give the exact value as a square root.
4. A random variable $X$ has $E[X]=4$ and $\mathrm{Var}(X)=1$. Let $Y=2X+3$. Find $\mathrm{Var}(Y)$.
5. A lottery ticket costs $5. With probability $0.05$ it pays a $100 prize; otherwise it pays nothing. Let $X$ be the net profit from buying one ticket (prize minus cost). Find $E[X]$.

Answers

1. $E[X]=2(0.5)+5(0.5)=3.5$.
2. $E[X]=2.1$ and $E[X^2]=4.9$, so $\mathrm{Var}(X)=4.9-2.1^2=0.49$.
3. $E[X]=3.5$ and $\mathrm{Var}(X)=35/12$, so the standard deviation is $\sqrt{35/12}$.
4. Only the multiplier $2$ matters for spread: $\mathrm{Var}(Y)=2^2\cdot 1=4$.
5. Net profit is $95$ with probability $0.05$ and $-5$ with probability $0.95$, so $E[X]=95(0.05)-5(0.95)=4.75-4.75=0$.

With expectation and variance in hand, the next lesson on estimation and regression shows how to use sample data to estimate these quantities and model relationships between variables.