Discrete Random Variables

Discrete random variables take a finite number of values. For instance, the number obtained when throwing a die(1, 2,3, 4,5,6). It is not possible to get values like 5.1, 5.2, etc. Let $x$ be a discrete random variable that takes value $x_i$ with probability $P_i$ , the sum of probability of every outcome adds up to one.

(1) $\begin{equation*}\sum_i P_=1\end{equation*}$

The mean or expected value of $x$ is:

(2) $\begin{equation*}\langle x\rangle=\sum_ix_iP_i\end{equation*}$

The average of any function $f(x)$ is equal to

(3) $\begin{equation*}\langle f(x)\rangle \sum_i f(x_i)P_i\end{equation*}$

The mean squared value of $x$ is defined as

(4) $\begin{equation*}\langle x^2\rangle = \sum_i x^2P_i\end{equation*}$

Note that $\langle x^2 \rangle \ne {\langle x\langle}^2$ .

Example (3.2)

Let x take values 0, 1, and 2 with probabilities 1/2, 1/4, and 1/4 respectively. Calculate $\langle x\rangle$ and ${\langle x\rangle}^2$

Solution

Recall that the expected value of a function is equal to

$\langle x\rangle = \sun_i x_i P_i$

where $x_i$ is the value of the outcome, and $P_i$ is the probability of obtaining the result. Given three possible outcomes and their corresponding probabilities, the expected value is equal to:

$\langle x\rangle=(0)\frac{1}{2}+(1)\frac{1}{4}+(2)\frac{1}{4}=\frac{3}{4}$

where $x_i$ is the value of the outcome, and $P_i$ is the probability of obtaining the result. Given three possible outcomes and their corresponding probabilities, the expected value is equal to:

$\langle x\rangle=(0)\frac{1}{2}+(1)\frac{1}{4}+(2)\frac{1}{4}=\frac{3}{4}$

The mean squared value of $x$ is equal to

$\langle x^2\rangle=\sum_i x_i^2P_i = 0(\frac{1}{2})+1^2(\frac{1}{4})+2^2(\frac{1}{4})=\frac{5}{4}$

Continuous Probability Distribution

Continuous random variables can take a range of possible values. Examples include the length of time spent in a waiting room, where the quantity is not restricted to a finite set of values. The waiting time can be $61s, 61.1s$ , or $61.000001s$ . The sum of probability is equal to one

(5) $\begin{equation*}\int P(x)dx=1\end{equation*}$

The mean is defined similar to 2, expecting that we are now integrating over a continuous region instead of adding over discrete values.

(6) $\begin{equation*}\langle x\rangle = \int xP(x)dx\end{equation*}$

For any arbitrary continuous function $f(x)$ , the mean value is

(7) $\begin{equation*}\langle f(x)\rangle = \int f(x)P(x)dx\end{equation*}$

The mean square value is defined as

(8) $\begin{equation*}\langle x^2\rangle = \int x^2 P(x)dx\end{equation*}$

Example(3.3): Guassian

Let $P(x) = Ce^{-x^2/2a^2}$ where $C$ and $a$ are constants. This probability is illustrated in Fig. 3.2 and this curve is known as a Gaussian. Calculate $⟨x⟩$ and $⟨x^2⟩$ given this probability distribution.

Solution

Before finding the expected value, the first thing to ensure that the probabilities sum to one is to ensure, allowing us to find the value of constant $c$ . Evaluating the total probabilities, we get:

$1=\int ^\infty_{-\infty} P(x)dx=C\int^\infty_{-\infty} e^{x^2/2a^2}dx=C\sqrt{2\pi a^2}$

Solving for $c$ , we get

$C=\frac{1}{\sqrt{2\pi a^2}}$

The mean of $x$ can be evaluated using:

$\langle x\rangle = C\int ^\infty_{-\infty} xe^{-x^2/2a^2}dx=\frac{1}{\sqrt{2\pi a^2}}\int ^\infty_{-\infty} xe^{-x^2/2a^2}dx$

There are two ways to evaluate the integral. One way is to use the symmetric argument of a even/odd function. Another way is to solve the integral explicitly.

Symmetric argument: Substituting into the integrate $xe^{-x^2/2a^2}$ , one can see that $f(x)=xe^{-x^2/2a^2}=f(-x)$ , indicating an odd function. Evaluating the integral over $(-\infty, \infty)$ , the leftward part of the curve (when $x<0$ ) is below the $x$ -axis, and the rightward part of the curve when $x>0$ is above the $x$ -axis. Both parts have the same area under the curve, except one is negative and one is positive. Adding up the two regions, the total “area” is zero.
Solve explicitly: Using u-substitution, we get $u=-x^2/2a^2$ and $du=-x/a^2dx$ . The integral can be rewritten as:

(9) $\begin{equation*} \langle x\rangle = \frac{1}{\sqrt{2\pi a^2}}\int ^\infty_{-\infty} xe^{-x^2/2a^2}dx \=-\frac{a^2}{\sqrt{2\pi a^2}}\int ^\infty_{-\infty} -\frac{x}{a^2}e^{-x^2/2a^2}dx \ =-\frac{a^2}{\sqrt{2\pi a^2}}\int^\infty_{-\infty} e^udu \ =-\frac{a^2}{\sqrt{2\pi a^2}}e^{-x^2/2a^2}|^\infty_{-\infty} \end{equation*}$

The exponent term $x^2$ is always positive, so as $x\rightarrow \pm\infty$ , $x^2/2a^2\rightarrow 0$ . Evaluating the integral, we get
$\langle x\rangle= \frac{1}{\sqrt{2\pi a^2}}\int ^\infty_{-\infty} xe^{-x^2/2a^2}dx = 0$
The exponent term $x^2$ is always positive, so as $x\rightarrow \pm\infty$ , $x^2/2a^2\rightarrow 0$ . Evaluating the integral, we get
$\langle x\rangle= \frac{1}{\sqrt{2\pi a^2}}\int ^\infty_{-\infty} xe^{-x^2/2a^2}dx = 0$

\end{enumerate}
The square mean value $\langle x^2\rangle$ is computed using
$\langle x^2\rangle = \int^\infty_{-\infty} Cx^2e^{x^2/2a^2}= \frac{1}{\sqrt{2\pi a^2}}\int^\infty_{-\infty} x^2e^{x^2/2a^2}$

We solve the integral using integral by parts. Let $u=x$ , we get
$u=x, \quad dv=xe^{-x^2/2a^2}dx, \quad du=dx, v=-a^2e^{-x^2/2a^2}$

Then, the integral is equal to
$\frac{1}{\sqrt{2\pi a^2}}\Big([-xe^{-x^2/2a^2}]^\infty_{-\infty}-\int^\infty_{-\infty}-a^2 e^{x^2/2a^2}dx\Big) =\frac{1}{\sqrt{2\pi a^2}} \Big (a^2\int^\infty_{\infty}-e^{x^2/2a^2}dx\Big) = =\frac{1}{\sqrt{2\pi a^2}} a^2\Big (\int^\infty_{\infty}-e^{x^2/2a^2}dx\Big)$

Does the integral look familiar? It is equal to sum of probability evaluated in part (a) multiplied by $a^2$ . From (a), we know that the sum of probability is equal to 1. Then, integral for $\langle x^2\rangle$ is equal to
$\langle x^2\rangle = a^2$

Linear Transformation

Let $y=ax+v$ , the average value of $y$ is given by

(10) $\begin{equation*}\langle y\rangle = a\langle x\rangle+b\end{equation*}$

Example: Linear Transformation

Temperatures in degrees Celsius and degrees Fahrenheit are related by $C=5/9(F-32)$ . Given that the average annual temperature in New York is $54^\circ$ , convert the temperature to degree Celsius using.

Solution

We know that

$\langle y\rangle = a\langle x\rangle+b$

Then,

$\langle y\rangle = \frac{5}{9}\langle F\rangle-32=\frac{5}{9}54-32=12^\circ$

Variance

We quantify the spread of values in a distribution by considering the deviation from
the mean for a particular value of $x$ , defined as

(11) $\begin{equation*}x-\langle x\rangle\end{equation*}$

The quantity tells us how much a particular value is above or below the mean value. Using linear transformation, the average of the deviation for all values of $x$ is:

(12) $\begin{equation*}\langle x-\langle x \rangle\rangle = \langle x\rangle-\langle\langle x\rangle\rangle = \langle x \rangle - \langle x \rangle = 0\end{equation*}$

Variance $\sigma^2_x$ is defined as

(13) $\begin{equation*}\sigma^2_x = \langle (x-\langle x\rangle)^2\rangle\end{equation*}$

where $\sigma_x$ is defined as the standard deviation and is the square root of the variance. The stand deviation presents the root mean square (rms) scatter in the data. A useful identity is:

(14) $\begin{align*}\sigma^2_x & = \langle (x-\langle x\rangle)^2\rangle \nonumber\ = \langle x^2-2x\langle x\rangle +\langle x\rangle ^2\rangle \nonumber\ = \langle x^2\rangle - 2\langle x\rangle \langle x\rangle + \langle x \rangle^2 \nonumber\ = \langle x^2\rangle - \langle x \rangle^2\end{align*}$

Independent Variables

If $u$ and $v$ are independent variables, the probability that $u$ is in the range of $u+du$ and $v$ is in the range of $v+dv$ is given by

(15) $\begin{equation*}P_u(u)duP_v(v)dv\end{equation*}$

The average value of the product is
\begin{euqation}
\langle uv \rangle = \iint P_u(u)duP_v(v)dv
\end{euqation}

Because $u$ and $v$ are independent, $P_v(v)$ is a function of $v$ only, and $P_u(d)$ is a function of $u$ only. Then, the double integral can be rewritten as:

(16) $\begin{align*}\langle uv \rangle & = \int P_u(u)du\int P_v(v)dv = \langle u \rangle \langle v \rangle\end{align*}$

Because integrates separate for independent random variables, the average value of the product of $u$ and $v$ are the product of their average values.

Binomial Random Variables

A binary variable is a variable that has two possible outcomes. For example, sex (male/female). The binomial distribution is a special discrete distribution where there are two distinct complementary outcomes: “success” and “failure”. The distribution is the discrete probability distribution $P(n,k)$ of getting $k$ successes from $n$ independent trials. \

Suppose that there are $n$ trails, with a probability of success $p$ . If there are $k$ trails of success, the number of trails for failure is $n-k$ , and the probability of failure is ( $1-p)$ . The number of ways of getting $k$ success from $n$ trials is given by $^kC_n$ . Then, the probability of $k$ trials of success and $(n-k)$ trials of failure is

(17) $\begin{equation*}P(n, k)=^nC_kp^k(1-p)^{n-k}\end{equation*}$

Since the binomial distribution is the sum of n independent Bernoulli trials, then

(18) $\begin{equation*}\langle k\rangle=np, \quad \sigma_k^2=np(1-p)\end{equation*}$

Example (3.9)

Coin tossing with a fair coin. In this case $p=1/2$ . Calculate the expected number of heads for $n=16$ and for $n=10^{20}$ .