Double integrals in probability

Often, we are interested in two random variables $X$ and $Y$, and want to understand how they are related. For instance, $X$ might be tomorrow's high temperature, and $Y$ might be tomorrow's barometric pressure. Or $X$ might be the concentration of virus in a patient's blood, and $Y$ might be the concentration of an antibody. Or $X$ and $Y$ might be the horizontal and vertical coordinates of the spot where a dart hits a board.

When we have two random variables $X$ and $Y$, the pair $(x,y)$ takes values in the plane, and we speak of the probability per unit area of $(x,y)$ landing in a certain region. This is called the joint probability density function, and is written $f_{X,Y}(x,y)$.

The probability of $(X,Y)$ landing in a region $R$ is $$P((X,Y) \in R) = \iint_R f_{X,Y}(x,y) dA.$$ The average value of $X$ is $$E(X) = \iint x f_{X,Y}(x,y) dA,$$ where we integrate over all possible values of $(X,Y)$.

If we know the joint distribution of $X$ and $Y$, we can compute the distribution of just $X$ or just $Y$. These are called marginal distributions, because for discrete random variables they are often written on the margins of a table. We get $f_X(x)$ by integrating $f_{X,Y}(x,y)$ over $y$, and $f_Y(y)$ by integrating $f_{X,Y}(x,y)$ over $x$: $$\begin{eqnarray*} f_X(x) & = & \int_{-\infty}^\infty f_{X,Y}(x,y) dy \\ f_Y(y) & = & \int_{-\infty}^\infty f_{X,Y}(x,y) dx \end{eqnarray*}$$

If $\mu_X$ and $\mu_Y$ are the average values of $X$ and $Y$, then the covariance of $X$ and $Y$ is $$Cov(X,Y) = E((X-\mu_X)(Y-\mu_Y)) = \iint (x-\mu_x)(y-\mu_y) f_{X,Y}(x,y) dA.$$ Just like the variance of one variable, this is more easily computed as $$Cov(X,Y) = E(XY) - \mu_X\mu_Y = \iint xy f_{X,Y}(x,y) dA - \mu_X\mu_Y.$$

The correlation between $X$ and $Y$ is $$Cor(X,Y) = \frac{Cov(X,Y)}{\sigma_X\sigma_Y}.$$ This is a number, often written $r$, between $-1$ and $1$. If $r$ is close to 1, then all the likely values of $(X,Y)$ lie close to a straight line of positive slope. In that case we say that $X$ and $Y$ are positively correlated. Increases in $X$ are then associated with increases in $Y$, and vice-versa. This is statistical evidence that $X$ and $Y$ are somehow related. Maybe $X$ causes $Y$, or maybe $Y$ causes $X$, or maybe something else causes both of them.

If $r$ is close to $-1$, then the likely values of $(X,Y)$ lie close to a line of negative slope, so increasing $X$ tends to decrease $Y$, and vice-versa. $X$ and $Y$ are said to be negatively correlated. Again, this is evidence that $X$ and $Y$ are related.

If $r$ is close to 0, then $X$ and $Y$ are said to be uncorrelated.

Example: Suppose that two numbers are chosen uniformly from the interval $[0,1]$. We call the larger of the two numbers $X$ and the smaller one $Y$. Their joint pdf is $f_{X,Y}(x,y)= 2$ if $0 \le y \le x \le 1$ and $0$ otherwise. Find the average value of $X$ and the marginal distribution of $X$.

Solution: We first compute the average value of $X$: $$\begin{eqnarray*} E(X) &=& \int_0^1 \int_0^1 x f_{X,Y}(x,y) dy dx\\ & = & \int_0^1 \int_0^x 2x dy dx \\ & = & \int_0^1 2x^2 dx \ = \ \frac{2}{3}. \end{eqnarray*}$$ As for the marginal distribution, we have $$f_X(x) = \int_0^1 f_{X,Y}(x,y) dy = \int_0^x 2 dy = 2x.$$