Conditional expectation

$\mathbb E[X\mid Y]$ was defined as a function of $Y$. Martingale theory needs conditioning on "everything known at time $n$", which is best encoded as a σ-algebra.

1. Information as a σ-algebra (Def. 1.1).
2. Conditional expectation (Def. 1.2), uniqueness (Prop. 1.3) and its rules (Thm 1.4).
3. Conditional Jensen (Prop. 1.5) and the projection picture (Thm 1.6), with examples (Example 1.7).

A σ-algebra $\mathcal G$ on $\Omega$ is a collection of events containing $\Omega$ and closed under complements and countable unions. For a countable partition $\{B_i\}$ of $\Omega$, $\sigma(\{B_i\})$ consists of all unions of blocks. For a random variable $X$, $\sigma(X) = \{\{X\in A\}: A\text{ Borel}\}$; if $X$ is discrete it is generated by the partition $\{X = x\}$. A random variable $Z$ is $\mathcal G$-measurable if $\{Z\le z\}\in\mathcal G$ for all $z$; for a partition this means $Z$ is constant on each block. Think of $\mathcal G$ as "what you know".

Let $\mathbb E|X| < \infty$. A conditional expectation of $X$ given $\mathcal G$ is a random variable $Z$ with (i) $Z$ $\mathcal G$-measurable and integrable, and (ii) $\mathbb E[X\mathbf 1_A] = \mathbb E[Z\mathbf 1_A]$ for every $A\in\mathcal G$. We write $Z = \mathbb E[X\mid\mathcal G]$. For a partition $\{B_i\}$ with $\mathbb P(B_i) > 0$,

the average of $X$ over whichever block occurred. (For general $\mathcal G$, existence follows from the Radon–Nikodym theorem, or from the projection picture (see Theorem 1.6) and approximation.)

If $Z$ and $Z'$ both satisfy Definition 1.2, then $Z = Z'$ almost surely.

For integrable $X, Y$ and σ-algebras $\mathcal H\subseteq\mathcal G$: (a) linearity: $\mathbb E[aX + bY\mid\mathcal G] = a\mathbb E[X\mid\mathcal G] + b\mathbb E[Y\mid\mathcal G]$; (b) positivity: $X\ge0$ ⇒ $\mathbb E[X\mid\mathcal G]\ge0$; (c) $\mathbb E[\mathbb E[X\mid\mathcal G]] = \mathbb E X$; (d) taking out what is known: if $Y$ is $\mathcal G$-measurable and $XY$ integrable, $\mathbb E[XY\mid\mathcal G] = Y\mathbb E[X\mid\mathcal G]$; (e) tower: $\mathbb E[\mathbb E[X\mid\mathcal G]\mid\mathcal H] = \mathbb E[X\mid\mathcal H]$; (f) independence: if $X$ is independent of $\mathcal G$, $\mathbb E[X\mid\mathcal G] = \mathbb E X$.

If $\phi$ is convex and $X, \phi(X)$ are integrable, then $\phi(\mathbb E[X\mid\mathcal G])\le\mathbb E[\phi(X)\mid\mathcal G]$ a.s. In particular $|\mathbb E[X\mid\mathcal G]|\le\mathbb E[|X|\mid\mathcal G]$ and $\mathbb E[X\mid\mathcal G]^2\le\mathbb E[X^2\mid\mathcal G]$.

If $\mathbb E X^2 < \infty$, then $\mathbb E[X\mid\mathcal G]$ is the orthogonal projection of $X$ onto the closed subspace of $\mathcal G$-measurable square-integrable variables (inner product $\mathbb E[UV]$): it minimises $\mathbb E[(X - Y)^2]$ over all such $Y$.

• Symmetry: if $X_1, \dots, X_n$ are i.i.d. and $S = \sum X_i$, then $\mathbb E[X_1\mid S] = \frac Sn$, since all $\mathbb E[X_i\mid S]$ are equal and sum to $S$.
• Random walk: for $S_n = X_1 + \cdots + X_n$ with i.i.d. steps of mean $\mu$ and $m < n$: $\mathbb E[S_n\mid\mathcal F_m] = S_m + (n - m)\mu$, where $\mathcal F_m = \sigma(X_1, \dots, X_m)$.
• Bivariate normal: $\mathbb E[Y\mid X] = \mu_Y + \rho\frac{\sigma_Y}{\sigma_X}(X - \mu_X)$, the regression line.

• Split $X$ into a part known given $\mathcal G$ (take it out) and a part independent of $\mathcal G$ (replace by its mean).
• Use symmetry for exchangeable variables.
• Verify a guess $Z$: is it $\mathcal G$-measurable, and does $\mathbb E[Z\mathbf 1_A] = \mathbb E[X\mathbf 1_A]$ for generating sets $A$?
• Sanity: $\mathbb E[\mathbb E[X\mid\mathcal G]] = \mathbb E X$.