Innovations sequence

Definition 
Suppose \( \{z_k\} \) is a sequence of jointly Gaussian random elements.   The innovations process \(\{\tilde{z}_k\} \) is such that \(\tilde{z}_k\) consists of that part of \(z_k\) containing new information not carried in \(z_{k-1}, z_{k-2}, \dotsc\).
\[ \tilde{z}_k = z_k - E[z_k | z_0, \dotsc, z_{k-1} ]  = z_k - E[z_k | Z^{k-1}] \] with \( \tilde{z}_0 = z_0 - E[z_0] \).

Properties

1. \(\tilde{z}_k\) independent of \( z_0, \dotsc, z_{k-1}\) by definition
2. (1) implies \(E[ \tilde{z}_k' \tilde{z}_l] = 0, l \neq k \)
3. \(E[z_k | Z^{k-1}]\) is a linear combination of \(z_0, \dotsc, z_{k-1}\)
4. The sequence \(\{\tilde{z}_k\} \) can be obtained from \(\{z_k\} \) by a causal linear operation.
5. The sequence \(\{z_k\} \) can be reconstructed from \(\{\tilde{z}_k\} \) by a causal linear operation. 
6. (4) and (5) implies \( E[z_k | Z^{k-1}] = E[z_k | \tilde{Z}^{k-1}] \) or more generally  \( E[w | Z^{k-1}] = E[w | \tilde{Z}^{k-1}] \) for jointly Gaussian \(w, \{z_k\} \)
7. For zero mean Gaussian \(\tilde{x}_k\), \(\tilde{z}_k\), we have \[ E[x_k|Z^{k-1}] = E[x_k|\tilde{Z}^{k-1}] = E[x_k| \tilde{z}_0] + \dotsb + E[x_k| \tilde{z}_{k-1}]    \]