Useful Matrix properties

Theorem

Let $A$ be $m\times n$ matrix, $B$ and $C$ $n\times n$ matrices, with $B$ symmetric, $x$ $n\times 1$ vector

1. $Ax=0 \; \forall x$ if and only if $A=0$,
2. $x^TAx \; \forall x$ if and only if $B=0$,
3. $x^TCx \; \forall x$ if and only if $C^T=-C$ (skew symmetric).

Concentration of measure

To characterize the deviation of a random quantity (typically a general function of weakly correlated variables) from it's mean.  

One can look at

• the tail end probability (moment method and chernoff's bound)
• the stability about it's mean (i.e. )
• the bounded difference ()

Circularly symmetric complex Gaussian

Special case: scalar variable $Z\sim\mathcal{CN}(0,1)$
$$\begin{aligned} f_Z(z) &= \frac{1}{\pi}e^{-|z|^2} \\ f_{|Z|,\Theta}(|z|,\theta) &= \frac{1}{\pi} |z| e^{-|z|^2} \end{aligned}$$
Note that the distribution is a function of the magnitude of $Z$, and therefore constant over all angles $\theta$.  To get the distribution of the magnitude of Z, we integrate over all angles $-\pi \le \theta \le \pi$.
$$\begin{aligned} f_{|Z|}(|z|) &= \int f_{|Z|,\Theta}(|z|,\theta) d\theta \\                   &= \int_{-\pi}^\pi d\theta \; \frac{1}{\pi} |z| e^{-|z|^2} \\                   &= 2 |z| e^{-|z|^2} ,\; |z| > 0 \end{aligned}$$
Define $Y=|Z|$ then,
$$f_Y(y) = 2 y e^{-y^2}, \quad y > 0$$
For magnitude squared distribution, we can perform a change of variable, with $X=Y^2=g(Y)$, since $P( \{-\sqrt{x}, x > 0\} ) = 0$, we only have one region (the set $\{ \sqrt{x}, x>0 \}$ ) to worry about.
$$\begin{aligned} f_X(x) &= f_Y(g^{-1}(x)) \left|\frac{ \partial g^{-1}(x) }{ \partial x} \right| \\             &= f_Y(\sqrt{x})  \left|\frac{ \partial \sqrt{x} }{ \partial x} \right|  \\             &= (2x^{\frac{1}{2}}e^{-x}) \frac{1}{2} x^{-\frac{1}{2}} \\             &= e^{-x}, \; x>0 \end{aligned}$$
Note:
Here we assume the probability spaces $(X,\mathcal{X},P_X)$, $(Y,\mathcal{Y},P_Y)$and that a mapping T exists that maps $A\in\mathcal{X}$ to $B\in\mathcal{Y}$.
Start from the joint cdf in the X-space, define $A\equiv\{\sqrt{X^2+Y^2}\leq |z|, \tan^{-1}(Y/X) \leq \theta_0\}$
$$F_{|Z|,\Theta}(|z|,\theta_0) = P(A)  = \underset{(x,y)\in A}{\iint} f(x,y) dx \; dy$$
With a change of variable, define $B\equiv\{|Z|\leq |z|, \Theta \leq \theta_0\}$
$$\begin{aligned} F_{|Z|,\Theta}(|z|,\theta_0) = P(B) &=  \underset{(r,\theta)\in B}{\iint} f(x(r,\theta), y(r,\theta)) r \; dr \; d\theta \\ &= \int_0^{|z|} \int_0^{\theta_0} f(x(r,\theta), y(r,\theta)) r \; dr \; d\theta \end{aligned}$$
Differentiate wrt $|z|$ and $\theta$ to get the pdf.

Theorem.  Given any function $g:\mathbb{R}^2 \rightarrow \mathbb{R}$ and T that maps the $A\in\mathcal{X}$ to $B\in\mathcal{Y}$ then
$$\iint_A g(x_1,x_2) dx_1 \; dx_2 = \iint_B g(x_1(y_1,y_2), x_2(y_1,y_2))  |J(y_1, y_2)| dy_1\; dy_2$$ where $J(y_1,y_2)$ is the Jacobian.

The inverse power of praise

New York Magazine article

Delay gratification

Newyorker article

The cult of Genius

Discover magazine blog

"... the brain is a muscle. Giving it a harder workout makes you smarter"