Properties of a regular family of parameterized distribution

A family of parameterized distribution defined by

\[ \mathcal{P} = \{ p_\theta(y)  | \theta \in \Theta \subset \mathbb{R}^P \}\]

is regular if it satisfies the following conditions

1. Support of \(p_\theta(y)\) does not depend on \(\theta\) for all \(\theta \in \Theta\)
2. \(\frac{\partial}{\partial \theta} p_\theta(y) \) exists
3. Optional \( \frac{\partial^2}{\partial \theta^2} p_\theta(y) \) exists

Note \[ \frac{\partial }{ \partial \theta } \ln p_\theta(y)  = \frac{1}{p_\theta(y) } \frac{\partial }{ \partial \theta } p_\theta(y) \quad \quad (4) \]

Define the score function (log := natural log)

\[  S_\theta (y) := \nabla_\theta \log p_\theta(y) \]

Note also

\[ E_\theta \{ 1 \} = 1 = \int_\mathcal{Y} p_\theta(y) dy \]

As a result of the above we have (Kay's definition of regular)
\begin{align*}  0 &= \frac{\partial }{ \partial \theta }E_\theta \{1\} \\
&= \frac{\partial }{ \partial \theta }\int p_\theta(y) dy \\
&\overset{1}{=} \int \frac{\partial }{ \partial \theta }  p_\theta(y) dy \\
&\overset{2,4}{=} \int p_\theta(y)  \frac{\partial }{ \partial \theta } \log p_\theta(y) dy \\
&= E_\theta \{ S_\theta(y) \}
\end{align*}