Properties of a regular family of parameterized distribution

A family of parameterized distribution defined by

$$\mathcal{P} = \{ p_\theta(y)  | \theta \in \Theta \subset \mathbb{R}^P \}$$

is regular if it satisfies the following conditions

1. Support of $p_\theta(y)$ does not depend on $\theta$ for all $\theta \in \Theta$
2. $\frac{\partial}{\partial \theta} p_\theta(y)$ exists
3. Optional $\frac{\partial^2}{\partial \theta^2} p_\theta(y)$ exists

Note $$\frac{\partial }{ \partial \theta } \ln p_\theta(y)  = \frac{1}{p_\theta(y) } \frac{\partial }{ \partial \theta } p_\theta(y) \quad \quad (4)$$

Define the score function (log := natural log)

$$S_\theta (y) := \nabla_\theta \log p_\theta(y)$$

Note also

$$E_\theta \{ 1 \} = 1 = \int_\mathcal{Y} p_\theta(y) dy$$

As a result of the above we have (Kay's definition of regular)
$$\begin{align*}  0 &= \frac{\partial }{ \partial \theta }E_\theta \{1\} \\ &= \frac{\partial }{ \partial \theta }\int p_\theta(y) dy \\ &\overset{1}{=} \int \frac{\partial }{ \partial \theta }  p_\theta(y) dy \\ &\overset{2,4}{=} \int p_\theta(y)  \frac{\partial }{ \partial \theta } \log p_\theta(y) dy \\ &= E_\theta \{ S_\theta(y) \} \end{align*}$$