Cramér-Rao inequality
Cramér-Rao 不等式
\( X=(X_1, \ldots, X_n)\)の分布が母数\( \theta\)を用いて\( f(X_1, \ldots, X_n|\theta)\)と表せるとする.
\( \varphi(X)\)が\( \psi(\theta)\)の不偏推定量, つまり\( \left\langle\varphi(X)\right\rangle=\psi(\theta)\)のとき,
$$
\begin{align}
\mathrm{Var}[\psi(X)] \geq \frac{(\partial_\theta \psi(\theta))^2}{\mathcal{I}(\theta)}.
\end{align}
$$
特に, \( X_1, \ldots, X_n\)がi.i.d. のとき,
$$
\begin{align}
\mathrm{Var}[\psi(X)] \geq \frac{(\partial_\theta \psi(\theta))^2}{n\mathcal{I}(\theta)}.
\end{align}
$$
ただし\( \mathcal{I}(\theta)\)はFisher情報量で,
$$
\begin{align}
\mathcal{I}(\theta) = \left\langle(\partial_\theta \ln f(X|\theta))^2\right\rangle
\end{align}
$$
証明
微分\( \partial_\theta\) と積分\( \int dx\)は交換可能とすると,
$$
\begin{align}
\partial_\theta \psi(\theta) &= \partial_\theta \left\langle\phi(X)\right\rangle\\
&= \int dx \partial_\theta f(x|\theta)\varphi(x)\\
&= \int dx f(x|\theta) \frac{\partial_\theta f(x|\theta)}{f(x|\theta)} \varphi(x)\\
&= \int dx f(x|\theta) \partial_\theta \ln f(x|\theta) \varphi(x)\\
&= \left\langle\varphi(X)\partial_\theta\ln f(X|\theta)\right\rangle. \tag{1}
\end{align}
$$
$$
\begin{align}
\left\langle\partial_\theta\ln f(x|\theta)\right\rangle &= \int dx f(x|\theta)\frac{\partial_\theta f(x|\theta)}{f(x|\theta)}\\
&= \int dx \partial_\theta f(x|\theta)\\
&= \partial_\theta \int dx f(x|\theta)\\
&= \partial_\theta 1\\
&= 0. \tag{2}
\end{align}
$$
(1)\(-\)(2)\( \times\left\langle\varphi(X)\right\rangle\)より,
$$
\begin{align}
\partial_\theta \psi(\theta) = \left\langle(\varphi(X)-\left\langle\varphi(X)\right\rangle)\partial_\theta\ln f(X|\theta)\right\rangle.
\end{align}
$$
両辺を2乗して,
$$
\begin{align}
(\partial_\theta \psi(X))^2 &= \left\langle(\varphi(X)-\left\langle\varphi(X)\right\rangle)\partial_\theta \ln f(X|\theta)\right\rangle^2\\
&\leq \left\langle(\varphi(X)-\left\langle\varphi(X)\right\rangle)^2\right\rangle\left\langle(\partial_\theta \ln f(X|\theta))^2\right\rangle\;(\because \mathrm{Cauchy-Schwartz\;不等式}).
\end{align}
$$
\( X_1, \ldots, X_n\)がi.i.d.のとき,
$$
\begin{align}
\partial_\theta \ln f(X|\theta) = \sum_{i=1}^n \partial_\theta \ln f(X_i|\theta).
\end{align}
$$
\( i\neq j\)とすると,
$$
\begin{align}
\left\langle\partial_\theta \ln f(X_i|\theta)\partial_\theta\ln f(X_j|\theta)\right\rangle &=
\left\langle\partial_\theta \ln f(X_i|\theta)\right\rangle\left\langle\partial_\theta\ln f(X_j|\theta)\right\rangle \\
&= 0
\end{align}
$$
であることに注意すれば,
$$
\begin{align}
\left\langle(\partial_\theta\ln f(X|\theta))^2\right\rangle = n\left\langle(\partial_\theta\ln f(X_1|\theta))^2\right\rangle .
\end{align}
$$