[Work Log] Post-CVPR-deadline; 2-part likelihood efficiency, 2-pass sampling
November 04, 2013
First day after CVPR deadline.
Comp exam - committee, planning.
Computer issues - upgrade, crashing, keyboard?
2-Pass sampling.
Evaluating the second likelihood term is still very slow, even after a 100x speedup.
First do MH using single-term likelihood. Then treat that as the proposal for the two-term likelihood. Since the first step satisfies detailed balance, we have:
\[
\begin{align}
\hat p(\theta) q(\theta' | \theta) &=
\hat p(\theta') q(\theta | \theta') \\
\frac{\hat p(\theta)}{\hat p(\theta')} &=
\frac{q(\theta | \theta')}{q(\theta' | \theta)}
\end{align}
\]
Where \(\hat p(\theta)\) is the surrogate posterior, using only the single-term likelihood. Substituting this identity into the full MCMC acceptance term, we get:
\[
\begin{align}
\alpha &= \left \{ \frac{p(\theta') q(\theta | \theta')}{p(\theta)q(\theta' | \theta)} \right \} \\
&= \left \{ \frac{p(\theta') \hat p(\theta)}{p(\theta)\hat p(\theta')} \right \} \\
&= \frac{L_2(\theta')}{L_2(\theta)}
\end{align}
\]
This obviously isn't applicable for traditional gibbs sampling, but gibbs could be used for proposal, and MH used to accept/reject
Posted by Kyle Simek