November 04, 2013

First day after CVPR deadline.

Comp exam - committee, planning.

Computer issues - upgrade, crashing, keyboard?

Evaluating the second likelihood term is still very slow, even after a 100x speedup.

First do MH using single-term likelihood. Then treat that as the proposal for the two-term likelihood. Since the first step satisfies detailed balance, we have:

\[
\begin{align}
\hat p(\theta) q(\theta' | \theta) &=
\hat p(\theta') q(\theta | \theta') \\
\frac{\hat p(\theta)}{\hat p(\theta')} &=
\frac{q(\theta | \theta')}{q(\theta' | \theta)}
\end{align}
\]

Where \(\hat p(\theta)\) is the surrogate posterior, using only the single-term likelihood. Substituting this identity into the full MCMC acceptance term, we get:

\[
\begin{align}
\alpha &= \left \{ \frac{p(\theta') q(\theta | \theta')}{p(\theta)q(\theta' | \theta)} \right \} \\
&= \left \{ \frac{p(\theta') \hat p(\theta)}{p(\theta)\hat p(\theta')} \right \} \\
&= \frac{L_2(\theta')}{L_2(\theta)}
\end{align}
\]

This obviously isn't applicable for traditional gibbs sampling, but gibbs could be used for proposal, and MH used to accept/reject

Posted by Kyle Simek