August 08, 2013

I reconstructed the curves using the models I trained yesterday. I was able to recover both the overall structure and track it's motion over 9 views. For each of these results, use the slider below to change between each of the 9 views (the changes are subtle).

I'm still struggling with smoothing variance being too low, which causes curves to be too straight. For all of these results, I manually changed `smoothing_variance`

to be 0.1.

Below is the reconstruction for the independent-perturbation model. The curves in each view are independently perturbed versions of central mean curves (not shown).

It is unclear how much of this motion is due to camera miscalibration, and how much is actual plant motion. Nevertheless, this shows that we can use the perturbation models to simultaneously triangulate and track over time.

First, although each *view's* perturbation is independent of the others, the perturbation is correlated between nearby points within the same view. In other words, perturbations don't violate the smoothness constraint.

Next is the Ornstein-Ulenbeck perturbation model. As opposed to the previous model, which modeled each view's perturbations as white noise, this model assumes Brownian motion over time. Thus, we see a more naturally evolving time-series.

The OU process models brownian motion

Finally we have the squared-exponential perturbation model. Now Brownian motion has been replaced by smooth motion. I'm doubtful that this is a natural motion model for these plants. The scale parameter is so low, I question whether it has any significant effect.

Need to determine why learned smoothing variance is so low.

Is it even valid to do max-likelihood to train the parameters of the covariance function?

Should we be training the smoothness parameter using the noiseless ground-truth data?

- First learn using noiseless data:
- Set rate-variance to fixed ~0.23 (see yesterday's results)
- Estimate position_variance by emperical distribution over all point positions in ground truth.
- Estimate smoothing_variance by maximum likelihood over ground truth.

- Learn the noise variance for the no-perturb model.
- Find point correspondence between detected curves and corresponding ground truth curve.
- Compute variance between projected points and observed points. This should maximize \(p(D \mid \Theta_0) \), where \(\Theta_0\) are the ground truth curves.

- Learn perturb model parameters: noise_variance, perturb_{smoothing_variance, rate_varaince, position_variance}
- Use nonlinear optimization to maximize \(p(D \mid \Theta_0) \), as defined below. Start with no-perturb parameters.

Let \(\theta_0\) be the ground-truth curve, and \(\{\theta_i\}\) be the set of all (unobserved) per-view curves \(\theta_i\). Let \(S_i\) be the virtual point precision matrices, assuming noise variance \(\sigma_n^2 = 1\). The likelihood conditioned on the ground truth data \(p(D \mid \theta_0)\) is given by:

\begin{align}
p(D \mid \theta_0) &= \int_{\{\theta_i\}} p(D_i, \{\theta_i\} \mid \theta_0) d\{\theta_i\} \\
&= \int_{\{\theta_i\}} \prod_{i=1}^N p(\theta_i \mid \theta_0) p(D_i | \theta_i) d\{\theta_i\} \\
&= \prod_{i=1}^N \int_{\theta_i} p(\theta_i \mid \theta_0) p(D_i | \theta_i) d\theta_i \\
&= \prod_{i=1}^N \int_{\theta_i} \mathcal{N}(\theta_i; \theta_0, \Sigma_p) \mathcal{N}(D_i; \theta_i, \sigma_n^2 S_i^{-1}) d\theta_i \\
&= \prod_{i=1}^N \int_{\theta_i} \mathcal{N}(D_i ; \theta_0, \Sigma_p + \sigma_n^2 S_i^{-1})
\end{align}

where \(\Sigma_p\) is the perturbation variance.

This should be an improvement over the current method, where we use only ground-truth labellings (not positions) and fit all parameters simultaneously. This method assumed too much noise variance and not enough smoothness variance.

Since we actually know the noiseless curves, we should train to that, to avoid the confounding of smoothness variance and noise variance. The ground truth curves are aso stronger sources of evidence, compared to the curves reconstructed from data.

- reproject per-view reconstructions and overlay with original image.
- obtain & visualize unobserved "central curve"
- investigate the low smoothness variance.

Later TODO

- automatic curve reversing?
- add branching