[Work Log] Debugging Index optimization

March 27, 2014

Still getting terrible stretching due to extreme index drift during optimization.

Experiment 1

Theory: camera linearization is messing with our 3D point localization. Propose solving by increasing noise covariance for noisy.

Assuming an effective depth uncertainty of (h), and in-plane uncertainty of (w), the in-plane uncertainty of the adjusted covariance matrix (w') is given by

[ w' = w \mathrm{cos}^2 \theta + h \mathrm{sin}^2 \theta ]

This is illustrated roughly below.

Implemented this using hand-set depth uncertainty (h = 3)mm. Didn't resolve the issue for extremely stretched curves, but the weak stretching seems improved.


Noise variance is CRAZY low. Small deviation from true curve is causing huge contortions to resolve. Why is it trained this low if contradictions result? This smells like overfitting, except we're testing and training on the same date at the moment so that seems to be ruled out.

Loosen and retrain?

We apparently need to multiply standard deviation by 4 to recover; according to yesterday's experiment.

Experiment #2

Plot likelihood (reference implementation) against noise variance.

Apparently it does want a higher noise variance, but only 2.5x more, not 4x. is this enough to improve reconstruction?


Experiment #3

Repeat yesterday's "Overextension" experiment #1. Multiplying variance by 16x seemed to fix issues, but now I suspect that we just got lucky. Try a few values near 16x.

With linearization enabled:

Without linearization:

Yes, tiny tweaks to variance cause us to revert to bad reconstructions. We probably just got lucky when we observed 16x working well yesterday.

Now we're back to square one. noise variance doesn't seem to be connected to this issue of extended


It seems linearization is a bane to index refinement. We introduced it to address reconstruction issues; perhaps we should only apply it in that case.


Re-merging and rechecking new datasets

Only one error, fixed and sent changes to Rebecca.

Regularizing immunity CSV; adding new column type: "numeric_missing:XXX", where missing data is represented by one of a few possible strings. Split the "date(other date)" column into "date" column and "other date" column. Parsed in matlab. comparing immunity dates to self-report dates for each subject; interpolating as needed. Only 9 notable anomalies out of 710.

Next: discretize, measure and visualize discretization errors, export to csv

Posted by Kyle Simek
blog comments powered by Disqus