2 ms·
I'd been wondering about the accuracy of the probabilities too. It's one thing if your dynamical model produces two state estimates and covariances, and anothe
by mturmon 6y ago
I'd been wondering about the accuracy of the probabilities too. It's one thing if your dynamical model produces two state estimates and covariances, and another entirely when they actually match reality quantitatively!
LeoLabs has a recent paper [1] that evaluates the state prediction accuracy. They're finding the right thing (Mahalanobis distance at various temporal separations between their state estimate and an object position). See secs. 4 and 5.
But: they are not getting the chi-squared distribution that you'd want. They are getting log-normals, with significant probability mass out to a Mahalanobis distance of 10 and higher (!) -- see the figure in sec. 5.
That's a very high deviation. It's not unexpected, because this is a hard problem, but the deviations do not appear to follow the expected law.
If their collision-probability takes this into account, that's good. But if not -- if they're assuming a Gaussian prediction cloud -- then their collision-probability will not be well-calibrated. If my cursory examination of their (well-done!) paper is correct, a poorly-calibrated probability would be very wrong indeed.
[1] https://amostech.com/TechnicalPapers/2019/Astrodynamics/Park.pdf https://amostech.com/TechnicalPapers/2019/Astrodynamics/Park...