4 ms·

PSA: If/when using these tools, know the difference between Kalman filter and Kalman smoother. And be especially careful when using to make forecasts, forward e

PSA: If/when using these tools, know the difference between Kalman filter and Kalman smoother. And be especially careful when using to make forecasts, forward estimates, predictions, etc. Beyond the scope of this comment and post, but mind two-sided filters (as the Kalman smoother/HP filter uses) as you could be incorporating future/unknown data into your model.

sorenjan 3y ago
I've actually been looking for a good description of the Kalman smoother, to process recorded data without phase shift, but I haven't found any. I have text books on the subject from school, but I can't really translate it to real code in this case. I understand using and updating the filter sequentially, but how do I use information from both previous and future data for each point?
- bp0017 3y ago
  This is my question as well, I've read multiple sources that imply the Kalman smoother would work for my problem, but not exactly _how_ in a way that I could make sense of programmatically.
- NwtnsMthd 3y ago
  A Kalman filter will give you the "best guess" for some state (x) at the current timestep (k). This estimate often has some lag in it, likely because you have some incomplete information that you couldn't model. Sometimes we care about the previous states (e.g., x_k-1). But if we just save these states and refer to them, we're not getting the most out of our data. The Kalman Smoother can be used to go back and update these past values with all the samples up to the current time. To update your previous measurements, you need to save the state of your filter at every timestep (x_k) and its associated covariance matrix (P_k). You can then apply Kalman Smoothing to reprocess previous data and update it with all current information. This will often remove the phase delay that you would otherwise observe in your estimate.
- joeyo 3y ago
  Max Welling's Kalman Filter tutorial [1] derives the smoother equations using pretty clear and easy to follow notation (and is a great resource generally). Briefly: you first run the filter equations "forwards", processing each datapoint sequentially from start to end. Then you run the smoother "backwards" in time on the same data going from end to start. 1. http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/papers/hmm/KF-welling-notes.pdf http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/...
  - sorenjan 3y ago
    Thank you, I'll have a look at it when I can find the time and frame of mind.
deleted 3y ago
[deleted]

sorenjan 3y ago

I've actually been looking for a good description of the Kalman smoother, to process recorded data without phase shift, but I haven't found any. I have text books on the subject from school, but I can't really translate it to real code in this case. I understand using and updating the filter sequentially, but how do I use information from both previous and future data for each point?

bp0017 3y ago
This is my question as well, I've read multiple sources that imply the Kalman smoother would work for my problem, but not exactly _how_ in a way that I could make sense of programmatically.
NwtnsMthd 3y ago
A Kalman filter will give you the "best guess" for some state (x) at the current timestep (k). This estimate often has some lag in it, likely because you have some incomplete information that you couldn't model. Sometimes we care about the previous states (e.g., x_k-1). But if we just save these states and refer to them, we're not getting the most out of our data. The Kalman Smoother can be used to go back and update these past values with all the samples up to the current time. To update your previous measurements, you need to save the state of your filter at every timestep (x_k) and its associated covariance matrix (P_k). You can then apply Kalman Smoothing to reprocess previous data and update it with all current information. This will often remove the phase delay that you would otherwise observe in your estimate.
joeyo 3y ago
Max Welling's Kalman Filter tutorial [1] derives the smoother equations using pretty clear and easy to follow notation (and is a great resource generally). Briefly: you first run the filter equations "forwards", processing each datapoint sequentially from start to end. Then you run the smoother "backwards" in time on the same data going from end to start. 1. http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/papers/hmm/KF-welling-notes.pdf http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/...
- sorenjan 3y ago
  Thank you, I'll have a look at it when I can find the time and frame of mind.

deleted 3y ago

[deleted]