4 ms·

Almost every explainer I have ever seen on Kalman filters starts with something like this: "Let's begin with an intuitive example: think of a thermostat as it

Almost every explainer I have ever seen on Kalman filters starts with something like this: "Let's begin with an intuitive example: think of a thermostat as it adjusts to the temperature. Got that example? Good! Next here is some advanced linear algebra to help make it more intuitive" Has anyone comes across an explanation of Kalman filters that doesn't immediately dive into the math?

agumonkey 3y ago
I have no answer for your question, but the "draw the rest of the owl" meme is so pervasive in maths it's not even a cliche.
reedf1 3y ago
Unfortunately there is no halfway with maths. Math is the language of logic, the purpose of mathematical formalism is to allow easy communication, not just naval gazing.
- bmitc 3y ago
  That's not really true, in several senses. Mathematics is not the language of logic. I'm not even sure what that was supposed to mean. Logic is the language of logic. Also, engineers and physicists can have rather intuitive understandings of various filters with almost no mathematical calculations being done, on paper or their head.
- justin66 3y ago
  To characterize a mode of communication the learner is struggling to understand as "easy communication" is badly missing the point.
  - btilly 3y ago
    It is only easy compared to the alternative. But I've seen both it and the alternative. It is literally easier to learn the math, and then use that to facilitate communication, than to attempt to directly communicate key concepts while avoiding the math.
    - tomjakubowski 3y ago
      I think it's a bit like trying to convey to someone in English the beauty of a symphony. Better to take them to see the orchestra play it.
  - deleted 3y ago
    [deleted]
- antegamisou 3y ago
  Hoped someone here would come out and say this. It's only supposed to be somewhat intuitive for those who already possess some intermediate level college math background. And no, rejecting the notion that not every advanced topic is supposed to be accurately ELI5-ed with a cringy Redditesque tone, means neither there's any gatekeeping taking place nor there's lack of understanding from the potential explainer's side.
- mayd 3y ago
  I strongly disagree. While rigorous mathematical proof (logic) is a big part of modern mathematical research output, it is an impenetrable barrier to learning for most students, including maths students. I doubt any mathematician ever learned their subject by mainly reading proofs. And I say that as someone who secretly enjoys reading and working through proofs of theorems. Understanding mathematics demands developing strong intuition of ideas before attempting their logical justification. That rigour and intuition in mathematics education are largely contradictory goals is widely acknowledged by maths teachers; 3BlueBrown, for example, has often mentioned this duality in his videos. It was only in latter part of the twentieth century that the idea of abandoning intuition in teaching mathematics was seriously attempted, by the Bourbakists I believe, and was embraced for a while, before fading away.
  - nyssos 3y ago
    > I doubt any mathematician ever learned their subject by mainly reading proofs. And I say that as someone who secretly enjoys reading and working through proofs of theorems. Understanding mathematics demands developing strong intuition of ideas before attempting their logical justification. This is half right: you don't learn math by reading proofs, you learn it by writing proofs. Developing strong intuition without proofs is possible (albeit difficult) for applied topics, but not viable at all for pure math.
extesy 3y ago
Try this one: https://praveshkoirala.com/2023/06/13/a-non-mathematical-introduction-to-kalman-filters-for-programmers/ https://praveshkoirala.com/2023/06/13/a-non-mathematical-int...
tim-fan 3y ago
For me the intuition came from Sebastian Thrun in his Ai for robotics online course. Running through everything in 1D definitely helped get the concepts before extending to multidimensional problems. This looks to be the course playlist: https://youtube.com/playlist?list=PLAwxTw4SYaPkCSYXw6-a_aAoXVKLDwnHK&si=bnG8A7Or8_iGs8gm https://youtube.com/playlist?list=PLAwxTw4SYaPkCSYXw6-a_aAoX... The kalman filter stuff starts at video "Tracking Intro - Artificial Intelligence for Robotics" Also the free course appears to be available here, although a login is required to access: https://www.udacity.com/course/intro-to-artificial-intelligence--cs271 https://www.udacity.com/course/intro-to-artificial-intellige...
- esafak 3y ago
  For my money -- as a machine learning engineer -- the simplest explanation is Christopher Bishop's: https://www.youtube.com/watch?v=QJSEQeH40hM https://www.youtube.com/watch?v=QJSEQeH40hM http://mlss.tuebingen.mpg.de/2013/2013/bishop_slides.pdf#page=86 http://mlss.tuebingen.mpg.de/2013/2013/bishop_slides.pdf#pag...
krackers 3y ago
Depends what level you are comfortable with. Given that the kalman filter makes certain statistical assumptions (normally distributed) you need to be familiar with at least statistics, and know what mean/covariance matrices are. If so you should be able to follow the derivation in https://sites.ualberta.ca/~dwiens/stat679/meinhold&singpurwalla.pdf https://sites.ualberta.ca/~dwiens/stat679/meinhold&singpurwa... up until section 4. (Beyond section 4 it's really just a matter of grinding out the math, and while necessary for implementing it in a practical fashion doesn't really give you any additional understanding).
salty_biscuits 3y ago
It is a weighed average of the propagated uncertainty of a state and the uncertainty of a measurement of that state. If you walk towards a wall with your eyes closed as time on you will be less and less certain where you are (prediction steps) then when your fingers touch the wall you will suddenly be much more certain where you are (measurement update). The linear algebra is just how you calculate the weights for a linear dynamical system with gaussian noise.
fzeindl 3y ago
This is not exactly what you are asking for but a very nice explanation and it tries to work without math as long as possible https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/ https://www.bzarg.com/p/how-a-kalman-filter-works-in-picture...
p5a0u9l 3y ago
Yeah, sounds about right. Real world problems are rarely 1D, constant (or no) velocity, etc. there are good examples out there, the one by rlabbe on GitHub is pretty dang good. It still didn’t quite get me as far as I needed to go, but it got me a lot further than I was when I started.
deleted 3y ago
[deleted]

agumonkey 3y ago

I have no answer for your question, but the "draw the rest of the owl" meme is so pervasive in maths it's not even a cliche.

reedf1 3y ago

Unfortunately there is no halfway with maths. Math is the language of logic, the purpose of mathematical formalism is to allow easy communication, not just naval gazing.

bmitc 3y ago
That's not really true, in several senses. Mathematics is not the language of logic. I'm not even sure what that was supposed to mean. Logic is the language of logic. Also, engineers and physicists can have rather intuitive understandings of various filters with almost no mathematical calculations being done, on paper or their head.
justin66 3y ago
To characterize a mode of communication the learner is struggling to understand as "easy communication" is badly missing the point.
- btilly 3y ago
  It is only easy compared to the alternative. But I've seen both it and the alternative. It is literally easier to learn the math, and then use that to facilitate communication, than to attempt to directly communicate key concepts while avoiding the math.
  - tomjakubowski 3y ago
    I think it's a bit like trying to convey to someone in English the beauty of a symphony. Better to take them to see the orchestra play it.
- deleted 3y ago
  [deleted]
antegamisou 3y ago
Hoped someone here would come out and say this. It's only supposed to be somewhat intuitive for those who already possess some intermediate level college math background. And no, rejecting the notion that not every advanced topic is supposed to be accurately ELI5-ed with a cringy Redditesque tone, means neither there's any gatekeeping taking place nor there's lack of understanding from the potential explainer's side.
mayd 3y ago
I strongly disagree. While rigorous mathematical proof (logic) is a big part of modern mathematical research output, it is an impenetrable barrier to learning for most students, including maths students. I doubt any mathematician ever learned their subject by mainly reading proofs. And I say that as someone who secretly enjoys reading and working through proofs of theorems. Understanding mathematics demands developing strong intuition of ideas before attempting their logical justification. That rigour and intuition in mathematics education are largely contradictory goals is widely acknowledged by maths teachers; 3BlueBrown, for example, has often mentioned this duality in his videos. It was only in latter part of the twentieth century that the idea of abandoning intuition in teaching mathematics was seriously attempted, by the Bourbakists I believe, and was embraced for a while, before fading away.
- nyssos 3y ago
  > I doubt any mathematician ever learned their subject by mainly reading proofs. And I say that as someone who secretly enjoys reading and working through proofs of theorems. Understanding mathematics demands developing strong intuition of ideas before attempting their logical justification. This is half right: you don't learn math by reading proofs, you learn it by writing proofs. Developing strong intuition without proofs is possible (albeit difficult) for applied topics, but not viable at all for pure math.

extesy 3y ago

Try this one: https://praveshkoirala.com/2023/06/13/a-non-mathematical-introduction-to-kalman-filters-for-programmers/ https://praveshkoirala.com/2023/06/13/a-non-mathematical-int...

tim-fan 3y ago

For me the intuition came from Sebastian Thrun in his Ai for robotics online course. Running through everything in 1D definitely helped get the concepts before extending to multidimensional problems. This looks to be the course playlist: https://youtube.com/playlist?list=PLAwxTw4SYaPkCSYXw6-a_aAoXVKLDwnHK&si=bnG8A7Or8_iGs8gm https://youtube.com/playlist?list=PLAwxTw4SYaPkCSYXw6-a_aAoX... The kalman filter stuff starts at video "Tracking Intro - Artificial Intelligence for Robotics" Also the free course appears to be available here, although a login is required to access: https://www.udacity.com/course/intro-to-artificial-intelligence--cs271 https://www.udacity.com/course/intro-to-artificial-intellige...

esafak 3y ago
For my money -- as a machine learning engineer -- the simplest explanation is Christopher Bishop's: https://www.youtube.com/watch?v=QJSEQeH40hM https://www.youtube.com/watch?v=QJSEQeH40hM http://mlss.tuebingen.mpg.de/2013/2013/bishop_slides.pdf#page=86 http://mlss.tuebingen.mpg.de/2013/2013/bishop_slides.pdf#pag...

krackers 3y ago

Depends what level you are comfortable with. Given that the kalman filter makes certain statistical assumptions (normally distributed) you need to be familiar with at least statistics, and know what mean/covariance matrices are. If so you should be able to follow the derivation in https://sites.ualberta.ca/~dwiens/stat679/meinhold&singpurwalla.pdf https://sites.ualberta.ca/~dwiens/stat679/meinhold&singpurwa... up until section 4. (Beyond section 4 it's really just a matter of grinding out the math, and while necessary for implementing it in a practical fashion doesn't really give you any additional understanding).

salty_biscuits 3y ago

It is a weighed average of the propagated uncertainty of a state and the uncertainty of a measurement of that state. If you walk towards a wall with your eyes closed as time on you will be less and less certain where you are (prediction steps) then when your fingers touch the wall you will suddenly be much more certain where you are (measurement update). The linear algebra is just how you calculate the weights for a linear dynamical system with gaussian noise.

fzeindl 3y ago

This is not exactly what you are asking for but a very nice explanation and it tries to work without math as long as possible https://www.bzarg.com/p/how-a-kalman-filter-works-in-pictures/ https://www.bzarg.com/p/how-a-kalman-filter-works-in-picture...

p5a0u9l 3y ago

Yeah, sounds about right. Real world problems are rarely 1D, constant (or no) velocity, etc. there are good examples out there, the one by rlabbe on GitHub is pretty dang good. It still didn’t quite get me as far as I needed to go, but it got me a lot further than I was when I started.

deleted 3y ago

[deleted]