16 ms·
Kalman filter from the ground up
- tonyarkles 3y agoI have a copy of this book and have used it in anger quite successfully. There’s a couple of spots in it where it’s a little bit awkward to follow even after re-reading multiple times, but it’s quite good overall!
- okamiueru 3y ago> used it in anger quite successfully Care to elaborate?
- riazrizvi 3y agoPossibly used to dash bugs into oblivion, or held up at arms length at a protest rally to condemn book banning? My guesses in case OP doesn't come back. I'm sure if we all make enough guesses, then take the average, we'll come up with the correct answer.
- aktenlage 3y agoDo not forget to weight the guesses by their uncertainty, considering the correlation between the guesses.
- tonyarkles 3y agoThe sibling posts figured it out. I have gone from picking up this book to using the material in it in production, using the contents of the book as my primary reference to get from a vague understanding to an implementation I don't think much about because it Just Works.
- mlsu 3y agoIt's a British-ism for "used in actual practice, rather than academically/theoretically" https://dictionary.cambridge.org/dictionary/english/in-anger https://dictionary.cambridge.org/dictionary/english/in-anger
- crazygringo 3y agoAh thank you -- as an American I've never come across that expression my whole life!
- btilly 3y agoI wasn't aware of this as a British/American thing. As a programmer, it is just a phrase that I picked up because other programmers used it and I liked it. And yes. I'm an American.
- tonyarkles 3y agoYeah I'm pretty sure I picked it up on HN :) Am Canadian.
- deleted 3y ago[deleted]
- variaga 3y agoThec phrase "used Y in anger" means "used seriously" or "used for its intended purpose". The origin phrase/trope-namer is "never fired its guns in anger" applied to a naval warship in peacetime to describe the case where it might have fired its guns in practice, but not in combat. Entering "never fired its guns in anger" in your favorite search engine should give several examples. Here's one in an article from last week. https://owlcation.com/humanities/What-If-The-USS-Arizona-Never-Sank https://owlcation.com/humanities/What-If-The-USS-Arizona-Nev... - "The USS Arizona never fired her guns in anger."
- tonyarkles 3y agoHah, yes, I see everyone else has filled you in already. I have several Kalman Filters modelling both linear (conventional KF) and non-linear (Extended) systems in production that were derived primarily from the material in this book. I did have to do some application-specific research for a couple of things, for example 3D attitude estimation is a deep dark hole with many different formulations with different pros and cons, but the book gave me a very solid foundation for being able to take attitude estimation papers from NASA and others and work through the approach I wanted to use.
- javitury 3y agoThere is no explanation about how Kalman filters work in the article, it is just an advertisement for a book. And the book preview doen't even cover how to apply Kalman filters to conventional state space models.
- okamiueru 3y agoYou can navigate on the left side of the page. Seems fairly well put together from a quick glance. https://www.kalmanfilter.net/kalman1d.html https://www.kalmanfilter.net/kalman1d.html
- javitury 3y agoThanks for pointing it out, it was not obvious on my phone. I think that this website needs to improve the experience of mobile users. First, the book index on the landing page didn't contain hyperlinks to the free book chapters and these chapters don't appear on the preview, so users may assume it is a shady marketing tactic (trial, requires retweet, subscription to book platform, ...). Second, towards the end of the page, there could be a one-sentence-description of the next section with an hyperlink. Right now there is only a "next" button that is easy to miss, and it can be confused with generic "next post" buttons used by many blog platforms that users learn to ignore because they tend to be useless.
- dpflan 3y agoHere is another tutorial on Kalman Filters, step-by-step video playlist -- https://www.youtube.com/watch?v=CaCcOwJPytQ&list=PLX2gX-ftPVXU3oUFNATxGXY90AULiqnWT&index=1 https://www.youtube.com/watch?v=CaCcOwJPytQ&list=PLX2gX-ftPV... Once you get the intuition, Kalman filters are really interesting. As are particle filters -- those are fun to work with and visualize.
- plusplusungood 3y agoI'm looking for something similar for PID controllers, or control systems in general. Specifically for a teenager. Most sources I can find are either college textbooks or overly simple summaries for FIRST Lego League.
- amelius 3y agohttps://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pdf https://www.wescottdesign.com/articles/pid/pidWithoutAPhd.pd...
- vero24 3y agohttps://file.tavsys.net/control/controls-engineering-in-frc.pdf https://file.tavsys.net/control/controls-engineering-in-frc.... This is pretty complex but its been super helpful for me in terms of understanding PID controllers.
- plusplusungood 3y agoThat is pretty much exactly what I was looking for. Maybe a touch more complicated than what my 11 year old is ready for; but something he can stretch a bit with. Thank you!
- deleted 3y ago[deleted]
- oger 3y agoElliott Williams (editor of Hackaday) held a very interesting talk this year at CCCamp that in very simple terms explained the components of a PID loop. Highly recommended! https://media.ccc.de/v/camp2023-57111-pid_loops_control_all_the_things https://media.ccc.de/v/camp2023-57111-pid_loops_control_all_...
- mhh__ 3y agoTerry Davis has a nice video of him demoing a physics program he wrote, where he is coding a PID for a rocket.
- 3y ago
- lucan 3y agoSee also: https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Python/ https://github.com/rlabbe/Kalman-and-Bayesian-Filters-in-Pyt...
- mtillman 3y agoI was at an adtech company in 2007 where the CEO and research teams became obsessed with Kalman filters for the purposes of optimizing ad campaigns on google and msn networks. It kind of worked from what I remember but I can’t find the patent filings any longer and Zeta & Walmart bought the tech.
- scottydog51834 3y agoStudent Dave's "Quailman" Kalman youtube tutorials are incredible. https://www.youtube.com/watch?v=FkCT_LV9Syk&list=PLzCl0zqNaIBuo957MnuuDyNF2YqOYfaDP https://www.youtube.com/watch?v=FkCT_LV9Syk&list=PLzCl0zqNaI...
- victor106 3y ago> The Kalman Filter algorithm is a powerful tool for estimating and predicting system states in the presence of uncertainty... How is this different than any of the ML models?
- dmillar 3y agotomato tomato
- radarsat1 3y agoYou could probably call a Kalman filter an ML model if you wish. One major difference is that you usually have to prescribe the prediction model. It doesn't learn to predict the next value like an LLM, instead it learns to find an optimal weighted sum of its prediction and the measured values. So it's a prediction-correction loop that requires an interpretable model, that has the side effect of allowing you to estimate system states. This is quite different than having an arbitrary hidden state with learned structure. In other words it performs well for certain applications specifically because it allows you to bring in domain knowledge in the form of the process model and known uncertainties. Whereas deep learning models try to generalize the model and learn implicit structure from data.
- ubitaco 3y agoIf you can make certain assumptions about the system (mainly that sources of noise follow gaussian distributions and are independent), then the Kalman filter gives the best possible estimate of the system state. And it can be computed cheaply, like on the Apollo guidance computer. You basically need to know some kind of a model for the system to run KF. Whereas ML is all about working out the model automatically. As for similarities, KF is a really efficient implementation of Bayesian inference. I think that any ML model that isn't fundamentally using Bayesian inference, is fundamentally flawed.
- vonwoodson 3y agoKalman Filters were developed outside the context of “Machine Learning” and found practical applications much earlier. If you define ML as an algorithm that is “trained by input data”, than any statistical model would be ML. So is the ambiguity around the terms ML and AI in general. Where a KF is really going to kick the pants of a multi-layer perception/neural network is how computationally efficient it is. A KF only takes a couple of matrices of size N^2, where N is the number of variables you’re trying to predict. Compare this to a NN with hundreds/thousands of nodes. Also, the KF is “online learning” in that it “is trained as you go” rather than some other ML models that require upfront training, a KF is very useful for live update-and-predict use cases. And, again, it’s extremely computationally efficient, and can run easily on embedded systems. The book ad, here, suggests tracking: so tracking an airplane with an air traffic control radar would be an effective use for a KF. (Where NNets have found any other uses I’m sure you’re aware of.) Another huge benefit of a KF is that, unlike NNets, KFs are “explainable”, and in fact extremely well understood by many professionals. This means that a KF can be better tuned to suit a purpose with less fear of unexpected results that may be more common in other ML models. Like, KF(S, x) will always return an explainable new state, where NN(x) may result in a surprise state and no amount of analysis can reveal why (and require training a new model, the “retrain and pray” solution). There’s a couple of differences for you.
- dmillar 3y agoPSA: If/when using these tools, know the difference between Kalman filter and Kalman smoother. And be especially careful when using to make forecasts, forward estimates, predictions, etc. Beyond the scope of this comment and post, but mind two-sided filters (as the Kalman smoother/HP filter uses) as you could be incorporating future/unknown data into your model.
- sorenjan 3y agoI've actually been looking for a good description of the Kalman smoother, to process recorded data without phase shift, but I haven't found any. I have text books on the subject from school, but I can't really translate it to real code in this case. I understand using and updating the filter sequentially, but how do I use information from both previous and future data for each point?
- bp0017 3y agoThis is my question as well, I've read multiple sources that imply the Kalman smoother would work for my problem, but not exactly _how_ in a way that I could make sense of programmatically.
- NwtnsMthd 3y agoA Kalman filter will give you the "best guess" for some state (x) at the current timestep (k). This estimate often has some lag in it, likely because you have some incomplete information that you couldn't model. Sometimes we care about the previous states (e.g., x_k-1). But if we just save these states and refer to them, we're not getting the most out of our data. The Kalman Smoother can be used to go back and update these past values with all the samples up to the current time. To update your previous measurements, you need to save the state of your filter at every timestep (x_k) and its associated covariance matrix (P_k). You can then apply Kalman Smoothing to reprocess previous data and update it with all current information. This will often remove the phase delay that you would otherwise observe in your estimate.
- joeyo 3y agoMax Welling's Kalman Filter tutorial [1] derives the smoother equations using pretty clear and easy to follow notation (and is a great resource generally). Briefly: you first run the filter equations "forwards", processing each datapoint sequentially from start to end. Then you run the smoother "backwards" in time on the same data going from end to start. 1. http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/papers/hmm/KF-welling-notes.pdf http://www.stat.columbia.edu/~liam/teaching/neurostat-spr12/...
- deleted 3y ago[deleted]
- kwant_kiddo 3y agoOften in textbooks and on here on HN Kalman Filters are praised, but folks from the industry often ridicule Kalman Filters. All People I have interacted with have never used/seen Kalman Filters outside Academica. Do anyone here have industry experience where Kalman Filters is actually used in prod?
- aktenlage 3y agoYes. Worked in a robotics startup and a tier 1 automotive supplier. In both, variants of the Kalman filter were used for some tasks. They are a simple tool, and simple often works. You just need to know the limitions and know which variant (if any) is best applicable to the problem at hand.
- UncleOxidant 3y agoI recently saw "experience with Kalman filters" listed in a job ad from a Robotics company.
- kwant_kiddo 3y agoedit: Extensively!! In control theory it seems :) It is just that I keep seeing KF be mentioned on Twitter and blogs etc. for using it in finance.
- AlotOfReading 3y agoDone plenty of automotive/robotics by this point and yep. A KF is usually my first (and often only) step between straight up not filtering and finding a controls expert to analyze the system properly. It's cheap, easy, and works well enough that I don't have to think too hard about it.
- ms512 3y agoI used them extensively when I worked on Radar contact tracking. They are an effective tool when used with a good understanding of the limitations.
- Randor 3y ago>> Do anyone here have industry experience where Kalman Filters is actually used in prod? Yes, ship/vessel navigation software heavily use Kalman Filters. Especially on the inputs received from the various position reference sensors.
- aktenlage 3y agoNone of the Kalman filter tutorials I have seen (including this one) gives a good explanation for the equations of the measurement update - including the covariance update. If anyone is curious, here is an explanation: https://postbits.de/kalman-measurement-update.html https://postbits.de/kalman-measurement-update.html Tl;dr: It's mathmatically equivalent to the Bayes filter equations (no big surprise, KF is a BF), and those equations are easier to understand intuitively, but less efficient to compute.
- tnecniv 3y agoYour TLDR is slightly wrong. The Kalman filter is exactly Bayes theorem for linear dynamics and additive Gaussian noise. Bayes theorem is more general, but these assumptions yield a simple set of linear equations that can be solved very quickly. In general, using Bayes theorem for filtering is often called the Bayes filter. Outside of a limited set of circumstances, it is not possible to implement. The Kalman filter is one set of circumstances because linear transforms of Gaussian distributions remain Gaussian, and we can specify a Gaussian process just by its mean and covariance. Another set of circumstances is when you have a discrete system with a small enough number of states. In general however, the Bayes filter results in an infinite dimensional dynamical system since it yields a typically continuous function representing the distribution of your state given your measurements. The Kalman filter, particle filter, and other methods approximate this problem via a more tractable finite dimensional representation.
- aktenlage 3y agoYou are correct, but I don't get where I am wrong. Is it just that I didn't write "equivalent to the Bayes filter equations for linear dynamics and additive Gaussian noise", or do I miss something else?
- alexpotato 3y agoAlmost every explainer I have ever seen on Kalman filters starts with something like this: "Let's begin with an intuitive example: think of a thermostat as it adjusts to the temperature. Got that example? Good! Next here is some advanced linear algebra to help make it more intuitive" Has anyone comes across an explanation of Kalman filters that doesn't immediately dive into the math?
- agumonkey 3y agoI have no answer for your question, but the "draw the rest of the owl" meme is so pervasive in maths it's not even a cliche.
- reedf1 3y agoUnfortunately there is no halfway with maths. Math is the language of logic, the purpose of mathematical formalism is to allow easy communication, not just naval gazing.
- bmitc 3y agoThat's not really true, in several senses. Mathematics is not the language of logic. I'm not even sure what that was supposed to mean. Logic is the language of logic. Also, engineers and physicists can have rather intuitive understandings of various filters with almost no mathematical calculations being done, on paper or their head.
- justin66 3y agoTo characterize a mode of communication the learner is struggling to understand as "easy communication" is badly missing the point.
- btilly 3y agoIt is only easy compared to the alternative. But I've seen both it and the alternative. It is literally easier to learn the math, and then use that to facilitate communication, than to attempt to directly communicate key concepts while avoiding the math.
- bmitc 3y agoDoes anyone remember a tutorial website posted here that taught PID control loops with interactive JavaScript animations? It stepped through the reasoning for and effect of each element of the PID. I have been trying to dig it up for a while but can't find it.
- hanche 3y agoI attended a short series of lectures by Kalman years ago. As I recall it, he very strongly emphasised the virtue of working directly with observed data, avoiding the biases resulting from positing a model and trying to adapt it to the data. He was quite insistent on this point, citing Newton’s _Principia_ as a good example of this principle at work: Newton, he said, did not cast around for models that might explain Kepler’s laws. Instead, he _derived_ the inverse square law of gravitation from Kepler’s laws, largely using geometric arguments. He was an excellent speaker, and highly opinionated. And of course he did explain the idea behind the Kalman filter. However, since I never needed them myself in my work, I have long since forgotten the details.
- p5a0u9l 3y agoThat’s a cool story. I feel like this is so overlooked. I’ve worked with scientists and engineers who’ve actually deployed systems that are incredibly complicated and pretty well known. Those folks are usually skeptical of trendy algorithms and usually start with first principles. 90% of the time, a boring old linear Kalman with simple (or no) motion model will do the trick.
- tnecniv 3y agoNo motion model? If your model doesn’t have dynamics, it’s not really doing Kalman filtering
- omneity 3y agoAs someone who has never heard of Kalman filters before, I found this page[0] useful to understand and build a mental model for it. [0]: https://www.mathworks.com/help/vision/ug/using-kalman-filter-for-object-tracking.html https://www.mathworks.com/help/vision/ug/using-kalman-filter...
- chubs 3y agoI've been asked by a friend recently to implement a Kalman filter for a side project. Among many others, I've read this website and still have no idea how to implement it. They all seem to be in the style of 'draw the rest of the owl'. Can anyone point to anything that explains it in a manner a programmer would explain (eg array iteration instead of sigma notation) ? As far as i understand, Kalman Filters seem to be a moving average of position, velocity, and (perhaps) acceleration, and use those 3 values to estimate the 'real value' as opposed to what the sensor is telling you.
- deleted 3y ago[deleted]
- dilyevsky 3y ago> As far as i understand, Kalman Filters seem to be a moving average of position, velocity, and (perhaps) acceleration, and use those 3 values to estimate the 'real value' as opposed to what the sensor is telling you. Sort of. The actual variables (v, r, a in your case) vary depending on your use case. What matters is you make an estimate based on that state, then you take a measurement and there’s an iterative way of adjusting your next state based on those two. It’s actually pretty easy to code once you understand formulas. Maybe this could help http://bilgin.esme.org/BitsAndBytes/KalmanFilterforDummies http://bilgin.esme.org/BitsAndBytes/KalmanFilterforDummies
- chubs 3y agoThanks for the link! That's the best i've read so far :)
- TFortunato 3y agoHoping I can help you grok the problem a bit more. Typing on phone, so please excuse the lack of decent formatting. A moving average is a good place to start thinking about it. Think about a case where you would use a moving average, and why. You are probably using it because you have some measurement (sensor) that you know is noisy, and so by averaging out the noise (taking the mean of a number of samples) you try to get a better estimate of the true value. If you know how noisy the sensor is, you can get an idea of how many samples you should average over to get a good measurement. You can also take the standard deviation and report both the mean and variance of your measurement over multiple samples if you wanted. For purposes of this example moving forward -- we are going to estimate all of our sensed or inferred values as a gaussian, parameterized by mean and variance. It's a simple way to give a measurement with some uncertainty around it. This can be a good start if you know nothing little about the system you are measuring, other then the sensor / measurement is noisy. However for many systems you may have multiple quantities you are interested in estimating, and you have some idea of how they relate to each other. Take your example of a physical system with acceleration, velocity and position. Basic physics will tell you that if at time t, you are at position p, moving at velocity v, then at time t+dt, your position should be roughly p+(v*dt). Similarly, you can update your velocity estimate using your estimate of acceleration. If this is a system under your control, then you can also take things like a force you commanded to update your acceleration model. This is great, by using physics, without any measuring after time 0, we can just figure everything out forward in time forever, by simply using our process model! However, because your initial estimates had some uncertainty, what you will find is that, if you just keep doing this, the uncertainty grows larger and larger with each time step, and eventually become so large as to be useless. Enter tha kalman filter. What the kalman filter does is tries to combine the information given by your sensors and combine it with your process model to give you a better estimate of the quantities you are interested in than you could get from either technique alone. Every time step the filter will make an state estimate using the process model based on your previous state estimate, and then use your current sensor measurements to update that state estimate, both in terms of the mean and uncertainty. In the basic kalman filter you assume your process model is linear, all of your estimates are simple gaussians, and then decide how much you want to weigh your model vs. your sensors via a simple multiplying factor, "the kalman gain" Sorry again I couldn't write this out as a program, and the likely horrible run-on sentences that come from typing on a phone, but I hope that a quick overview of what the technique is trying to do will help make it a little easier to fill in the owl!
- p5a0u9l 3y agoI love ELI5 articles. I work in fields that use Kalman Filters. I have an MS in EE. I’ve been in industry for more than ten years and tried my hand with more than one KF. I know the terms and mostly grok the math. I’m a Senior trending to Principal at a faang, working in robotics. I STILL STRUGGLE and have never deployed a KF to production. Side story - a friend of mine tried to recruit me to a well known robotics startup in the commercial autonomous air vehicle space (DM me if you’re curious who). The technical portion was basically, “implement a state estimator for radar target tracking, In one day, here’s some data”. Meanwhile, I have a day job. Lol, I told ‘em, “I’m not a good fit if this is what you’re looking for”. Oh yeah, my recruiter/friend also said they were looking for rock stars. I feel like I really dodged a bullet there.
- deleted 3y ago[deleted]
- zogomoox 3y agoThis reminds me of the great "the missile knows" clip: https://www.youtube.com/watch?v=bZe5J8SVCYQ https://www.youtube.com/watch?v=bZe5J8SVCYQ
- jiggawatts 3y agoKalman filters are great when there is limited compute available, but I’m a bigger fan of the newer and more advanced models such as “particle filters”. The advantage of particle filters is that they can handle complex scenarios with nonlinear physics and non-Gaussian distributions. For example, a vehicle GPS unit could use street maps to eliminate impossible locations based on the recent history of turns. Gaussian filters can’t do that and just result in blurry blobs that cover several blocks.
- Scipio_Afri 3y agoDo you have any recommended resources for learning about particle filters? Also l, what other types of newer models or more advanced models other than the particle filter would you recommend for offline processing?
- jiggawatts 3y agoUnfortunately I don't have anything at hand better than what Google would provide.
- esafak 3y agoLook up the positioning and tracking literature; e.g., Probabilistic Robotics by Thrun et al. The method is also called "sequential Monte Carlo". I found this article useful in learning the method: Bayesian Filtering for Location Estimation (https://rse-lab.cs.washington.edu/postscripts/bayes-filter-pervasive-03.pdf https://rse-lab.cs.washington.edu/postscripts/bayes-filter-p...) It might help to first review importance sampling. https://en.wikipedia.org/wiki/Importance_sampling https://en.wikipedia.org/wiki/Importance_sampling A comprehensive survey (for 2003) is Zhe Chen's "Bayesian Filtering: From Kalman Filters to Particle Filters, and Beyond". A contemporary survey would surely incorporate machine learning. https://people.bordeaux.inria.fr/pierre.delmoral/chen_bayesian.pdf https://people.bordeaux.inria.fr/pierre.delmoral/chen_bayesi... For extensions see "An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo" https://people.eecs.berkeley.edu/~jordan/sail/readings/cappe-godsill-moulines.pdf https://people.eecs.berkeley.edu/~jordan/sail/readings/cappe...
- mojomark 3y agoI always get excited when I see these "tutorials for dummies" (like me). Like "finally, I get to take an evening to understand this concept that's eluded me for years." Generally, I get let down. This time is no exception. They always start off well, then inevitably there's a concept or key terminology that gets glossed over without sufficient explanation. . "The random variable is described by the probability density function. The probability density function is characterized by moments. The moments of the random value are expected values of powers of the random variable. We are interested in two types of moments" This is where my journey ended this time. OK, so is that the expected value of the expontnent.of the random value orthe expected value of the random value to some power... and what makes the power of a randome value so special vice just operating on the randome value itself. It's like these authors get tired of "dumbing" things down at random points in their chain of thought and just decide to skip over stuff. Or, perhaps they don't understand the underlying concepts themselves and simply can't explain them to others. Who knows. Just frustrating. At least in Udemy you can write to instructors and ask questions. Unlike a book author who aren't paid to respond.
- dkarras 3y agoI feel this frustration in my bones, and I felt the same way up until the proliferation of LLMs. Just paste your comment to Claude (or ChatGPT maybe?) and it will explain everything you want to know in realtime.
- mojomark 3y agoHa, no joke. I definitely find myself doing that more and more. Good reminder.
- bigbillheck 3y agoSure, if you don't care if it's actually correct or not.
- bootsmann 3y agoThere are so many explanations of moments in the training set, it should be correct.
- donpark 3y agoAnother good article on the subject: Is the Kalman filter a low-pass filter? Sometimes! https://jbconsulting.substack.com/p/is-the-kalman-filter-just-a-low-pass https://jbconsulting.substack.com/p/is-the-kalman-filter-jus... PS: I've used it to remove jitter in virtual camera movement while cropping video around faces in real-time, streaming detected face locations to a Kalman filter worker and get back stream of stable camera locations.
- akexbe 3y agoHello mojomark, My name is Alex Becker, and I am the author of the "Kalman Filter from the Ground Up book." First, I would like to thank you for your feedback. As an author, it is important for me to receive feedback from the readers. In the book's second edition, I made many changes based on the feedback from the book readers. I've elaborated explanations of the topics some readers failed to understand and added several appendixes. You are right, book authors don't get paid to respond. However, the KalmanFilter.Net is a project, not just a book. I've worked on the project for 6 years, and you will see more in the future. The goal of this project is to help people to understand and implement the Kalman Filter. The website includes a contact page: https://www.kalmanfilter.net/contact.aspx https://www.kalmanfilter.net/contact.aspx, and the book purchasers receive my email. I do my best to answer emails and help my readers if they fail to understand specific points. Regarding the issues that you and other responders raised. Scientists use terminology that looks weird to non-scientists. But I can't write an engineering book without addressing scientific terms. I agree that "Random Variable" sounds unintuitive, but this is the term. Almost any physical quantity is a random variable. For example, the temperature of your body is a random variable. You can measure the temperature using a thermometer, but it doesn't mean that its reading is your actual temperature. Your temperature is a random variable that can be described by two parameters – the thermometer reading and the thermometer precision. Understanding the "Random Variables" concept is essential for understanding the Kalman Filter. If you make multiple measurements, the temperature is described by mean and variance. You can read again the https://www.kalmanfilter.net/background.html https://www.kalmanfilter.net/background.html page, where this concept is explained. Mathematicians call the mean and the variance – statistical moments. "Moments" also sounds weird. I mention this term in the introduction, but I don't use it again in the book. If the term "moments" causes confusion, I can consider removing it from the introduction. Regarding the Linear Algebra. You don't need any prior Linear Algebra knowledge to understand the concept of the Kalman Filter and implement the Kalman Filter in one dimension. However, if you want to tackle the multi-dimensional Kalman Filter, you need some basic Linear Algebra knowledge, at least matrix multiplication. Thanks again for your post, Alex Becker.
- alex_be 3y agoHello mojomark, My name is Alex Becker, and I am the author of the "Kalman Filter from the Ground Up book." First, I would like to thank you for your feedback. As an author, it is important for me to receive feedback from the readers. In the book's second edition, I made many changes based on the feedback from the book readers. I've elaborated explanations of the topics some readers failed to understand and added several appendixes. You are right, book authors don't get paid to respond. However, the KalmanFilter.Net is a project, not just a book. I've worked on the project for 6 years, and you will see more in the future. The goal of this project is to help people to understand and implement the Kalman Filter. The website includes a contact page: https://www.kalmanfilter.net/contact.aspx https://www.kalmanfilter.net/contact.aspx, and the book purchasers receive my email. I do my best to answer emails and help my readers if they fail to understand specific points. Regarding the issues that you and other responders raised. Scientists use terminology that looks weird to non-scientists. But I can't write an engineering book without addressing scientific terms. I agree that "Random Variable" sounds unintuitive, but this is the term. Almost any physical quantity is a random variable. For example, the temperature of your body is a random variable. You can measure the temperature using a thermometer, but it doesn't mean that its reading is your actual temperature. Your temperature is a random variable that can be described by two parameters – the thermometer reading and the thermometer precision. Understanding the "Random Variables" concept is essential for understanding the Kalman Filter. If you make multiple measurements, the temperature is described by mean and variance. You can read again the https://www.kalmanfilter.net/background.html https://www.kalmanfilter.net/background.html page, where this concept is explained. Mathematicians call the mean and the variance – statistical moments. "Moments" also sounds weird. I mention this term in the introduction, but I don't use it again in the book. If the term "moments" causes confusion, I can consider removing it from the introduction. Regarding the Linear Algebra. You don't need any prior Linear Algebra knowledge to understand the concept of the Kalman Filter and implement the Kalman Filter in one dimension. However, if you want to tackle the multi-dimensional Kalman Filter, you need some basic Linear Algebra knowledge, at least matrix multiplication. Thanks again for your post, Alex Becker.
- edub 3y agoI wasn't familiar with Kalman, and I tried to understand it by reading the primers on this website and it wasn't clicking for me. Last night I was thinking about the sample graphs, and it occurred to me that it just looked like an Exponential Moving Average, which is a much simpler concept to grasp. I just searched Google for a comparison and found that EMA is as good as Kalman for "random walk plus noise" on a Stats Stack Exchange, and that a 2003 paper from Brown University (Joseph J. LaViola) showed that a Double Exponential Smoothing algorithm is of equal quality to Kalman (and extended Kalman) but 135 times faster, and a simpler approach. I find Double Exponential Smoothing to be much easier to understand than Kalman, and assuming the LaViola paper is correct, I'm not going to put additional effort into understanding Kalman.