4 ms·

Probability notation generally works best for the people who already understand the concept in question. Let me take a crack at your question. The equation in

Probability notation generally works best for the people who already understand the concept in question. Let me take a crack at your question. The equation in question is P(A|B) = P(B|A) P(A) / P(B). In modern Kalman filter literature, this would be stated as something like: P(x_k | z_k) = P(z_k | x_k) P(x_k) / P(z_k). It is generally left as implicit in these sorts of equations that everything is also conditioned on the sequence z_1 to z_{k-1}. In this equation, x_k is a free variable in the state space (possibly multi-dimensional, so a vector), while z_k is the measurement, which is a realization of the random variable distributed as N(H(x_k), R_k). The result is a PDF over the free variable x_k. So let's tackle the terms one by one: 1. P(z_k | x_k) - this is the probability that we measured z_k, given that the true object is at x_k. This is the aforementioned normal distribution N(H(x_k), R_k). 2. P(x_k) - this is the prior probability of the state estimate, generally after propagation through the motion model from P(x_{k-1} | x_{k-1}). In the Kalman filter, this is also Gaussian, and conveniently has the same mean as the term above (see note below). 3. P(z_k) - this is the denominator that someone else mentioned earlier can be effectively ignored, which is right - you only need it to normalize the numerator. If you must compute it, it can be factored as the integral over the entire state space of P(z_k|x_k)*p(x_k). Given z_k, this is a number, not a function. 4. P(x_k | z_k) - The result, which is a Gaussian PDF. You can arrive at it numerically by plugging in specific values for x_k, in which case (1) and (2) are numbers. Or symbolically, in which case (1) and (2) are functions, and you'll end up with the form of a Gaussian PDF. Note: The original article quotes a distribution for the product of two Gaussians with arbitrary means. It does not state that this is an approximation, which is exact only in the case of equal means. This is why unbiased measurements are one of the Kalman assumptions.

gugagore 9y ago
I think the crux of the complaint of is the imprecision in saying e.g. "P(z_k | x_k) - this is the probability that we measured z_k, given that the true object is at x_k" Technically, you can only give a useful answer about the probability density of the random variable Z_k at the value z_k, conditioned on X_k = x_k. In the Bayesian interpretation of the Kalman filter, you never have an event "I measured z_k" (that event has probability 0, of course). I agree that the probability notation is the issue here. Look at how wikipedia shows Bayes' rule for continuous random variables on both sides of the |. : https://en.wikipedia.org/wiki/Bayes%27_theorem#Random_variables https://en.wikipedia.org/wiki/Bayes%27_theorem#Random_variab... That's the kind of explicit and precise notation I would use to help someone understand the Kalman filter from a Bayesian perspective. Once you use that definition of Bayes' rule, then you can substitute the definitions of the multivariate normal pdf, Do The Math, and derive the Kalman filter recursive updates.
- civility 9y ago
  Thank you for your reply here, and the one below. I wish I had seen it before posting my sibling message to this one. I'm a bit too tired to go any further with this tonight, but I plan to look at your links tomorrow. Cheers.
civility 9y ago
In term 1, you say P(z_k | x_k) is a probability (which should be a scalar value between 0.0 and 1.0 inclusive). How did we get from the vector pdfs of measurement and estimate to a scalar probability? Maybe you meant evaluating a pdf at an arbitrary point z_k, getting its density (scalar non-negative value, between 0.0 and infinity) at that point? I have the same confusion on your other enumerated terms. If they are probabilities, I'm only familiar with getting those by integrating the pdfs over some set of bounds or range. I've seen the k subscripts plenty before, and some people use + and - to indicate pre and post parts of an operation, others put conditional vertical bars in the subscripts, but your definition for the pdf of z_k is a bit different than I'm used to. If we focus on the current instant of time (dropping the subscripts), ignore the motion model part, and only deal with a single measurement, I would say the estimate and measurement are defined like this: estimate ~ Normal(mean=x, covar=P) measurement ~ Normal(mean=z, covar=R) Then there is an elegant formula looking at it as an information filter, using Matlab-ish notation: Y = inv(P) + H'*inv(R)*H y = inv(P)*x + H'*inv(R)*z update ~ Normal(mean=inv(Y)*y, covar=inv(Y)) Ignoring the information filter stuff, and getting back to your list, Term 1 looks like a pdf with the dimensionality of z, but the result in Term 4 is a pdf with the dimensionality of x. In otherwords, the resultant Term 4 is a pdf (function) which takes a vector argument in the x space, and I don't see where to pick any given z for the pdf in Term 1. It's getting late, and I'm getting spacey, but it's tempting to try and write the type signatures of these functions in some strongly typed pseudocode so I can show you my confusion.
- abstrakraft 9y ago
  Concerning the difference between our definitions of measurement, I was referring to the prior probability of making a measurement given the true state, which is P(z|x), or, if you prefer, f(z,x) = P(Z=z|X=x). This PDF is Normal(Hx, R). The distribution you refer to is that of the image of the true state in measurement space (Hx), which is Normal(z, R). Concerning the general question of dimensionality and numbers vs. functions, I'm afraid I still don't understand the fundamental issue. I agree that probability notation isn't the best, but each of those terms can be replaced with a Gaussian (or in the case of the denominator, an integral of the product of two Gaussians). You end up with a function in terms of x_k and the sequence of z's. You can then choose which variables are free or bound and evaluate the function as your application requires. If all variables are bound (the case that you know the measurement z_k, and are evaluating the posterior probability of the state at a given point), all the terms are numbers (yes, densities), and there are no dimensional issues. If you leave x_k free, you end up with two functions in the numerator, and a number in the denominator. Those two functions are Mahalanobis distances (which are scalars) in an exponent, and again, no dimensional issues. In general, such operations with PDFs represent various combinations and manipulations of event spaces that need not be commensurate. Computing the PDF of a coin flip given the distribution of all of the positions and velocities of the molecules of air through which the coin flies requires comparing a two-state PMF to a virtually infinite dimensional PDF. Notationally speaking, no problem.

gugagore 9y ago

I think the crux of the complaint of is the imprecision in saying e.g. "P(z_k | x_k) - this is the probability that we measured z_k, given that the true object is at x_k" Technically, you can only give a useful answer about the probability density of the random variable Z_k at the value z_k, conditioned on X_k = x_k. In the Bayesian interpretation of the Kalman filter, you never have an event "I measured z_k" (that event has probability 0, of course). I agree that the probability notation is the issue here. Look at how wikipedia shows Bayes' rule for continuous random variables on both sides of the |. : https://en.wikipedia.org/wiki/Bayes%27_theorem#Random_variables https://en.wikipedia.org/wiki/Bayes%27_theorem#Random_variab... That's the kind of explicit and precise notation I would use to help someone understand the Kalman filter from a Bayesian perspective. Once you use that definition of Bayes' rule, then you can substitute the definitions of the multivariate normal pdf, Do The Math, and derive the Kalman filter recursive updates.

civility 9y ago
Thank you for your reply here, and the one below. I wish I had seen it before posting my sibling message to this one. I'm a bit too tired to go any further with this tonight, but I plan to look at your links tomorrow. Cheers.

civility 9y ago

In term 1, you say P(z_k | x_k) is a probability (which should be a scalar value between 0.0 and 1.0 inclusive). How did we get from the vector pdfs of measurement and estimate to a scalar probability? Maybe you meant evaluating a pdf at an arbitrary point z_k, getting its density (scalar non-negative value, between 0.0 and infinity) at that point? I have the same confusion on your other enumerated terms. If they are probabilities, I'm only familiar with getting those by integrating the pdfs over some set of bounds or range. I've seen the k subscripts plenty before, and some people use + and - to indicate pre and post parts of an operation, others put conditional vertical bars in the subscripts, but your definition for the pdf of z_k is a bit different than I'm used to. If we focus on the current instant of time (dropping the subscripts), ignore the motion model part, and only deal with a single measurement, I would say the estimate and measurement are defined like this: estimate ~ Normal(mean=x, covar=P) measurement ~ Normal(mean=z, covar=R) Then there is an elegant formula looking at it as an information filter, using Matlab-ish notation: Y = inv(P) + H'*inv(R)*H y = inv(P)*x + H'*inv(R)*z update ~ Normal(mean=inv(Y)*y, covar=inv(Y)) Ignoring the information filter stuff, and getting back to your list, Term 1 looks like a pdf with the dimensionality of z, but the result in Term 4 is a pdf with the dimensionality of x. In otherwords, the resultant Term 4 is a pdf (function) which takes a vector argument in the x space, and I don't see where to pick any given z for the pdf in Term 1. It's getting late, and I'm getting spacey, but it's tempting to try and write the type signatures of these functions in some strongly typed pseudocode so I can show you my confusion.

abstrakraft 9y ago
Concerning the difference between our definitions of measurement, I was referring to the prior probability of making a measurement given the true state, which is P(z|x), or, if you prefer, f(z,x) = P(Z=z|X=x). This PDF is Normal(Hx, R). The distribution you refer to is that of the image of the true state in measurement space (Hx), which is Normal(z, R). Concerning the general question of dimensionality and numbers vs. functions, I'm afraid I still don't understand the fundamental issue. I agree that probability notation isn't the best, but each of those terms can be replaced with a Gaussian (or in the case of the denominator, an integral of the product of two Gaussians). You end up with a function in terms of x_k and the sequence of z's. You can then choose which variables are free or bound and evaluate the function as your application requires. If all variables are bound (the case that you know the measurement z_k, and are evaluating the posterior probability of the state at a given point), all the terms are numbers (yes, densities), and there are no dimensional issues. If you leave x_k free, you end up with two functions in the numerator, and a number in the denominator. Those two functions are Mahalanobis distances (which are scalars) in an exponent, and again, no dimensional issues. In general, such operations with PDFs represent various combinations and manipulations of event spaces that need not be commensurate. Computing the PDF of a coin flip given the distribution of all of the positions and velocities of the molecules of air through which the coin flies requires comparing a two-state PMF to a virtually infinite dimensional PDF. Notationally speaking, no problem.