3 ms·

Hi. As other posters have mentioned there is a closed form solution that comes straight from an easy differntial equation. I'd like to make a point to help th

by physPop 12y ago

Hi. As other posters have mentioned there is a closed form solution that comes straight from an easy differntial equation. I'd like to make a point to help the author out, since this post is in the spirit of simple explanations: a good way of thinking of Markov models is that they are fundamentally a discreet form of differential equations, and are useful when we have finitely few states that are generally: a) easily enumerable b) countably small enough that it doesn't make sense to just replace it with a continuous model This is similar to the way in which martingales are discreet representations of random walks. Sadly, our education system does a bad job of discreet math, and its typically left to stats courses -- so everything has a different name and few people draw the connections, IMO! Cheers

indeed30 12y ago
I'm struggling with this comment, I don't necessarily disagree with the spirit of it, but I feel that you're doing Markov chains something of a disservice by claiming they're mainly useful for "finitely few states". You're completely ignoring immensely important topics like MCMC which (can) operate on uncountable spaces. The vast, vast majority of MCMC research and usage is with continuous random variables.
joe_the_user 12y ago
Well, Markov chains as used by programmers are generally discreet-time. Continuous-time Markov chains are continuous. Indeed, in the original model he begins with what seems to be a continuous-time Markov chain. You are correct you get to solve a linear differential equation with constant coefficients to get the distribution at any one time. Another interesting way to consider this stuff is in terms of the "logarithms" and "exponentiation" of Markov matrices. http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Continuous-time_Markov_chain http://en.wikipedia.org/wiki/Continuous-time_Markov_chain
mturmon 12y ago
This comment is problematic. You're confusing discrete-state-space models with discrete-time models. You can have any combination of continuous/discrete Markov process: Discrete states, discrete time: text, etc. Discrete states, continuous time: Poisson counting process Continuous states, discrete time: gambling earnings Continuous states, continuous time: Brownian motion Of course, the term "Markov chain" is restricted to discrete time. But you have said "Markov model", which includes all processes with the Markov property (i.e., they forget old state). The model in the OP was not originally a chain, because the radioactive decay process is continuous-time, but he made it into one. When you say that Markov chains are discretized differential equations, it tempts the reader to think "discretized in time" -- but that's called a difference equation. I think you mean it's discretized in state. I.e., there is a flow of mass from one state to another that is smooth in a diff. eq., and stochastic in the Markov chain. But that's just guesswork. Also, I would not say a martingale is a discrete random walk. Your statement here introduces the same confusion. Like Markov processes, martingales can be continuous or discrete in both state-space and time (e.g., Brownian motion is a martingale). I think of a martingale as a mathematical model for a fair game (expected future earnings is zero given available information).
- physPop 12y ago
  Fair concerns, thank you for clarifying the notation, you are correct in my misuse of model/chain. Was typing fast :) After some further reading , I also agree re: martingales. To be honest I have only ever encountered them in the common discreet-time situation.

by physPop 12y ago

indeed30 12y ago

I'm struggling with this comment, I don't necessarily disagree with the spirit of it, but I feel that you're doing Markov chains something of a disservice by claiming they're mainly useful for "finitely few states". You're completely ignoring immensely important topics like MCMC which (can) operate on uncountable spaces. The vast, vast majority of MCMC research and usage is with continuous random variables.

joe_the_user 12y ago

Well, Markov chains as used by programmers are generally discreet-time. Continuous-time Markov chains are continuous. Indeed, in the original model he begins with what seems to be a continuous-time Markov chain. You are correct you get to solve a linear differential equation with constant coefficients to get the distribution at any one time. Another interesting way to consider this stuff is in terms of the "logarithms" and "exponentiation" of Markov matrices. http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Markov_chain http://en.wikipedia.org/wiki/Continuous-time_Markov_chain http://en.wikipedia.org/wiki/Continuous-time_Markov_chain

mturmon 12y ago

This comment is problematic. You're confusing discrete-state-space models with discrete-time models. You can have any combination of continuous/discrete Markov process: Discrete states, discrete time: text, etc. Discrete states, continuous time: Poisson counting process Continuous states, discrete time: gambling earnings Continuous states, continuous time: Brownian motion Of course, the term "Markov chain" is restricted to discrete time. But you have said "Markov model", which includes all processes with the Markov property (i.e., they forget old state). The model in the OP was not originally a chain, because the radioactive decay process is continuous-time, but he made it into one. When you say that Markov chains are discretized differential equations, it tempts the reader to think "discretized in time" -- but that's called a difference equation. I think you mean it's discretized in state. I.e., there is a flow of mass from one state to another that is smooth in a diff. eq., and stochastic in the Markov chain. But that's just guesswork. Also, I would not say a martingale is a discrete random walk. Your statement here introduces the same confusion. Like Markov processes, martingales can be continuous or discrete in both state-space and time (e.g., Brownian motion is a martingale). I think of a martingale as a mathematical model for a fair game (expected future earnings is zero given available information).

physPop 12y ago
Fair concerns, thank you for clarifying the notation, you are correct in my misuse of model/chain. Was typing fast :) After some further reading , I also agree re: martingales. To be honest I have only ever encountered them in the common discreet-time situation.