3 ms·

The prior represents your current state of knowledge, whatever that may be. If you have a total lack of knowledge, your prior is uniform over all possible state

by _yb2s 3y ago

The prior represents your current state of knowledge, whatever that may be. If you have a total lack of knowledge, your prior is uniform over all possible states. It is never proper to fabricate a biased (e.g. non-uniform) prior without any knowledge/information, that is not in any way part of Bayesian Inference. If you do have some extremely weak evidence, you use it accordingly with an extremely weak prior. To paraphrase E.T. Jaynes, the rules of probability theory (e.g. Bayes Theorem) are the unique logically consistent way to reason about uncertainty.

lqr 3y ago
How can your prior be uniform if the hypothesis class is unbounded?
- kgwgk 3y ago
  How can the hypothesis class be unbounded? Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.
- _yb2s 3y ago
  I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems. As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.
- yellowcake0 3y ago
  This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is. A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.
yellowcake0 3y ago
I'm not sure total lack of knowledge can ever be encoded into a prior distribution, and if it can, I think Jeffreys prior makes a more compelling case than a uniform prior. After all, to believe that all possible values of the prior parameter are equally likely, is to believe that all possible values of its square are not.

by _yb2s 3y ago

lqr 3y ago

How can your prior be uniform if the hypothesis class is unbounded?

kgwgk 3y ago
How can the hypothesis class be unbounded? Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.
_yb2s 3y ago
I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems. As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.
yellowcake0 3y ago
This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is. A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.

yellowcake0 3y ago

I'm not sure total lack of knowledge can ever be encoded into a prior distribution, and if it can, I think Jeffreys prior makes a more compelling case than a uniform prior. After all, to believe that all possible values of the prior parameter are equally likely, is to believe that all possible values of its square are not.