4 ms·

Bayesian statistics is dumb. What's the point of using prior assumptions that are based on speculation? It's better to admitting your lack of knowledge, not jum

Bayesian statistics is dumb. What's the point of using prior assumptions that are based on speculation? It's better to admitting your lack of knowledge, not jumping to conclusions without sufficient data, and processing data in an unbiased manner.

kgwgk 3y ago
Too bad that the prior is fixed and you cannot change it to represent your lack of knowledge, eh?
- tesdinger 3y ago
  The prior is a way to condense knowledge into math. A prior doesn't make sense if there is a lack of knowledge.
  - kgwgk 3y ago
    What does “lack of knowledge” mean? If it means “any value is as plausible as any other as far as I know” there is a prior for that. Etc.
    - tesdinger 3y ago
      > any value is as plausible as any other as far as I know” That isn't possible to use as a prior as the uniform distribution from negative to positive infinity is zero everywhere.
      - kgwgk 3y ago
        It’s easy to use a uniform prior in a suitably huge interval - say from minus one gazillion to plus one gazillion - and you can look at the limit when the endpoints go to minus/plus infinity if you really have doubts about whether the interval was huge enough for your problem to fit comfortably within it. If you don’t think that this is possible that says more about you than about the shortcomings of Bayesian statistics.
        tesdinger 3y ago
        What would you do if you see an observation outside of the interval?
        kgwgk 3y ago
        Use a larger interval? As I said, you can define the improper uniform prior solution as the limit of a sequence of solutions corresponding to a sequence of increasingly wider intervals with endpoints that go to +/-infinity. (And as I said, you can start with a suitably huge region. Say that you want to determine the position of something and use a uniform prior that extends to a distance of 10^27m - a perfectly bounded prior from a mathematical point of view that covers the whole observable universe. If you observe something outside it, it’s not with the prior that you have a problem.)
      - _yb2s 3y ago
        Can you give an example of a problem where there is no possible prior information, and states all the way from negative to positive infinity are equally plausible? I don't think the laws of physics (not to mention, the practical limits of computation) allow for real world inference problems where there are no bounds on a prior of any kind. Failing that, I don't think there are any human usable data sources that could report observations over an infinite interval.
deleted 3y ago
[deleted]
_yb2s 3y ago
The prior represents your current state of knowledge, whatever that may be. If you have a total lack of knowledge, your prior is uniform over all possible states. It is never proper to fabricate a biased (e.g. non-uniform) prior without any knowledge/information, that is not in any way part of Bayesian Inference. If you do have some extremely weak evidence, you use it accordingly with an extremely weak prior. To paraphrase E.T. Jaynes, the rules of probability theory (e.g. Bayes Theorem) are the unique logically consistent way to reason about uncertainty.
- lqr 3y ago
  How can your prior be uniform if the hypothesis class is unbounded?
  - kgwgk 3y ago
    How can the hypothesis class be unbounded? Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.
  - _yb2s 3y ago
    I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems. As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.
  - yellowcake0 3y ago
    This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is. A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.
- 3y ago

kgwgk 3y ago

Too bad that the prior is fixed and you cannot change it to represent your lack of knowledge, eh?

tesdinger 3y ago
The prior is a way to condense knowledge into math. A prior doesn't make sense if there is a lack of knowledge.
- kgwgk 3y ago
  What does “lack of knowledge” mean? If it means “any value is as plausible as any other as far as I know” there is a prior for that. Etc.
  - tesdinger 3y ago
    > any value is as plausible as any other as far as I know” That isn't possible to use as a prior as the uniform distribution from negative to positive infinity is zero everywhere.
    - kgwgk 3y ago
      It’s easy to use a uniform prior in a suitably huge interval - say from minus one gazillion to plus one gazillion - and you can look at the limit when the endpoints go to minus/plus infinity if you really have doubts about whether the interval was huge enough for your problem to fit comfortably within it. If you don’t think that this is possible that says more about you than about the shortcomings of Bayesian statistics.
      - tesdinger 3y ago
        What would you do if you see an observation outside of the interval?
        kgwgk 3y ago
        Use a larger interval? As I said, you can define the improper uniform prior solution as the limit of a sequence of solutions corresponding to a sequence of increasingly wider intervals with endpoints that go to +/-infinity. (And as I said, you can start with a suitably huge region. Say that you want to determine the position of something and use a uniform prior that extends to a distance of 10^27m - a perfectly bounded prior from a mathematical point of view that covers the whole observable universe. If you observe something outside it, it’s not with the prior that you have a problem.)
    - _yb2s 3y ago
      Can you give an example of a problem where there is no possible prior information, and states all the way from negative to positive infinity are equally plausible? I don't think the laws of physics (not to mention, the practical limits of computation) allow for real world inference problems where there are no bounds on a prior of any kind. Failing that, I don't think there are any human usable data sources that could report observations over an infinite interval.

deleted 3y ago

[deleted]

_yb2s 3y ago

The prior represents your current state of knowledge, whatever that may be. If you have a total lack of knowledge, your prior is uniform over all possible states. It is never proper to fabricate a biased (e.g. non-uniform) prior without any knowledge/information, that is not in any way part of Bayesian Inference. If you do have some extremely weak evidence, you use it accordingly with an extremely weak prior. To paraphrase E.T. Jaynes, the rules of probability theory (e.g. Bayes Theorem) are the unique logically consistent way to reason about uncertainty.

lqr 3y ago
How can your prior be uniform if the hypothesis class is unbounded?
- kgwgk 3y ago
  How can the hypothesis class be unbounded? Anyway, you can define a sequence of solutions with bounded uniform priors and calculate the limiting solution. For any given data set when the endpoints of the intervals go to +/-infinity the solution will converge to the uniform prior one - if it exist.
- _yb2s 3y ago
  I discussed this in another reply in this thread, but I can't personally think of any examples where the hypothesis is unbounded. The laws of physics, computation, the human mind, measuring instruments, etc. all impose bounds on real world problems. As someone that does Bayesian Inference a lot for my work (computational biology), I very often use uniform priors, but the structure of all real world problems I have ever encountered allows me specify hard bounds to the edges of non-zero probability.
- yellowcake0 3y ago
  This isn't always possible, but sometimes you can define what's called an improper prior p(µ), such that even if ∫p(µ) is not finite, the posterior distribution p(µ|x) is. A common example is when p(x|µ,v) is a Gaussian dist. with a prior on the mean set to p(µ)=1.
3y ago