4 ms·

Priors are not just numbers between 0 and 1. They are probability distributions. If they were not, it would be impossible to update beliefs based on evidence.

Priors are not just numbers between 0 and 1. They are probability distributions. If they were not, it would be impossible to update beliefs based on evidence. My prior beliefs about the coin include the possibility that the coin has a bias other than 50%. This distribution may be centered at 50%, but I may assign more or less probability to extreme biases, and this determines how I update my belief when I observe several heads in a row.

lisper 4y ago
Of course it is possible to update beliefs based on evidence if priors are numbers. If I flip a coin but don't show you the outcome, you will rationally assign a prior of 0.5 to the proposition "The coin landed heads-up". If I then show you that the coin did indeed land heads-up (or tails-up) you will update your assessment of the probability of "The coin landed heads up" to something close to 1 (or 0). Now, it is possible to do Bayesian analysis on probability distributions rather than "bare" priors, but that is just a way of assigning priors to a family of propositions rather than single propositions. For example, "This is a fair coin" is not a proposition to which one can assign a prior because it isn't a well-defined proposition. The intuition behind "fair coin" is something like "the ratio of heads to tails will approach 1 as the number of flips goes to infinity". But no coin can be flipped an infinite number of times, so this is a meaningless definition. Furthermore, consider a coin that always landed on the face opposite that of the previous flip. The ratio of heads to tails for such a bizarre coin would indeed approach 1 as the number of flips approached infinity, but that is not what is generally meant by a "fair coin". So the usual way of dealing with coins is assigning priors to the family of propositions, "If I flip this coin N times the probability that I will get M heads is X" for all possible values of N and M. But each of the propositions in that family gets a prior which is a number.
- cf 4y ago
  That framing actually stops holding for continuous distributions. Like in a distribution on heights of people there isn't a probability or prior associated with a particular height. Probabilities are assigned to subsets of this distribution (sometimes called sigma algebras). Priors stands for prior distribution and it really is best to see Bayesian analysis as a machine that consumes probability distributions as input and spits out probability distributions as output.
  - lisper 4y ago
    > That framing actually stops holding for continuous distributions. No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number. > Priors stands for prior distribution No. Bayes's theorem is: P(H|E) = P(E|H) * P(H) / P(E) The "prior" is P(H) which is a probability, i.e. a number between 0 and 1, as are all the other "P" values. The "*" and "/" operations in Bayes's theorems are scalar multiplication and division.
    - cf 4y ago
      > No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number. A continuous distribution does not assign probabilities to each proposition but subsets of them. To see this concretely consider the continuous Uniform distribution from 1 to 1.5. It will for all values of its support have a probability density of 2. Most people would not consider 2 a probability. For continuous distributions, Bayes's theorem becomes about probability densities and not probabilities.
      - lisper 4y ago
        > A continuous distribution does not assign probabilities to each proposition but subsets of them. Nope. It assigns probabilities to individual propositions. > Most people would not consider 2 a probability. That's true, but in your example 2 is not a probability but a probability density, and a probability density is not the same as a probability. To get a probability out of a probability density you have to integrate. For each possible interval over which you could integrate there is a corresponding proposition whose probability of being true is exactly the value of the integral.
        cf 4y ago
        Ok I see where my confusion was. For you propositions are things that can be associated with measurable sets.
- jwarden 4y ago
  But priors are not *just* numbers between 0 and 1, as GP claimed. It is true that in your example, the outcome is binary: there is only a single proposition (the coin landed heads). And so the prior distribution is a Bernoulli distribution, but I would agree you could think of this prior as a single number between 0 and 1. But in such a scenario, the only possibilities for updating beliefs based on evidence is to observe heads, then update the posterior to P(Heads)=100%, or observe tails, then update posterior to P(Heads)=0%. I might say that this is updating your beliefs based on evidence, but only trivially so. If you observe heads and form a posterior of P(Heads)=99.999%, then it means that your prior probability distribution was more complex: it included the possibility that "the coin landed heads but I mistakenly observed tails". In this case, your priors include a family of propositions, not a single number. In GPs example, the prior included the possibility that the coin was magnetic, etc. So it could not be represented by a single number.
  - lisper 4y ago
    > the prior distribution is a Bernoulli distribution No, it isn't. The prior on the outcome of multiple tosses is a Bernoulli distribution, but the prior on the outcome of a single toss is a number. And for a fair coin, that number is 0.5 before you see the result, and it is either 0 or 1 after. Note that "This coin will land/landed heads" is a different proposition than "this coin is a fair coin." The latter is much more problematic because it is hard to define what a "fair coin" is. > your prior probability distribution was more complex No. You are confusing my prior on a single proposition (a number) with the set of priors on all propositions which I have considered in my lifetime. The latter is obviously a set of numbers, one for each proposition. > "the coin landed heads but I mistakenly observed tails" Yes, that is two separate propositions: 1. The coin landed heads. 2. My senses are reliable. All evidence must be assessed relative to some "background model", a set of propositions whose priors are very nearly 1. And it is also possible to model an infinite family of propositions about continuous properties, e.g. The length of that rod is between X and Y for any two real numbers X and Y. But none of this changes the fact that for any single proposition the prior is a number. > it could not be represented by a single number That depends on what you mean by "it". The probability that the coin landed heads certainly can be represented by a single number, as can the probability that the coin is magnetic. Two different propositions, hence two different priors.

lisper 4y ago

Of course it is possible to update beliefs based on evidence if priors are numbers. If I flip a coin but don't show you the outcome, you will rationally assign a prior of 0.5 to the proposition "The coin landed heads-up". If I then show you that the coin did indeed land heads-up (or tails-up) you will update your assessment of the probability of "The coin landed heads up" to something close to 1 (or 0). Now, it is possible to do Bayesian analysis on probability distributions rather than "bare" priors, but that is just a way of assigning priors to a family of propositions rather than single propositions. For example, "This is a fair coin" is not a proposition to which one can assign a prior because it isn't a well-defined proposition. The intuition behind "fair coin" is something like "the ratio of heads to tails will approach 1 as the number of flips goes to infinity". But no coin can be flipped an infinite number of times, so this is a meaningless definition. Furthermore, consider a coin that always landed on the face opposite that of the previous flip. The ratio of heads to tails for such a bizarre coin would indeed approach 1 as the number of flips approached infinity, but that is not what is generally meant by a "fair coin". So the usual way of dealing with coins is assigning priors to the family of propositions, "If I flip this coin N times the probability that I will get M heads is X" for all possible values of N and M. But each of the propositions in that family gets a prior which is a number.

cf 4y ago
That framing actually stops holding for continuous distributions. Like in a distribution on heights of people there isn't a probability or prior associated with a particular height. Probabilities are assigned to subsets of this distribution (sometimes called sigma algebras). Priors stands for prior distribution and it really is best to see Bayesian analysis as a machine that consumes probability distributions as input and spits out probability distributions as output.
- lisper 4y ago
  > That framing actually stops holding for continuous distributions. No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number. > Priors stands for prior distribution No. Bayes's theorem is: P(H|E) = P(E|H) * P(H) / P(E) The "prior" is P(H) which is a probability, i.e. a number between 0 and 1, as are all the other "P" values. The "*" and "/" operations in Bayes's theorems are scalar multiplication and division.
  - cf 4y ago
    > No, it doesn't. A continuous distribution is just a way of assigning probabilities to a parameterized set of propositions where the parameters are continuous and so the set is infinite. But for any given proposition in that set the prior is a number. A continuous distribution does not assign probabilities to each proposition but subsets of them. To see this concretely consider the continuous Uniform distribution from 1 to 1.5. It will for all values of its support have a probability density of 2. Most people would not consider 2 a probability. For continuous distributions, Bayes's theorem becomes about probability densities and not probabilities.
    - lisper 4y ago
      > A continuous distribution does not assign probabilities to each proposition but subsets of them. Nope. It assigns probabilities to individual propositions. > Most people would not consider 2 a probability. That's true, but in your example 2 is not a probability but a probability density, and a probability density is not the same as a probability. To get a probability out of a probability density you have to integrate. For each possible interval over which you could integrate there is a corresponding proposition whose probability of being true is exactly the value of the integral.
      - cf 4y ago
        Ok I see where my confusion was. For you propositions are things that can be associated with measurable sets.
jwarden 4y ago
But priors are not *just* numbers between 0 and 1, as GP claimed. It is true that in your example, the outcome is binary: there is only a single proposition (the coin landed heads). And so the prior distribution is a Bernoulli distribution, but I would agree you could think of this prior as a single number between 0 and 1. But in such a scenario, the only possibilities for updating beliefs based on evidence is to observe heads, then update the posterior to P(Heads)=100%, or observe tails, then update posterior to P(Heads)=0%. I might say that this is updating your beliefs based on evidence, but only trivially so. If you observe heads and form a posterior of P(Heads)=99.999%, then it means that your prior probability distribution was more complex: it included the possibility that "the coin landed heads but I mistakenly observed tails". In this case, your priors include a family of propositions, not a single number. In GPs example, the prior included the possibility that the coin was magnetic, etc. So it could not be represented by a single number.
- lisper 4y ago
  > the prior distribution is a Bernoulli distribution No, it isn't. The prior on the outcome of multiple tosses is a Bernoulli distribution, but the prior on the outcome of a single toss is a number. And for a fair coin, that number is 0.5 before you see the result, and it is either 0 or 1 after. Note that "This coin will land/landed heads" is a different proposition than "this coin is a fair coin." The latter is much more problematic because it is hard to define what a "fair coin" is. > your prior probability distribution was more complex No. You are confusing my prior on a single proposition (a number) with the set of priors on all propositions which I have considered in my lifetime. The latter is obviously a set of numbers, one for each proposition. > "the coin landed heads but I mistakenly observed tails" Yes, that is two separate propositions: 1. The coin landed heads. 2. My senses are reliable. All evidence must be assessed relative to some "background model", a set of propositions whose priors are very nearly 1. And it is also possible to model an infinite family of propositions about continuous properties, e.g. The length of that rod is between X and Y for any two real numbers X and Y. But none of this changes the fact that for any single proposition the prior is a number. > it could not be represented by a single number That depends on what you mean by "it". The probability that the coin landed heads certainly can be represented by a single number, as can the probability that the coin is magnetic. Two different propositions, hence two different priors.