4 ms·

Occam's razor, as stated by GP is not about correctness, but tractability. In fact, if you look at it probabilistically, the hypothesis founded on more assumpti

Occam's razor, as stated by GP is not about correctness, but tractability. In fact, if you look at it probabilistically, the hypothesis founded on more assumptions is more likely to be correct: Suppose you have a hypothesis, H, which is based on assumptions A1, A2, ..., Ak. This can be phrased logically as an implication: (A1 & A2 & ... & Ak) -> H Decomposing the implication, we get: !A1 | !A2 | ... | !Ak | H Then Pr(!A1 | !A2 | ... | !Ak | H) = 1 - Pr( A1 & A2 & ... & Ak & !H) So, appealing to the conjunction fallacy, assuming that we are adding more assumptions on top, rather than having a greater number of different assumptions, the probability of success actually goes up.

vog 10y ago
Sorry for the mathematical nitpick here, but that seems to me like a strawman. You silently moved from the original question: What is the probability that the hypothesis is correct? To the very different question: What is the probability that the implication "from the assumptions follows the hypothesis" is correct? Moreover, this different question has a clear answer for every logically consistent theory: It is 1, because it is always true! Why? Because that's exactly what the theory proves logically. The theory can't tell you whether A1, ..., Ak are all true in the real world, but it does tell you that _if_ these are true, H is also true. So this is really a typical strawman argument (although maybe unintendedly): It is different from the original question, and it boils down to a trivial but misleading answer. ------------------ Going back to the original question, you'd have to compare the two hypotheses H1 and H2, where the set of assumptions of H1 are a strict subset of the assumptions of H2: A1 & A2 & ... & Ak -> H1 A1 & A2 & ... & Ak & ... & An -> H2 It is clear that: P(A1 & A2 & ... & Ak) > P(A1 & A2 & ... & Ak & ... & An) But from here it is surprisingly hard to conclude "P(H1) > P(H2)", because we have implications and not equivalences. That is, H1 may be true even though the assumptions don't hold. It may be true for different reasons and derived from a different set of assumptions that turn out to be true. Same for H2. So we need to take into account the probabilities for H1 and H2 to be "true for different reasons", which we'll name Pd1 and Pd2: Pd1 = P(not(A1 & A2 & ... & Ak) & H1) Pd2 = P(not(A1 & A2 & ... & Ak & ... & An) & H2) To prove the probability variant of Occam's razor, we need to make the following additional meta-assumption: The probabilities that H1 and H2 are "true for different reasons" are very small, and moreover almost identical. So we have: Pd1 = Pd2 But with that meta-assumption, we can finally prove the probability variant of Ocamm's razor, as we can now express P(H1) and P(H2): P(H1) = P(not(A1 & A2 & ... & Ak) & H1) + P((A1 & A2 & ... & Ak) & H1) = Pd1 + P((A1 & A2 & ... & Ak) & H1) = Pd1 + P(A1 & A2 & ... & Ak) = Pd2 + P(A1 & A2 & ... & Ak) > Pd2 + P(A1 & A2 & ... & Ak & ... & An) = Pd2 + P((A1 & A2 & ... & Ak & ... & An) & H2) = P(not(A1 & A2 & ... & Ak & ... & An) & H2) + P((A1 & A2 & ... & Ak & ... & An) & H2) = P(H2) In short: P(H1) > P(H2)
- asQuirreL 10y ago
  You are right, they are different questions, but the straw man was not intentional, I thought the original phrasing was ambiguous enough that it could be interpreted in both ways ;) In other words, it was unclear to me what the answer to the question "Are the assumptions part of the hypothesis?" was. If, as I did, we assume that "yes, they are" then I don't think it follows that the probabilities will both be `1`, because we do not have logical proofs for the claims, the implication could only be true in the model (they are not necessarily entailments). The waters are muddied further still when the hypothesis itself is phrased as an implication. EDIT It also strikes me that for your line of reasoning to hold, it is not sufficient that Pd1 = Pd2 are small, but instead `Pd1 = 0 = Pd2`, in order to justify this line: > = Pd1 + P((A1 & A2 & ... & Ak) & H1) > = Pd1 + P(A1 & A2 & ... & Ak) Which is tantamount to saying (A1 & A2 & ... & Ak) <-> H1 & (A1 & A2 & ... & Ak & ... & An) <-> H2 Is it not? EDIT (2) Ignore that, it is not tantamount, it is a weaker condition.
  - vog 10y ago
    Maybe the final part of the proof I gave is a bit dense, so here are some additional notes. First of all, if you know that (A1 & A2 & ... & Ak) -> H1 then the following two terms are logically equivalent: A1 & A2 & ... & Ak (A1 & A2 & ... & Ak) & H1 Also, for the proof which I gave it is sufficient that Pd1 = Pd2. It does not need them to be zero.
    - asQuirreL 10y ago
      Ah yes, I see. I guess I was looking for a place where the fact that `Pd1` and `Pd2` were small, but I guess that's not necessary.
      - vog 10y ago
        No, the "both are small" was just meant to be a justification for assuming Pd1=Pd2.
philh 10y ago
> (A1 & A2 & ... & Ak) -> H This is backwards. It should be H => (A1 & A2 & ... & Ak) It's not "if these assumptions hold, the hypothesis is true". It's "for this hypothesis to be true, these assumptions must hold". Suppose you have the hypothesis that Bruce Wayne is Superman. Then you see the two of them in the same room together. It's still possible that Bruce Wayne is Superman, but only if he has an identical twin. Your credence that Bruce Wayne is Superman should decrease accordingly.
- vog 10y ago
  Umm .. no. Assumptions are the left-hand side of an implication, by definition. (And the right-hand side is called "conclusion".) The relevant statement here is not "for this hypothesis to be true, these assumptions must hold". It is: "for this hypothesis to be derived this way, these assumptions must hold". There is always the possibility that a hypothesis can be proved in a different way from different assumptions. Unless, of course, your theory not only proves "(A1 & A2 & ... & Ak) -> H" but "(A1 & A2 & ... & Ak) <-> H". That is, if your theory shows that your hypothesis does not only follow from the assumptions, but is equivalent to its assumptions. That's quite a rare case, though.
  - philh 10y ago
    If you see Bruce Wayne and Superman in the same room, then "Bruce Wayne is Superman" can only be true if you assume something you didn't have to assume before. It means you should be less confident that Bruce Wayne is Superman. I'm using the word "assumption" in a natural way. (Also in the way that it's used in Occam's razor.) If you have a definition that says I'm using it wrong, then your definition is silly.
    - vog 10y ago
      This example is totally unclear to be. Although you declared a clear hypothesis in your very first comment, it is totally unclear what exactly your assumptions are that would lead to this hypothesis.

vog 10y ago

Sorry for the mathematical nitpick here, but that seems to me like a strawman. You silently moved from the original question: What is the probability that the hypothesis is correct? To the very different question: What is the probability that the implication "from the assumptions follows the hypothesis" is correct? Moreover, this different question has a clear answer for every logically consistent theory: It is 1, because it is always true! Why? Because that's exactly what the theory proves logically. The theory can't tell you whether A1, ..., Ak are all true in the real world, but it does tell you that _if_ these are true, H is also true. So this is really a typical strawman argument (although maybe unintendedly): It is different from the original question, and it boils down to a trivial but misleading answer. ------------------ Going back to the original question, you'd have to compare the two hypotheses H1 and H2, where the set of assumptions of H1 are a strict subset of the assumptions of H2: A1 & A2 & ... & Ak -> H1 A1 & A2 & ... & Ak & ... & An -> H2 It is clear that: P(A1 & A2 & ... & Ak) > P(A1 & A2 & ... & Ak & ... & An) But from here it is surprisingly hard to conclude "P(H1) > P(H2)", because we have implications and not equivalences. That is, H1 may be true even though the assumptions don't hold. It may be true for different reasons and derived from a different set of assumptions that turn out to be true. Same for H2. So we need to take into account the probabilities for H1 and H2 to be "true for different reasons", which we'll name Pd1 and Pd2: Pd1 = P(not(A1 & A2 & ... & Ak) & H1) Pd2 = P(not(A1 & A2 & ... & Ak & ... & An) & H2) To prove the probability variant of Occam's razor, we need to make the following additional meta-assumption: The probabilities that H1 and H2 are "true for different reasons" are very small, and moreover almost identical. So we have: Pd1 = Pd2 But with that meta-assumption, we can finally prove the probability variant of Ocamm's razor, as we can now express P(H1) and P(H2): P(H1) = P(not(A1 & A2 & ... & Ak) & H1) + P((A1 & A2 & ... & Ak) & H1) = Pd1 + P((A1 & A2 & ... & Ak) & H1) = Pd1 + P(A1 & A2 & ... & Ak) = Pd2 + P(A1 & A2 & ... & Ak) > Pd2 + P(A1 & A2 & ... & Ak & ... & An) = Pd2 + P((A1 & A2 & ... & Ak & ... & An) & H2) = P(not(A1 & A2 & ... & Ak & ... & An) & H2) + P((A1 & A2 & ... & Ak & ... & An) & H2) = P(H2) In short: P(H1) > P(H2)

asQuirreL 10y ago
You are right, they are different questions, but the straw man was not intentional, I thought the original phrasing was ambiguous enough that it could be interpreted in both ways ;) In other words, it was unclear to me what the answer to the question "Are the assumptions part of the hypothesis?" was. If, as I did, we assume that "yes, they are" then I don't think it follows that the probabilities will both be `1`, because we do not have logical proofs for the claims, the implication could only be true in the model (they are not necessarily entailments). The waters are muddied further still when the hypothesis itself is phrased as an implication. EDIT It also strikes me that for your line of reasoning to hold, it is not sufficient that Pd1 = Pd2 are small, but instead `Pd1 = 0 = Pd2`, in order to justify this line: > = Pd1 + P((A1 & A2 & ... & Ak) & H1) > = Pd1 + P(A1 & A2 & ... & Ak) Which is tantamount to saying (A1 & A2 & ... & Ak) <-> H1 & (A1 & A2 & ... & Ak & ... & An) <-> H2 Is it not? EDIT (2) Ignore that, it is not tantamount, it is a weaker condition.
- vog 10y ago
  Maybe the final part of the proof I gave is a bit dense, so here are some additional notes. First of all, if you know that (A1 & A2 & ... & Ak) -> H1 then the following two terms are logically equivalent: A1 & A2 & ... & Ak (A1 & A2 & ... & Ak) & H1 Also, for the proof which I gave it is sufficient that Pd1 = Pd2. It does not need them to be zero.
  - asQuirreL 10y ago
    Ah yes, I see. I guess I was looking for a place where the fact that `Pd1` and `Pd2` were small, but I guess that's not necessary.
    - vog 10y ago
      No, the "both are small" was just meant to be a justification for assuming Pd1=Pd2.

philh 10y ago

> (A1 & A2 & ... & Ak) -> H This is backwards. It should be H => (A1 & A2 & ... & Ak) It's not "if these assumptions hold, the hypothesis is true". It's "for this hypothesis to be true, these assumptions must hold". Suppose you have the hypothesis that Bruce Wayne is Superman. Then you see the two of them in the same room together. It's still possible that Bruce Wayne is Superman, but only if he has an identical twin. Your credence that Bruce Wayne is Superman should decrease accordingly.

vog 10y ago
Umm .. no. Assumptions are the left-hand side of an implication, by definition. (And the right-hand side is called "conclusion".) The relevant statement here is not "for this hypothesis to be true, these assumptions must hold". It is: "for this hypothesis to be derived this way, these assumptions must hold". There is always the possibility that a hypothesis can be proved in a different way from different assumptions. Unless, of course, your theory not only proves "(A1 & A2 & ... & Ak) -> H" but "(A1 & A2 & ... & Ak) <-> H". That is, if your theory shows that your hypothesis does not only follow from the assumptions, but is equivalent to its assumptions. That's quite a rare case, though.
- philh 10y ago
  If you see Bruce Wayne and Superman in the same room, then "Bruce Wayne is Superman" can only be true if you assume something you didn't have to assume before. It means you should be less confident that Bruce Wayne is Superman. I'm using the word "assumption" in a natural way. (Also in the way that it's used in Occam's razor.) If you have a definition that says I'm using it wrong, then your definition is silly.
  - vog 10y ago
    This example is totally unclear to be. Although you declared a clear hypothesis in your very first comment, it is totally unclear what exactly your assumptions are that would lead to this hypothesis.