4 ms·

@jerf Why did you assume I did not read the article? I read it, and I find the writing to be unclear. Maybe "you'd expect" means what you think; maybe not. Ind

by dj-wonk 12y ago

@jerf Why did you assume I did not read the article? I read it, and I find the writing to be unclear. Maybe "you'd expect" means what you think; maybe not. Independent of my or your opinion, (or precisely because reasonable people may disagree over something so simple) the writing is unnecessarily unclear. It would be simple to add a quick note saying, more-or-less, that the author will revisit the point later. I also disagree with another commenter who says that the later writing clears up this issue. The point about 33% not being a fair assumption isn't addressed head on in the way that I think matters most. Here is my main point, which was not received in the spirit I intended. Many smart people, even programmers, sometimes incorrectly assume a discrete uniform distribution (e.g. 1/3, 1/3, 1/3 in the above example), probably because they think it makes the fewest assumptions. I'll put it another way with a related example. Alexa gives Bart a two-sided coin, tells him it may or not be fair, and asks the expected probability of getting heads. Bart reasons as follows: "I know that the coin may be unfair, ranging from, say, 0%/100% to 50%/50% to 100%/0%. By symmetry, for each 1%/99% coin there is a 99%/1% coin. So, in the end, the coin will average out to 50/50, so the best answer is 50%." Chuck comes along and says, "Bart got it wrong. Any point estimate must be incorrect, because we don't have enough information. The correct answer must be a distribution. Since we know nothing, the best answer is a flat (uniform) distribution ranging from 0% to 100%." Danielle chimes in and adds "Yes, Bart was wrong, and Chuck improved on the analysis, but Chuck did not go far enough. Both Bart and Chuck assumed symmetry, without justification. Chuck's act of assuming a uniform distribution is tantamount to knowing the process that generated the coin. Does Chuck really know the chances of a 0-10% coin are the same as a 10-20% coin, and so on? No, he does not. Therefore, even answering the question with a distribution is incorrect. The only correct answer is that the probability of heads ranges from 0 to 100%, but no distribution can be given." Danielle is correct. The author may well know this, but it is not communicated clearly in the article. I would be willing to wager that many people, when confronted with Alex's question, would answer like Bob or Chuck did.

vidarh 12y ago
> I also disagree with another commenter who says that the later writing clears up this issue. The point about 33% not being a fair assumption isn't addressed head on in the way that I think matters most. How would you want it addressed? A large part of the article is exactly about how the - to some - "intuitive" idea of 3 evenly split categories is incorrect. E.g. later in the article he writes: "It turns out, we weren’t supposed to be reasoning ‘there are 3 categories of possible relationships, so we start with 33%’, but rather: ‘there is only one explanation “A causes B”, only one explanation “B causes A”, but there are many explanations of the form “C1 causes A and B”, “C2 causes A and B”, “C3 causes A and B”…’, and the more nodes in a field’s true causal networks (psychology or biology vs physics, say), the bigger this last category will be." Presumably this is why two people separately assumed you did not read the whole article.
- dj-wonk 12y ago
  Ok, to move this forward, let's move past the who has read what part, which doesn't seem to be helping. Anyhow, my point is relatively subtle, I think, and getting lost in communication somehow. I get what the article is saying, but I'm not sure that everyone here is getting what I'm saying. Let me try to elaborate. My points are: 1. The article does indeed criticize the 1/3, 1/3, 1/3 split, but not for the same reasons that I am. 2. I would prefer that the 1/3, 1/3, 1/3 split not be used as a prior at all. (Yes, I get that Bayesian priors don't matter that much if you have enough data to update them. The article, on my reading, does not offer a way to get the needed observations, so I think the priors dominate.) 3. Yes, later, the article dismisses the even split as naive but does not go far enough when suggesting a better way. You see, the "better way" still boils down to the naive assumption of counting bins: this is to say that if we have more bins, then we should have a higher expected probability. In my opinion, the "better way" is still falling into a conceptual trap: this is not a combinatorics problem. For combinatorics problems to work they have to rely on real, countable things, not arbitrary notions of how you slice and dice a problem. Yes, I get that as you add more variables to the DAG, you can certainly say that there are more possible paths through the graph. But we cannot assume that just adding more nodes in the DAG will change reality. You see, the DAG is a mental construct. We should not assume that adding more mental bins (an arbitrary way of slicing reality) will affect the distribution. Here is an example. Francis tells Greg there are three marbles in a jar (green, red, blue) but does not reveal the distribution, so Greg cannot rightly assume a 1/3, 1/3, 1/3 distribution. But let's say Greg does anyway. Next, Francis tells Greg, "Sorry, I was oversimplifying before; really there are 4 types of marbles in a jar (forest green, lime green, red, blue)." What should Greg do? Update the prior to 1/4, 1/4, 1/4, 1/4? Well, that would be inconsistent, since 1/4 (forest green) + 1/4 (lime green) does not equal 1/3 (green). Should Greg suggest a 1/6 (forest), 1/6 (lime), 1/3 (red), 1/3 (blue) split? Nope. My point is this: down that road lies arbitrariness and contradictions. Hopefully this helps explain what I'm talking about. I like the article, but I don't want people to get into the habit of assuming a distribution. I also don't like people counting up concepts that are arbitrary and turning that into a distribution.

by dj-wonk 12y ago

vidarh 12y ago

> I also disagree with another commenter who says that the later writing clears up this issue. The point about 33% not being a fair assumption isn't addressed head on in the way that I think matters most. How would you want it addressed? A large part of the article is exactly about how the - to some - "intuitive" idea of 3 evenly split categories is incorrect. E.g. later in the article he writes: "It turns out, we weren’t supposed to be reasoning ‘there are 3 categories of possible relationships, so we start with 33%’, but rather: ‘there is only one explanation “A causes B”, only one explanation “B causes A”, but there are many explanations of the form “C1 causes A and B”, “C2 causes A and B”, “C3 causes A and B”…’, and the more nodes in a field’s true causal networks (psychology or biology vs physics, say), the bigger this last category will be." Presumably this is why two people separately assumed you did not read the whole article.

dj-wonk 12y ago
Ok, to move this forward, let's move past the who has read what part, which doesn't seem to be helping. Anyhow, my point is relatively subtle, I think, and getting lost in communication somehow. I get what the article is saying, but I'm not sure that everyone here is getting what I'm saying. Let me try to elaborate. My points are: 1. The article does indeed criticize the 1/3, 1/3, 1/3 split, but not for the same reasons that I am. 2. I would prefer that the 1/3, 1/3, 1/3 split not be used as a prior at all. (Yes, I get that Bayesian priors don't matter that much if you have enough data to update them. The article, on my reading, does not offer a way to get the needed observations, so I think the priors dominate.) 3. Yes, later, the article dismisses the even split as naive but does not go far enough when suggesting a better way. You see, the "better way" still boils down to the naive assumption of counting bins: this is to say that if we have more bins, then we should have a higher expected probability. In my opinion, the "better way" is still falling into a conceptual trap: this is not a combinatorics problem. For combinatorics problems to work they have to rely on real, countable things, not arbitrary notions of how you slice and dice a problem. Yes, I get that as you add more variables to the DAG, you can certainly say that there are more possible paths through the graph. But we cannot assume that just adding more nodes in the DAG will change reality. You see, the DAG is a mental construct. We should not assume that adding more mental bins (an arbitrary way of slicing reality) will affect the distribution. Here is an example. Francis tells Greg there are three marbles in a jar (green, red, blue) but does not reveal the distribution, so Greg cannot rightly assume a 1/3, 1/3, 1/3 distribution. But let's say Greg does anyway. Next, Francis tells Greg, "Sorry, I was oversimplifying before; really there are 4 types of marbles in a jar (forest green, lime green, red, blue)." What should Greg do? Update the prior to 1/4, 1/4, 1/4, 1/4? Well, that would be inconsistent, since 1/4 (forest green) + 1/4 (lime green) does not equal 1/3 (green). Should Greg suggest a 1/6 (forest), 1/6 (lime), 1/3 (red), 1/3 (blue) split? Nope. My point is this: down that road lies arbitrariness and contradictions. Hopefully this helps explain what I'm talking about. I like the article, but I don't want people to get into the habit of assuming a distribution. I also don't like people counting up concepts that are arbitrary and turning that into a distribution.