4 ms·

>> You have no choice but to accept that the major conclusions of these studies are true. > I am surprised at the blind spot I had when first reading it – Kahn

>> You have no choice but to accept that the major conclusions of these studies are true. > I am surprised at the blind spot I had when first reading it – Kahneman’s overconfidence didn’t register with me. Wow, this hits me pretty hard too. My impression of the book was of Kahneman being a subtle and humble thinker, in stark contrast to the bombast of Nassim Taleb of "The Black Swan". A great post, if only for this and drawing my attention to the fact that the "hot hand" effect may actually exist... I've definitely used it as an example of when people are liable to see patterns in the noise :-(

StavrosK 10y ago
Kahneman is half-right, though. You have to accept the conclusion of a study as true. You can't accept the study as true and then keep believing in the thing the study just disproved. What you can do, though, is question whether the study itself is true. Sometimes, it's not.
- jknoepfler 10y ago
  One can consistently accept a study as well-implemented/operationalized while acknowledging a reasonable chance that the results are garbage, and that repeating the study might not yield significant results.The proper conclusion of a statistically oriented study is a probability distribution across several potentially contradictory propositions. One is never forced to "exclude the middle" as a result of a study, and conclude either A or not A. Re-reading what you said I think we're in agreement already. I was nit-picking at the idea that a study has a single conclusion which is either true or not true. The conclusion of most studies is properly a vector and a hefty pinch of salt imo. edit: clarity
- freshhawk 10y ago
  Actually I would say that you are more likely to be correct if you wait until there are multiple replicated results before you "have to accept" it as true. One study is interesting evidence, depending on the protocol, but publication bias/filedrawer effect is a real thing and studies are massively biased towards positive results.
- coldtea 10y ago
  >You have to accept the conclusion of a study as true. You can't accept the study as true and then keep believing in the thing the study just disproved. People doing that weren't who Kahneman addressed there though. It was people thinking the study was BS/fishy/wrong.
verhaust 10y ago
I'd heard and read about the "hot hand" effect being a fallacy in a lot of places. It never made sense to me. I'm glad it is finally getting debunked. Anyone who has played sports for a length of time would likely say that they feel more in control of their body at certain times. It could be because the brain chemistry becomes temporarily balanced, reducing anxiety, so their brain doesn't distract them as much. It could be a lingering pain/headache suddenly disappearing so it no longer becomes a physical distraction. It could be something directly related to muscle control, like minor tremors going away giving you more direct control over your body. In the case of sports with "fine tuning" like basketball where if you feel you are shooting short, you make the change to put more strength into it to "hone in" on the basket and start hitting shots. So, there are definitely times where you not only feel more capable of doing what you need to do over a short period of time. Due to current circumstances, you actually are more capable.
- forgetsusername 10y ago
  >Anyone who has played sports for a length of time would likely say I played sports, one particularly to a highly competitive level, and I'm not sure I agree with feeling a conscious sense of more control of my body. Sure, some days feel better than other. But there are just far too many variables, so you can always point to something as the "cause" of your sudden string of what I'd consider good luck, but might also look like ability. In fact, I'd say this is the major fallacy of sports analysis: post-hoc reasoning of results. Caveat: I haven't read the "hot hand" research.
- ScottBurson 10y ago
  Yes, whatever the reason, we all have good days and bad days. What you're talking about is what the statisticians call nonstationarity: on one day you make 30% of your shots, while on another day you make 40%, or whatever. Then your teammates, estimating your probability of making the next shot based on how they've observed you playing that day, decide whether to give you the ball more or less often. That would be one explanation -- and to my mind, a perfectly reasonable one -- of the "hot hand" theory. But that's not what the authors of the original paper measured; and interestingly enough, it's also not the theory that Collins has now un-debunked. That involves a different measure, called autocorrelation, which measures "streakiness": how the odds of your making the next shot change based on whether you made the previous one. Autocorrelation and nonstationarity are orthogonal -- you can have either one without the other.
superswordfish 10y ago
The arrogance is deep: The book repeats "this is true...you have no choice...this is you" and such, but that isn't true at all! The studies, assuming they're trustworthy, prove things about people in general, and the conclusions are useful if you're a public speaker or work in marketing. But what's dangerous is the suggestion that these things are true in the small, true for the individual. That's why the conclusions are less interesting, less useful than the author suggests. But it's written like typical popular science, with each chapter ending with pithy application of the knowledge, usually presented in the small (usage examples such as "I did X because of the availibility heuristic"). I checked out a copy based on Alan Kay's recommendation and returned it after getting about half-way through, realizing it was typical pop-science sophistry.

StavrosK 10y ago

Kahneman is half-right, though. You have to accept the conclusion of a study as true. You can't accept the study as true and then keep believing in the thing the study just disproved. What you can do, though, is question whether the study itself is true. Sometimes, it's not.

jknoepfler 10y ago
One can consistently accept a study as well-implemented/operationalized while acknowledging a reasonable chance that the results are garbage, and that repeating the study might not yield significant results.The proper conclusion of a statistically oriented study is a probability distribution across several potentially contradictory propositions. One is never forced to "exclude the middle" as a result of a study, and conclude either A or not A. Re-reading what you said I think we're in agreement already. I was nit-picking at the idea that a study has a single conclusion which is either true or not true. The conclusion of most studies is properly a vector and a hefty pinch of salt imo. edit: clarity
freshhawk 10y ago
Actually I would say that you are more likely to be correct if you wait until there are multiple replicated results before you "have to accept" it as true. One study is interesting evidence, depending on the protocol, but publication bias/filedrawer effect is a real thing and studies are massively biased towards positive results.
coldtea 10y ago
>You have to accept the conclusion of a study as true. You can't accept the study as true and then keep believing in the thing the study just disproved. People doing that weren't who Kahneman addressed there though. It was people thinking the study was BS/fishy/wrong.

verhaust 10y ago

I'd heard and read about the "hot hand" effect being a fallacy in a lot of places. It never made sense to me. I'm glad it is finally getting debunked. Anyone who has played sports for a length of time would likely say that they feel more in control of their body at certain times. It could be because the brain chemistry becomes temporarily balanced, reducing anxiety, so their brain doesn't distract them as much. It could be a lingering pain/headache suddenly disappearing so it no longer becomes a physical distraction. It could be something directly related to muscle control, like minor tremors going away giving you more direct control over your body. In the case of sports with "fine tuning" like basketball where if you feel you are shooting short, you make the change to put more strength into it to "hone in" on the basket and start hitting shots. So, there are definitely times where you not only feel more capable of doing what you need to do over a short period of time. Due to current circumstances, you actually are more capable.

forgetsusername 10y ago
>Anyone who has played sports for a length of time would likely say I played sports, one particularly to a highly competitive level, and I'm not sure I agree with feeling a conscious sense of more control of my body. Sure, some days feel better than other. But there are just far too many variables, so you can always point to something as the "cause" of your sudden string of what I'd consider good luck, but might also look like ability. In fact, I'd say this is the major fallacy of sports analysis: post-hoc reasoning of results. Caveat: I haven't read the "hot hand" research.
ScottBurson 10y ago
Yes, whatever the reason, we all have good days and bad days. What you're talking about is what the statisticians call nonstationarity: on one day you make 30% of your shots, while on another day you make 40%, or whatever. Then your teammates, estimating your probability of making the next shot based on how they've observed you playing that day, decide whether to give you the ball more or less often. That would be one explanation -- and to my mind, a perfectly reasonable one -- of the "hot hand" theory. But that's not what the authors of the original paper measured; and interestingly enough, it's also not the theory that Collins has now un-debunked. That involves a different measure, called autocorrelation, which measures "streakiness": how the odds of your making the next shot change based on whether you made the previous one. Autocorrelation and nonstationarity are orthogonal -- you can have either one without the other.

superswordfish 10y ago

The arrogance is deep: The book repeats "this is true...you have no choice...this is you" and such, but that isn't true at all! The studies, assuming they're trustworthy, prove things about people in general, and the conclusions are useful if you're a public speaker or work in marketing. But what's dangerous is the suggestion that these things are true in the small, true for the individual. That's why the conclusions are less interesting, less useful than the author suggests. But it's written like typical popular science, with each chapter ending with pithy application of the knowledge, usually presented in the small (usage examples such as "I did X because of the availibility heuristic"). I checked out a copy based on Alan Kay's recommendation and returned it after getting about half-way through, realizing it was typical pop-science sophistry.