3 ms·

Certainly an interesting read. Although, it really makes me think of the words of the Big Lebowski: "That's just like, your opinion, man" which I think aligns

by tims33 5y ago

Certainly an interesting read. Although, it really makes me think of the words of the Big Lebowski: "That's just like, your opinion, man" which I think aligns with his ultimate conclusion.

didibus 5y ago
Ya, I think too often we have a tendency to take a bet on propositions where there simply is no logical conclusion to be had quite yet given the current information. I wish more author instead of saying, this is where my bet would be, they would just say, ya there's no good information and this whole thing is up in the air, we just don't know and here are the next steps if we wanted to get closer to knowing. Because otherwise it leads to a weird bias. For some reason, we have the tendency to believe that someone who's done the research would be more likely to be correct in their bet, but inconclusive research doesn't actually make this true. If the research is inconclusive, having done it doesn't help the prediction.
- 2arrs2ells 5y ago
  I think that’s unfair to Scott when the entire article is prefaced by “Epistemic status: Extremely confused! Low confidence in all of this” The rationalist movement is all about being able to accurately forecast the accuracy of your predictions. Saying we shouldn’t discuss low accuracy predictions at all seems odd - by making early/fuzzy predictions (and acknowledging them as such) we may ex find that we are more accurate than we realize.
  - didibus 5y ago
    Ya, this article did an okay job, I was talking more generally, though I still think there is a few places where he's making bets in that article. > Saying we shouldn’t discuss low accuracy predictions at all seems odd I don't think we're even talking low accuracy here, low accuracy still implies some accuracy, what I'm seeing here is random or potentially even worse than random prediction. The data points to things that are way to broad and general to reason about the cause. I think knowing that is valuable, don't get me wrong, but my conclusion would be, don't bet on this, unless you like playing roulette.
- cornel_io 5y ago
  Scott Alexander is a very careful quantifier of his bets, he tracks them year over year and always goes back to evaluate them and make sure he's well calibrated. I take your point, but he is one of a very few people on this planet who would reliably say he just didn't know if that was a more accurate statement than what he said. The whole thing is not up in the air, there is limited but useful information available now, and he's trying to correctly integrate that.
  - didibus 5y ago
    Can he accurately predict the accuracy of his predictions? I think that's an important factor. That said, you're right, but I was not focused on him, but on the bias that can be produced in other people when reading things like this. Though to be honest, I even worry that I have this bias as well. Like because I researched something, yet pretty inconclusive, I feel somehow better equipped to bet on it, and yet I have no idea if at that point I'm doing better than random.
    - mousepilot 5y ago
      I think its ok, I'd start with hypotheses and when I can, I would test them, and in the absense of a spare planet and mammalian species as your test tube, the only thing I can think of as a proxy for the proverbial test tube would be what statistics we can find. I thought it was well worth the read and he left no uncertainty about the uncertainty of his test results.
    - cornel_io 5y ago
      He does quantify his degree of belief in his predictions, and he compares to what they would look like if he was perfectly calibrated, see https://slatestarcodex.com/2020/04/08/2019-predictions-calibration-results/ https://slatestarcodex.com/2020/04/08/2019-predictions-calib... Maybe not the most 100% rigorous analysis that one could do, but certainly well beyond how self-accountable most people are.

by tims33 5y ago

Certainly an interesting read. Although, it really makes me think of the words of the Big Lebowski: "That's just like, your opinion, man" which I think aligns with his ultimate conclusion.

didibus 5y ago

Ya, I think too often we have a tendency to take a bet on propositions where there simply is no logical conclusion to be had quite yet given the current information. I wish more author instead of saying, this is where my bet would be, they would just say, ya there's no good information and this whole thing is up in the air, we just don't know and here are the next steps if we wanted to get closer to knowing. Because otherwise it leads to a weird bias. For some reason, we have the tendency to believe that someone who's done the research would be more likely to be correct in their bet, but inconclusive research doesn't actually make this true. If the research is inconclusive, having done it doesn't help the prediction.

2arrs2ells 5y ago
I think that’s unfair to Scott when the entire article is prefaced by “Epistemic status: Extremely confused! Low confidence in all of this” The rationalist movement is all about being able to accurately forecast the accuracy of your predictions. Saying we shouldn’t discuss low accuracy predictions at all seems odd - by making early/fuzzy predictions (and acknowledging them as such) we may ex find that we are more accurate than we realize.
- didibus 5y ago
  Ya, this article did an okay job, I was talking more generally, though I still think there is a few places where he's making bets in that article. > Saying we shouldn’t discuss low accuracy predictions at all seems odd I don't think we're even talking low accuracy here, low accuracy still implies some accuracy, what I'm seeing here is random or potentially even worse than random prediction. The data points to things that are way to broad and general to reason about the cause. I think knowing that is valuable, don't get me wrong, but my conclusion would be, don't bet on this, unless you like playing roulette.
cornel_io 5y ago
Scott Alexander is a very careful quantifier of his bets, he tracks them year over year and always goes back to evaluate them and make sure he's well calibrated. I take your point, but he is one of a very few people on this planet who would reliably say he just didn't know if that was a more accurate statement than what he said. The whole thing is not up in the air, there is limited but useful information available now, and he's trying to correctly integrate that.
- didibus 5y ago
  Can he accurately predict the accuracy of his predictions? I think that's an important factor. That said, you're right, but I was not focused on him, but on the bias that can be produced in other people when reading things like this. Though to be honest, I even worry that I have this bias as well. Like because I researched something, yet pretty inconclusive, I feel somehow better equipped to bet on it, and yet I have no idea if at that point I'm doing better than random.
  - mousepilot 5y ago
    I think its ok, I'd start with hypotheses and when I can, I would test them, and in the absense of a spare planet and mammalian species as your test tube, the only thing I can think of as a proxy for the proverbial test tube would be what statistics we can find. I thought it was well worth the read and he left no uncertainty about the uncertainty of his test results.
  - cornel_io 5y ago
    He does quantify his degree of belief in his predictions, and he compares to what they would look like if he was perfectly calibrated, see https://slatestarcodex.com/2020/04/08/2019-predictions-calibration-results/ https://slatestarcodex.com/2020/04/08/2019-predictions-calib... Maybe not the most 100% rigorous analysis that one could do, but certainly well beyond how self-accountable most people are.