3 ms·

Thanks for the points and taking the time to write up so much! I am enjoying this interaction, even if we don't fully agree :} > The difference in irreducible

by narush 4y ago

Thanks for the points and taking the time to write up so much! I am enjoying this interaction, even if we don't fully agree :} > The difference in irreducible uncertainty between "shuffled cards" and "anything else" is a matter of degree, not kind. Difference in degree or difference in kind is not the important point to sort out. The important question is what computation methods actually work in practice for humans. Some algorithms that work well on degree-small datasets fail to terminate in the course of the universe for large-datasets. If you've ever operated in a particular uncertain environment (like, as a first time startup founder), it immediately becomes clear how totally useless explicit probability calculations are when making decisions. I was a self-proclaimed rationalist and tried reasonable hard to be a good Baysian; I ended up deciding I might as well sacrifice a goat and read it's entrails. > The point isn't to follow the numbers off a cliff, but if you're well-calibrated (as in, have a track record with a decent brier score, or something), then pretending those numbers are totally useless is a bit silly. I'm actually trying to make a stronger point that these numbers are "totally useless." I think in practice many applications of explicit probability calculation are actually quite harmful. Rationalists love to talk about map and territory as the two items to be concerned about, but there's also another thing called "agent's belief in the effectiveness of their map." What models/explicit math/probability gets you in improved mapping, it takes away from you by making one's belief in the effectiveness of their map much much more. If you look at large-scale model failure in practice, it's almost always because someone thought the math they were doing captured the full system in a complete way (and then it didn't). 2008 is a fantastic example of this. > Well, you'd be able to do that in a world where we cared about keeping track of that sort of thing. Too bad people keep finding reasons not to do that, huh? I'd love to start tracking predictions generally! I agree with you here! > Surely you aren't telling me that you've updated your priors on observed evidence, and as a result have a different expectation about future world states (that is, "how useful would explicit Bayesian reasoning be if I tried using it")? To be clear: it's the explicit probability calculation and mathification that I take my major issue with. I am most definitely not against learning from what I observe :-) > Their level of performance is not compatible with the "the small number of people out millions for whom the the coin flip came up heads 10x in a row" hypothesis Can you link this math? I'd love to see it - not flippant, genuinely looking to check it out and be educated here!

comp_throw7 4y ago
Yeah, to clarify, my original objections were generated by Deutsch's dismissals of Bayesian epistemology on grounds of fundamental invalidity, rather than practicality. I agree that practicality is often a serious concern! I've also attempted a startup and not once did I ever run an explicit Bayesian calculation. Naive applications of many powerful techniques often turn out to be actively harmful, unfortunately. But without explicit probability calculation and mathification, we can't actually track our predictions (often of hugely uncertain events) over time, and if you can't track your predictions over time you'll have a hard time improving. https://goodjudgment.com/resources/the-superforecasters-track-record/ https://goodjudgment.com/resources/the-superforecasters-trac... I don't know if anyone's written up the math, but it should be pretty intuitive to get a sense of the differential. Consider that you need 20 bits of information to distinguish one thing from approximately a million. (Isomorphic: the odds of a coin coming up heads 20 times in a row is 1/2^20.) Now, of course, superforecasters aren't perfect predictors, but they're also predicting things much harder than coin flips - there's no such thing as "playing it safe" with brier scores. Matching or outperforming domain experts across a wide variety of domains, over a long period of time, involving many questions (hundreds if not thousands) is not really the sort of thing that happens by coincidence.
- narush 4y ago
  > fundamental invalidity, rather than practicality I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions. I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions. My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools. To paint the shape of what I think these decision making tools should look like: 1. They avoid mechanistic explanations. 2. They are explicitly not mathy. 3. They apply simple and robust heuristics to generate lots of options. 4. They exist in a purposely constructed iterative environment so that you get lots of attempts. These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable! I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of). Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!

by narush 4y ago

comp_throw7 4y ago

Yeah, to clarify, my original objections were generated by Deutsch's dismissals of Bayesian epistemology on grounds of fundamental invalidity, rather than practicality. I agree that practicality is often a serious concern! I've also attempted a startup and not once did I ever run an explicit Bayesian calculation. Naive applications of many powerful techniques often turn out to be actively harmful, unfortunately. But without explicit probability calculation and mathification, we can't actually track our predictions (often of hugely uncertain events) over time, and if you can't track your predictions over time you'll have a hard time improving. https://goodjudgment.com/resources/the-superforecasters-track-record/ https://goodjudgment.com/resources/the-superforecasters-trac... I don't know if anyone's written up the math, but it should be pretty intuitive to get a sense of the differential. Consider that you need 20 bits of information to distinguish one thing from approximately a million. (Isomorphic: the odds of a coin coming up heads 20 times in a row is 1/2^20.) Now, of course, superforecasters aren't perfect predictors, but they're also predicting things much harder than coin flips - there's no such thing as "playing it safe" with brier scores. Matching or outperforming domain experts across a wide variety of domains, over a long period of time, involving many questions (hundreds if not thousands) is not really the sort of thing that happens by coincidence.

narush 4y ago
> fundamental invalidity, rather than practicality I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions. I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions. My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools. To paint the shape of what I think these decision making tools should look like: 1. They avoid mechanistic explanations. 2. They are explicitly not mathy. 3. They apply simple and robust heuristics to generate lots of options. 4. They exist in a purposely constructed iterative environment so that you get lots of attempts. These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable! I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of). Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!