3 ms·

Yeah, to clarify, my original objections were generated by Deutsch's dismissals of Bayesian epistemology on grounds of fundamental invalidity, rather than pract

Yeah, to clarify, my original objections were generated by Deutsch's dismissals of Bayesian epistemology on grounds of fundamental invalidity, rather than practicality. I agree that practicality is often a serious concern! I've also attempted a startup and not once did I ever run an explicit Bayesian calculation. Naive applications of many powerful techniques often turn out to be actively harmful, unfortunately. But without explicit probability calculation and mathification, we can't actually track our predictions (often of hugely uncertain events) over time, and if you can't track your predictions over time you'll have a hard time improving. https://goodjudgment.com/resources/the-superforecasters-track-record/ https://goodjudgment.com/resources/the-superforecasters-trac... I don't know if anyone's written up the math, but it should be pretty intuitive to get a sense of the differential. Consider that you need 20 bits of information to distinguish one thing from approximately a million. (Isomorphic: the odds of a coin coming up heads 20 times in a row is 1/2^20.) Now, of course, superforecasters aren't perfect predictors, but they're also predicting things much harder than coin flips - there's no such thing as "playing it safe" with brier scores. Matching or outperforming domain experts across a wide variety of domains, over a long period of time, involving many questions (hundreds if not thousands) is not really the sort of thing that happens by coincidence.

narush 4y ago
> fundamental invalidity, rather than practicality I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions. I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions. My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools. To paint the shape of what I think these decision making tools should look like: 1. They avoid mechanistic explanations. 2. They are explicitly not mathy. 3. They apply simple and robust heuristics to generate lots of options. 4. They exist in a purposely constructed iterative environment so that you get lots of attempts. These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable! I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of). Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!

narush 4y ago

> fundamental invalidity, rather than practicality I'm not sure there's any difference between these in practice - same with the difference being degree or kind. At the end of the day, the question is: what can I actually implement in my own brain to make good decisions. I think the biggest place we might disagree here is about _what the goals_ are of prediction in the first place. "if you can't track your predictions over time you'll have a hard time improving" might be true, but for me, the terminal goal is _not_ to make increasingly accurate predictions. My goal is to operate as effectively as possible. We both agree that explicit Bayesian calculations aren't useful in a startup world. We also both agree that Bayesian techniques are very easy to footgun with in the wild. For me, both of these point pretty squarely in the direction called: we need non-Bayesian decision making tools. To paint the shape of what I think these decision making tools should look like: 1. They avoid mechanistic explanations. 2. They are explicitly not mathy. 3. They apply simple and robust heuristics to generate lots of options. 4. They exist in a purposely constructed iterative environment so that you get lots of attempts. These decision making tools are built around a) acknowledging that we cannot mathematically reason about the uncertainty we're dealing with in any legit capacity (without just foot-gunning), and that b) spending time on _decisions_ vs. on _execution_ is silly in most contexts, as execution is where you actually learn things (and thus you should make lots of quick and dirty decisions), and c) more good options are always valuable! I can talk more about what this looks like in practice, as it's sort of the shape of how my cofounders and I run our startup. It's a WIP of course, but in practice we find the iterative, quick and dirty heuristic approach to lead to much faster and more robust growth than long-term predictions (which we used to do a lot of). Also, do you have links to work that has informed your thinking on superforcasters? Or links to the specific set of superforecasters you're talking about? It seems you're thinking about a specific set of people, and I'd love to learn more about em!