7 ms·

Gigerenzer’s simple rules of thumb often outperform complex models

183 points by haltingproblem 6y ago

ToJans 6y ago
This feels like Gigerenzer has read my mind. Two weeks back I blogged about a simple heuristic to model uncertainty in software estimates [1], and I received confirmation from several people that my proposed technique might be spot-on: estimate your happy path, figure out how many dimensions your uncertainty has, and multiply the happy path estimates by (sqrt PI ^ dimensions) [1] https://tojans.me/posts/software-estimates-done-right/ https://tojans.me/posts/software-estimates-done-right/
- ulucs 6y ago
  Okay, I'm stumped. Isn't the gaussian function a probability density function, which means it should have an area of 1 by definition? Are you taking it as f(x) = exp(-x^2)? To keep f(0) equal to 1?
  - em500 6y ago
    You're stumped because most of the "statistics primer" section in that post doesn't make any sense. The connection between the Gaussian density and the sqrt(pi) heuristic is mostly imaginary. The original heuristic (pi, sqrt(pi), pi^2) works pretty much the same with 3 instead of pi, so you can view the pi versions as numerology or charitably a nice mnemonic.
    - ToJans 6y ago
      Could you elaborate? Assume we convert all our happy path estimates to minutes. What I'm saying is that each "estimated minute" is more likely to have a gaussian distribution than a uniform standard distribution,because normal distribution is more likely to occur in nature. while I can understand that this is a controversial assumption, I'm not the first one to make it. Referring to numerology seems a bit odd? I'm really looking for a proper way here, I'm quite a rational person, so numerology is not really my cup of tea TBH.
      - ToJans 6y ago
        Addendum: this is what I was referring to: https://en.wikipedia.org/wiki/Gaussian_integral https://en.wikipedia.org/wiki/Gaussian_integral
      - em500 6y ago
        Your blog post has so many errors that I don't even know where to start. As another poster mentioned, areas under non-degenerate probability density functions are 1 by definition, whether they're uniform, Gaussian, standardized or not. What you described as a "standard uniform distribution" is really a degenerate distribution[1], meaning that you assume no uncertainty at all (stdev=0). There's nothing "uniform" about that, you might just as well start with a Gaussian with stdev=0. "converting from a standard uniform distribution to a Gaussian distribution" as you described does not make any sense at all. If you replace an initial assumption of a degenerate distribution with a Gaussian, as you seem to be doing, you replace a no-uncertainty (stdev=0) assumption with some uncertainty (so the uncertainty blow-up is infinite), but it doesn't affect point estimates such as the mean or median, unless you make separate assumptions about that. There is nothing in your story that leads to multiplying some initial time estimate by sqrt(pi). The only tenuous connection with sqrt(pi) in the whole story is that the Gaussian integral happens to be sqrt(pi). There are some deep mathematical reasons for that, which has to do with polar coordinates transformations. But it has nothing to do with adjusting uncertainties or best estimates. [1] https://en.wikipedia.org/wiki/Degenerate_distribution https://en.wikipedia.org/wiki/Degenerate_distribution
        ToJans 6y ago
        Thank you for your valuable feedback; it will take some time to process, and I will adjust the blog post as my insights grow (potentially discarding the whole idea, but for me it's a learning process.)
      - yorwba 6y ago
        > What I'm saying is that each "estimated minute" is more likely to have a gaussian distribution If that were your actual assumption, you should measure the variance of the difference between your estimate and the actual time taken, use that to determine a confidence interval (e.g. 95% of the time, the additional delay is less than x) and then add it to your estimate.
  - ToJans 6y ago
    My apologies, I'm by no means well versed in math, so I might use the wrong terminology. Here is a good example video of what I am alluding to: https://www.youtube.com/watch?v=9CgOthUUdw4 https://www.youtube.com/watch?v=9CgOthUUdw4 Maybe I should refine my blog post or include the video in the explanation?
    - sumtechguy 6y ago
      You are the second person I have come across that uses PI to get software estimates. "I am not sure why but it works, except when it doesnt then I know exactly why" What I found was most people over/under estimate things. They also tend to do it consistently at the same rate. Typically they have a scaling factor you can use. Around 3 seems to be the sweet spot for most people. You could just as easily use 3.1 or 3.2 and get a similar answer. I usually go for 3 because I can do that in my head without a calculator.
- ToJans 6y ago
  Update: looks like my assumptions might be completely wrong... Please ignore my initial post, and I will check if/when I can provide an update to the blog post, potentially discarding the sqrt PI idea.
- mjburgess 6y ago
  I think the only reason this works is that it's a power law. A power law gives you a semi-principled way of talking about different scales: atomic, table, house, world, galaxy... The rate of the power increase corresponds to our "magnification factor" Probably many sequences of powers works, and many bases: base^0.5, base^1, base^1.5, base^2, ... eg., 4^0.5, 4^1, 4^1.5, 4^2, 4^2.5 = [2, 4, 8, 16, 32] If the estimate is 3days, then the scope is: [3, 6, 12, 24, 48, 96] Now we have a "semi-principled" set of numbers to choose from, the estimator applies their domain knowledge to select the relevant scales: 3 to 24 say, based on "how big the project could be". ie., is it both possibly a small change in one area (to a table), and possibly "reworking the foundations of the house" -- if so, you have the small-scale starter number, and you can dial up using this sequence to whatever magnification suits. Probably with base pi, and sequence (0.5, 1, 2) we capture the three most relevant scales for most properly discrete software tasks. That's essentially conincidental, in that i'm sure a (2 to 4) base with several sequences would wokr.
- ToJans 6y ago
  **** I stand corrected and marked that section from the blog post as BS; please accept my apologies. ****
bujak300 6y ago
I can recommend the "Simple heuristics that make us smart" book by Gigerenzer, it's an insightful and entertainig read: for this book he and his co-authors put their money where their mouth is and invested on the stock market based on the most simple heuristics imaginable.
- wrp 6y ago
  Gigerenzer has published several books, but I think that is the best one, because later books are just repeating the ideas. Although I think his ideas are good, they have drawn a lot of criticism. There are several discussion articles in issue 23(5) of Behavioral and Brain Science.
- daemoncoder 6y ago
  TED Talk https://youtu.be/-Lg7G8TMe_A https://youtu.be/-Lg7G8TMe_A
  - cwillu 6y ago
    TEDx you mean. Which matters quite a bit, given that there are effectively no restrictions on who can give a TEDx talk :p Which isn't to say that it's useless, just that the "TED" gatekeeping role isn't actually happening here, instead the gatekeeper is "people who have heard of TEDx", which is very different and imo quite a bit less useful.
    - yboris 6y ago
      Even I had a TEDx talk ;) https://www.youtube.com/watch?v=DPe9-OM7vgU https://www.youtube.com/watch?v=DPe9-OM7vgU
sva_ 6y ago
(2017)
rramadass 6y ago
I find this entire field of study fascinating. The reason is simple; ever since the start of the Scientific Revolution everything has become quite complex and multivariate and we have gotten into the habit of trying to solve them all using mathematical models. This has the advantage that once proven, it can be applied "mindlessly" towards a desired result. With Heuristics so many factors have to come together in a individual for any given event that repeatability is not certain. Therein lies the difficulty.
cwillu 6y ago
Effectively, every "knob" your model has is a major point against the model, but because people don't really get that (it's another "Linda is a bank teller and is active in the feminist movement" instance) they tend to overcomplicate their models compared to the amount of validation they've performed on the model. An alternative to putting nearly all the weight on a single complicated model created ex nihilo is to have some reasonable "zero information" prior distribution over models (weighted by complexity) where, because rules-of-thumb are very simple models, those simple models dominate. In fact, the alternative offered is itself a simple rule-of-thumb approach to this, basically saying that the more complicated models can be almost entirely neglected, and so you can really just do an even split over rules-of-thumb and still outperform any model that naïvely tries to “know best” by accounting for everything up front. In short, it's really fucking hard to properly validate that a model applies to reality, and until you appreciate just how hard it is, you're much better off radically biased towards simplicity.
- phkahler 6y ago
  Optimized systems are fragile. Probably true of models. The most profound class I had in college was thermodynamics, where all the complexity of physical systems is lumped into a few parameters. It seems crude but those parameters include all the energy and can be used to make very useful calculations.
  - cwillu 6y ago
    Is this a fair restatement: “Models (and systems) that accurately reflect reality with a high fidelity due in part to their accurate assignment of values to a larger number of parameters are fragile”? I agree that that's true, but my point (and I think part of what the article is saying) is that most “optimized” systems/models are not in fact doing that. In which case, the point I'd take (if I read you correctly) is that even once you _have_ managed to accurately set the parameters (which is harder to do and validate than it looks), you still have a fragile model/system. Alternatively [edited to add]: Define a model as a system that is attempting to match another system in behaviour. And so now you have two problems: ① a complicated model is itself fragile (by analogy to the system it's modelling) ② a complicated system has more ways to diverge from the system it's trying to model Hmm, in which case it's kinda the same thing after all, except one is about making the model in the first place, and the other is maintaining it, maybe?
HPsquared 6y ago
Simple models have fewer sources of error, and the small number of parameters can be better validated against reality.
- rramadass 6y ago
  That certainly sounds plausible, however i feel that it is the applicability/non-applicability of parameters themselves that is the key. You need to choose the right parameters before even bothering about the model function and this is where prior knowledge and experience are important. The works of Gary Klein and Nassim Taleb are also relevant here.
  - levosmetalo 6y ago
    Nassim Taleb actually cites Prof. Gigerenzer in the topic of simple heuristics.
johndoe42377 6y ago
Oh, really. So applying probabilities to partially observable systems yields bullshit, and estimated probabilities based on observations of the past events does not predict anything for evolving systems? I got banned here for such assertions lmao. What else is new?
prionassembly 6y ago
A documentary I strongly recommend to everybody (it's on YouTube): Tao Russell's "Being in the world". It's ostensibly about the life and work of Hubert Dreyfus, the famous MIT philosopher that was an AI skeptic. But it interviews carpenters and jazz and flamenco musicians and cooks etc. and (way oversimplifying here) the takeaway is that the skill is not in you, it's in this moment of skillfull interaction with the world. Gigerenzer seems to come awfully close to this point, but keeps missing it. At the critical moment the genius baseball batter is one with his bat, with the wind, with the noises, with each signal coming from his musculature -- this is not a Cartesian subject metaphorically manipulating levers and keys to move a robot, this is a situation. It's a whole thing. This is why even in sports where pitch sizes are regulamented (like soccer) there's a substantial home game advantage even in the absence of crowds. That's where they train. This is why pure mathematicians obsess over that particular chalk. This is why Kenny G sounds so shitty, he's recording against a backing track in an aquarium in a studio, and why improvised jazz sounds so great.
- techer 6y ago
  "Standard pitch measurements. Not all pitches are the same size, though the preferred size for many professional teams' stadiums is 105 by 68 metres (115 yd × 74 yd) with an area of 7,140 square metres (76,900 sq ft; 1.76 acres; 0.714 ha)." The field dimensions are within the range found optimal by FIFA: 110–120 yards (100–110 m) long by 70–80 yards (64–75 m) wide. Managers can instruct groundskeepers to alter the pitch size at home. I heard that referees are not feeling pressured into giving or not giving penalties that they may have been pressured into due to home crowds. Some sports have fixed sizes. I’ll watch your recommendation. Thanks
haltingproblem 6y ago
This result is still not widely known but a simple equal weighed rule outperforms Markowitz mean variance for most alpha trading strategies i.e. strategies that predict outperformance. Peter Muller shows this in an original paper in 1993 [1]. Peter Mueller went on to start PDT and make Morgan Stanley billions of dollars in profits. The same effect can be shown to a limited extent for index portfolios[2] though less scalable because higher cap stocks have more float. Related is the Kelly Betting criteria - about 50 years old [3] which gives a simple rule for picking and allocating capital to bets. These 2 ideas - equal weighted portfolios and fractional betting would put a lot of quants out of business. [1] https://www.semanticscholar.org/paper/Financial-optimization%3A-Empirical-tests-of-biases-Muller/bdb8f14a030b9f9e5154f662b2dba3ec155d6c0c https://www.semanticscholar.org/paper/Financial-optimization... [2] https://www.semanticscholar.org/paper/Why-Does-an-Equal-Weighted-Portfolio-Outperform-and-Plyakha-Uppal/9337e16283c852fa261cc4d47cc17bd5d22c15bf https://www.semanticscholar.org/paper/Why-Does-an-Equal-Weig... [3] https://en.wikipedia.org/wiki/Kelly_criterion https://en.wikipedia.org/wiki/Kelly_criterion
- soVeryTired 6y ago
  Decent quants know about the shortcomings of Markowitz allocation. The mean is more or less impossible to measure, and covariances shift over time and are awkward to measure concurrently. So the art is in specifying the parameters. I've used zero mean and block-diagonal covariance in asset allocation before, which isn't far off 1/N allocation. The issue with Kelly betting is the same as with Markowitz: knowing the parameters of the problem. If you overstate your probability of win by mis-estimating, you'll get ruined. For what it's worth, the version of Kelly betting I'm familiar with is for Bernoulli trials. I think the Gaussian version of the Kelly criterion essentially reduces to Markowitz. Side note: kelly betting isn't new! Daniel Bernoulli knew about it back in the 1700s. Kelly just connected the formula to ideas in information theory.
  - perl4ever 6y ago
    >covariances shift over time This is what I thought when I first ran across the theory in a book. I feel like it's a widespread pattern, there is some big intractable problem, and some genius says "aha, I can reduce this to..." but what they have reduced it to is also an intractable problem, only because of the novelty it feels like progress. Everybody goes around saying "ha, you can't predict future returns from prior returns, aren't we smart", but how can you predict future covariances from prior covariances any better? I'm not sure how useful Kelly betting is though, for similar reasons. Isn't it premised on knowing your edge, so knowing how much to bet is not such a valuable secret; you need to figure out how to identify your edge reliably. Particularly that it's not negative. I eventually decided the way to go is (1) start with every idea or security equal weighted, and (2) basically never rebalance, because things that outperform do tend to continue to do so and the "imbalance" reflects new knowledge disseminated by the market.
kashyapc 6y ago
This is less well-known, but worth noting: Gerd Gigerenzer is a long-time critic of Kahneman and Tversky's work (quite popular here, understandably). So it is instructive to see the other side of the coin. I'm a little embarrassed that I only learnt of Gigerenzer's work only a few months ago, when I accidentally discovered an EconTalk podcast[1] with him. He talks about the "fast and frugal trees"[2] heuristic, criticism of the "Linda Problem" (mentioned in the article; and, incidentally, was also discussed on HN last week on that 'cognitive bias' thread[3]), and other interesting bits. It's definitely worth a careful listen—especially if you've previously read Kahneman's book. [1] https://www.econtalk.org/gerd-gigerenzer-on-gut-feelings/ https://www.econtalk.org/gerd-gigerenzer-on-gut-feelings/ [2] https://en.wikipedia.org/wiki/Fast-and-frugal_trees https://en.wikipedia.org/wiki/Fast-and-frugal_trees [3] https://news.ycombinator.com/item?id=26263458 https://news.ycombinator.com/item?id=26263458
- beaconstudios 6y ago
  nice to see someone being critical of Kahneman's ideas. I broadly like the rationalist community in general, but the focus on trying to system-2 all the things is ludicrous.
  - haltingproblem 6y ago
    I never got the system I and system II distinction. Sounds like a modern version of the bicameral mind. Is the System I/II notion falsifiable? The duality also reminds of the mind/matter boundary. But what do I know? Kahneman got a Nobel for it and along with Tversky is considered one of the most brilliant psychologists of the 20th century.
    - beaconstudios 6y ago
      my reading of it is that system 1 is instinctual and system 2 is intellectual, but of course the boundaries are blurry and arbitrarily defined. > The duality also reminds of the mind/matter boundary Nice, I'm a nondualist too. Yeah it's a strange delineation into "good" system 2 and "bad" system 1. > Kahneman got a Nobel for it and along with Tversky is considered one of the most brilliant psychologists of the 20th century. I don't think the Nobel committee have a monopoly on picking good or bad ideas ;)
sieste 6y ago
It really depends on what you mean by "outperform". If your goal is to catch the baseball, a simple gaze heuristic works well. If you want to understand how angle of impact, air resistance, rotation etc effect where the ball will land, of course you need a "complex" model that includes these factors. The most complete model, that provides our best explanation of the system, isn't always the model that yields the best predictions.

This feels like Gigerenzer has read my mind. Two weeks back I blogged about a simple heuristic to model uncertainty in software estimates [1], and I received confirmation from several people that my proposed technique might be spot-on: estimate your happy path, figure out how many dimensions your uncertainty has, and multiply the happy path estimates by (sqrt PI ^ dimensions) [1] https://tojans.me/posts/software-estimates-done-right/ https://tojans.me/posts/software-estimates-done-right/

ulucs 6y ago
Okay, I'm stumped. Isn't the gaussian function a probability density function, which means it should have an area of 1 by definition? Are you taking it as f(x) = exp(-x^2)? To keep f(0) equal to 1?
- em500 6y ago
  You're stumped because most of the "statistics primer" section in that post doesn't make any sense. The connection between the Gaussian density and the sqrt(pi) heuristic is mostly imaginary. The original heuristic (pi, sqrt(pi), pi^2) works pretty much the same with 3 instead of pi, so you can view the pi versions as numerology or charitably a nice mnemonic.
  - ToJans 6y ago
    Could you elaborate? Assume we convert all our happy path estimates to minutes. What I'm saying is that each "estimated minute" is more likely to have a gaussian distribution than a uniform standard distribution,because normal distribution is more likely to occur in nature. while I can understand that this is a controversial assumption, I'm not the first one to make it. Referring to numerology seems a bit odd? I'm really looking for a proper way here, I'm quite a rational person, so numerology is not really my cup of tea TBH.
    - ToJans 6y ago
      Addendum: this is what I was referring to: https://en.wikipedia.org/wiki/Gaussian_integral https://en.wikipedia.org/wiki/Gaussian_integral
    - em500 6y ago
      Your blog post has so many errors that I don't even know where to start. As another poster mentioned, areas under non-degenerate probability density functions are 1 by definition, whether they're uniform, Gaussian, standardized or not. What you described as a "standard uniform distribution" is really a degenerate distribution[1], meaning that you assume no uncertainty at all (stdev=0). There's nothing "uniform" about that, you might just as well start with a Gaussian with stdev=0. "converting from a standard uniform distribution to a Gaussian distribution" as you described does not make any sense at all. If you replace an initial assumption of a degenerate distribution with a Gaussian, as you seem to be doing, you replace a no-uncertainty (stdev=0) assumption with some uncertainty (so the uncertainty blow-up is infinite), but it doesn't affect point estimates such as the mean or median, unless you make separate assumptions about that. There is nothing in your story that leads to multiplying some initial time estimate by sqrt(pi). The only tenuous connection with sqrt(pi) in the whole story is that the Gaussian integral happens to be sqrt(pi). There are some deep mathematical reasons for that, which has to do with polar coordinates transformations. But it has nothing to do with adjusting uncertainties or best estimates. [1] https://en.wikipedia.org/wiki/Degenerate_distribution https://en.wikipedia.org/wiki/Degenerate_distribution
      - ToJans 6y ago
        Thank you for your valuable feedback; it will take some time to process, and I will adjust the blog post as my insights grow (potentially discarding the whole idea, but for me it's a learning process.)
    - yorwba 6y ago
      > What I'm saying is that each "estimated minute" is more likely to have a gaussian distribution If that were your actual assumption, you should measure the variance of the difference between your estimate and the actual time taken, use that to determine a confidence interval (e.g. 95% of the time, the additional delay is less than x) and then add it to your estimate.
- ToJans 6y ago
  My apologies, I'm by no means well versed in math, so I might use the wrong terminology. Here is a good example video of what I am alluding to: https://www.youtube.com/watch?v=9CgOthUUdw4 https://www.youtube.com/watch?v=9CgOthUUdw4 Maybe I should refine my blog post or include the video in the explanation?
  - sumtechguy 6y ago
    You are the second person I have come across that uses PI to get software estimates. "I am not sure why but it works, except when it doesnt then I know exactly why" What I found was most people over/under estimate things. They also tend to do it consistently at the same rate. Typically they have a scaling factor you can use. Around 3 seems to be the sweet spot for most people. You could just as easily use 3.1 or 3.2 and get a similar answer. I usually go for 3 because I can do that in my head without a calculator.
ToJans 6y ago
Update: looks like my assumptions might be completely wrong... Please ignore my initial post, and I will check if/when I can provide an update to the blog post, potentially discarding the sqrt PI idea.
mjburgess 6y ago
I think the only reason this works is that it's a power law. A power law gives you a semi-principled way of talking about different scales: atomic, table, house, world, galaxy... The rate of the power increase corresponds to our "magnification factor" Probably many sequences of powers works, and many bases: base^0.5, base^1, base^1.5, base^2, ... eg., 4^0.5, 4^1, 4^1.5, 4^2, 4^2.5 = [2, 4, 8, 16, 32] If the estimate is 3days, then the scope is: [3, 6, 12, 24, 48, 96] Now we have a "semi-principled" set of numbers to choose from, the estimator applies their domain knowledge to select the relevant scales: 3 to 24 say, based on "how big the project could be". ie., is it both possibly a small change in one area (to a table), and possibly "reworking the foundations of the house" -- if so, you have the small-scale starter number, and you can dial up using this sequence to whatever magnification suits. Probably with base pi, and sequence (0.5, 1, 2) we capture the three most relevant scales for most properly discrete software tasks. That's essentially conincidental, in that i'm sure a (2 to 4) base with several sequences would wokr.
ToJans 6y ago
**** I stand corrected and marked that section from the blog post as BS; please accept my apologies. ****

bujak300 6y ago

I can recommend the "Simple heuristics that make us smart" book by Gigerenzer, it's an insightful and entertainig read: for this book he and his co-authors put their money where their mouth is and invested on the stock market based on the most simple heuristics imaginable.

wrp 6y ago
Gigerenzer has published several books, but I think that is the best one, because later books are just repeating the ideas. Although I think his ideas are good, they have drawn a lot of criticism. There are several discussion articles in issue 23(5) of Behavioral and Brain Science.
daemoncoder 6y ago
TED Talk https://youtu.be/-Lg7G8TMe_A https://youtu.be/-Lg7G8TMe_A
- cwillu 6y ago
  TEDx you mean. Which matters quite a bit, given that there are effectively no restrictions on who can give a TEDx talk :p Which isn't to say that it's useless, just that the "TED" gatekeeping role isn't actually happening here, instead the gatekeeper is "people who have heard of TEDx", which is very different and imo quite a bit less useful.
  - yboris 6y ago
    Even I had a TEDx talk ;) https://www.youtube.com/watch?v=DPe9-OM7vgU https://www.youtube.com/watch?v=DPe9-OM7vgU

sva_ 6y ago

(2017)

rramadass 6y ago

I find this entire field of study fascinating. The reason is simple; ever since the start of the Scientific Revolution everything has become quite complex and multivariate and we have gotten into the habit of trying to solve them all using mathematical models. This has the advantage that once proven, it can be applied "mindlessly" towards a desired result. With Heuristics so many factors have to come together in a individual for any given event that repeatability is not certain. Therein lies the difficulty.

cwillu 6y ago

Effectively, every "knob" your model has is a major point against the model, but because people don't really get that (it's another "Linda is a bank teller and is active in the feminist movement" instance) they tend to overcomplicate their models compared to the amount of validation they've performed on the model. An alternative to putting nearly all the weight on a single complicated model created ex nihilo is to have some reasonable "zero information" prior distribution over models (weighted by complexity) where, because rules-of-thumb are very simple models, those simple models dominate. In fact, the alternative offered is itself a simple rule-of-thumb approach to this, basically saying that the more complicated models can be almost entirely neglected, and so you can really just do an even split over rules-of-thumb and still outperform any model that naïvely tries to “know best” by accounting for everything up front. In short, it's really fucking hard to properly validate that a model applies to reality, and until you appreciate just how hard it is, you're much better off radically biased towards simplicity.

phkahler 6y ago
Optimized systems are fragile. Probably true of models. The most profound class I had in college was thermodynamics, where all the complexity of physical systems is lumped into a few parameters. It seems crude but those parameters include all the energy and can be used to make very useful calculations.
- cwillu 6y ago
  Is this a fair restatement: “Models (and systems) that accurately reflect reality with a high fidelity due in part to their accurate assignment of values to a larger number of parameters are fragile”? I agree that that's true, but my point (and I think part of what the article is saying) is that most “optimized” systems/models are not in fact doing that. In which case, the point I'd take (if I read you correctly) is that even once you _have_ managed to accurately set the parameters (which is harder to do and validate than it looks), you still have a fragile model/system. Alternatively [edited to add]: Define a model as a system that is attempting to match another system in behaviour. And so now you have two problems: ① a complicated model is itself fragile (by analogy to the system it's modelling) ② a complicated system has more ways to diverge from the system it's trying to model Hmm, in which case it's kinda the same thing after all, except one is about making the model in the first place, and the other is maintaining it, maybe?

HPsquared 6y ago

Simple models have fewer sources of error, and the small number of parameters can be better validated against reality.

rramadass 6y ago
That certainly sounds plausible, however i feel that it is the applicability/non-applicability of parameters themselves that is the key. You need to choose the right parameters before even bothering about the model function and this is where prior knowledge and experience are important. The works of Gary Klein and Nassim Taleb are also relevant here.
- levosmetalo 6y ago
  Nassim Taleb actually cites Prof. Gigerenzer in the topic of simple heuristics.

johndoe42377 6y ago

Oh, really. So applying probabilities to partially observable systems yields bullshit, and estimated probabilities based on observations of the past events does not predict anything for evolving systems? I got banned here for such assertions lmao. What else is new?

prionassembly 6y ago

A documentary I strongly recommend to everybody (it's on YouTube): Tao Russell's "Being in the world". It's ostensibly about the life and work of Hubert Dreyfus, the famous MIT philosopher that was an AI skeptic. But it interviews carpenters and jazz and flamenco musicians and cooks etc. and (way oversimplifying here) the takeaway is that the skill is not in you, it's in this moment of skillfull interaction with the world. Gigerenzer seems to come awfully close to this point, but keeps missing it. At the critical moment the genius baseball batter is one with his bat, with the wind, with the noises, with each signal coming from his musculature -- this is not a Cartesian subject metaphorically manipulating levers and keys to move a robot, this is a situation. It's a whole thing. This is why even in sports where pitch sizes are regulamented (like soccer) there's a substantial home game advantage even in the absence of crowds. That's where they train. This is why pure mathematicians obsess over that particular chalk. This is why Kenny G sounds so shitty, he's recording against a backing track in an aquarium in a studio, and why improvised jazz sounds so great.

techer 6y ago
"Standard pitch measurements. Not all pitches are the same size, though the preferred size for many professional teams' stadiums is 105 by 68 metres (115 yd × 74 yd) with an area of 7,140 square metres (76,900 sq ft; 1.76 acres; 0.714 ha)." The field dimensions are within the range found optimal by FIFA: 110–120 yards (100–110 m) long by 70–80 yards (64–75 m) wide. Managers can instruct groundskeepers to alter the pitch size at home. I heard that referees are not feeling pressured into giving or not giving penalties that they may have been pressured into due to home crowds. Some sports have fixed sizes. I’ll watch your recommendation. Thanks

haltingproblem 6y ago

This result is still not widely known but a simple equal weighed rule outperforms Markowitz mean variance for most alpha trading strategies i.e. strategies that predict outperformance. Peter Muller shows this in an original paper in 1993 [1]. Peter Mueller went on to start PDT and make Morgan Stanley billions of dollars in profits. The same effect can be shown to a limited extent for index portfolios[2] though less scalable because higher cap stocks have more float. Related is the Kelly Betting criteria - about 50 years old [3] which gives a simple rule for picking and allocating capital to bets. These 2 ideas - equal weighted portfolios and fractional betting would put a lot of quants out of business. [1] https://www.semanticscholar.org/paper/Financial-optimization%3A-Empirical-tests-of-biases-Muller/bdb8f14a030b9f9e5154f662b2dba3ec155d6c0c https://www.semanticscholar.org/paper/Financial-optimization... [2] https://www.semanticscholar.org/paper/Why-Does-an-Equal-Weighted-Portfolio-Outperform-and-Plyakha-Uppal/9337e16283c852fa261cc4d47cc17bd5d22c15bf https://www.semanticscholar.org/paper/Why-Does-an-Equal-Weig... [3] https://en.wikipedia.org/wiki/Kelly_criterion https://en.wikipedia.org/wiki/Kelly_criterion

soVeryTired 6y ago
Decent quants know about the shortcomings of Markowitz allocation. The mean is more or less impossible to measure, and covariances shift over time and are awkward to measure concurrently. So the art is in specifying the parameters. I've used zero mean and block-diagonal covariance in asset allocation before, which isn't far off 1/N allocation. The issue with Kelly betting is the same as with Markowitz: knowing the parameters of the problem. If you overstate your probability of win by mis-estimating, you'll get ruined. For what it's worth, the version of Kelly betting I'm familiar with is for Bernoulli trials. I think the Gaussian version of the Kelly criterion essentially reduces to Markowitz. Side note: kelly betting isn't new! Daniel Bernoulli knew about it back in the 1700s. Kelly just connected the formula to ideas in information theory.
- perl4ever 6y ago
  >covariances shift over time This is what I thought when I first ran across the theory in a book. I feel like it's a widespread pattern, there is some big intractable problem, and some genius says "aha, I can reduce this to..." but what they have reduced it to is also an intractable problem, only because of the novelty it feels like progress. Everybody goes around saying "ha, you can't predict future returns from prior returns, aren't we smart", but how can you predict future covariances from prior covariances any better? I'm not sure how useful Kelly betting is though, for similar reasons. Isn't it premised on knowing your edge, so knowing how much to bet is not such a valuable secret; you need to figure out how to identify your edge reliably. Particularly that it's not negative. I eventually decided the way to go is (1) start with every idea or security equal weighted, and (2) basically never rebalance, because things that outperform do tend to continue to do so and the "imbalance" reflects new knowledge disseminated by the market.

kashyapc 6y ago

This is less well-known, but worth noting: Gerd Gigerenzer is a long-time critic of Kahneman and Tversky's work (quite popular here, understandably). So it is instructive to see the other side of the coin. I'm a little embarrassed that I only learnt of Gigerenzer's work only a few months ago, when I accidentally discovered an EconTalk podcast[1] with him. He talks about the "fast and frugal trees"[2] heuristic, criticism of the "Linda Problem" (mentioned in the article; and, incidentally, was also discussed on HN last week on that 'cognitive bias' thread[3]), and other interesting bits. It's definitely worth a careful listen—especially if you've previously read Kahneman's book. [1] https://www.econtalk.org/gerd-gigerenzer-on-gut-feelings/ https://www.econtalk.org/gerd-gigerenzer-on-gut-feelings/ [2] https://en.wikipedia.org/wiki/Fast-and-frugal_trees https://en.wikipedia.org/wiki/Fast-and-frugal_trees [3] https://news.ycombinator.com/item?id=26263458 https://news.ycombinator.com/item?id=26263458

beaconstudios 6y ago
nice to see someone being critical of Kahneman's ideas. I broadly like the rationalist community in general, but the focus on trying to system-2 all the things is ludicrous.
- haltingproblem 6y ago
  I never got the system I and system II distinction. Sounds like a modern version of the bicameral mind. Is the System I/II notion falsifiable? The duality also reminds of the mind/matter boundary. But what do I know? Kahneman got a Nobel for it and along with Tversky is considered one of the most brilliant psychologists of the 20th century.
  - beaconstudios 6y ago
    my reading of it is that system 1 is instinctual and system 2 is intellectual, but of course the boundaries are blurry and arbitrarily defined. > The duality also reminds of the mind/matter boundary Nice, I'm a nondualist too. Yeah it's a strange delineation into "good" system 2 and "bad" system 1. > Kahneman got a Nobel for it and along with Tversky is considered one of the most brilliant psychologists of the 20th century. I don't think the Nobel committee have a monopoly on picking good or bad ideas ;)

sieste 6y ago

It really depends on what you mean by "outperform". If your goal is to catch the baseball, a simple gaze heuristic works well. If you want to understand how angle of impact, air resistance, rotation etc effect where the ball will land, of course you need a "complex" model that includes these factors. The most complete model, that provides our best explanation of the system, isn't always the model that yields the best predictions.