10 ms·

"The point we will be making here is that logically, neither trial and error nor "chance" and serendipity can be behind the gains in technology and empirical sc

by jjk166 8y ago

"The point we will be making here is that logically, neither trial and error nor "chance" and serendipity can be behind the gains in technology and empirical science attributed to them. By definition chance cannot lead to long term gains (it would no longer be chance); trial and error cannot be unconditionally effective: errors cause planes to crash, buildings to collapse, and knowledge to regress." This is an extremely flawed initial assumption. There is no requirement for chance to be centered around zero. Consider rolling a dice: sometimes you'll get more than the mean, sometimes less, but you'll never roll a negative number. You can certainly win on chance in the long run, that's the foundation of casinos and insurance companies. It's hard to imagine a scenario where trial and error can possibly lead to knowledge regressing. Consider randomly digging holes in the ground: after enough holes you will eventually strike gold, and you will never lose physical gold in the process. However, you may lose significant time, wealth, and effort that could have been better converted to gold. The optimal way to strike gold is not to dig more, shallower holes, but to learn enough geology to understand where gold is likely to be found and concentrate your prospecting there. No experiment could ever possibly hurt scientific knowledge. People tinkering will certainly make occasional discoveries. In a brand new field with a lot of low hanging fruit, these discoveries will be numerous and the cost will be low. But in a developed field where people have a good idea where the remaining discoveries are likely to be found and the effort to conduct such experiments is substantial, targeted approaches become optimal. Reducing the unit cost of experiments is always nice, but is not generally feasible. This strategy of "convexity" is a very poor substitute in the real world for understanding.

doovd 8y ago
> This is an extremely flawed initial assumption. There is no requirement for chance to be centered around zero. Consider rolling a dice: sometimes you'll get more than the mean, sometimes less, but you'll never roll a negative number. You can certainly win on chance in the long run, that's the foundation of casinos and insurance companies. It's hard to imagine a scenario where trial and error can possibly lead to knowledge regressing. Casinos and Insurance companies don't win on "chance". There is an expected value of the events that are in consideration here, and these firms price their services such that their return is higher than the expected value...very little down to chance. It's as if we played a die roll game where I paid you the value on the dice each time you rolled it and you paid me >3.5 units per roll to take your turn. That's the house edge.
iso1337 8y ago
My take was that he is using a slightly different definition of "chance" or "trial and error" than what is perhaps common. I don't quite grasp his definition enough to be able to do it justice. I don't know enough about geology to be able to comment intelligently on your digging holes in the ground point. However, from my own background in biology research, a lot of what he writes strikes true with some caveats. Biological systems are highly complex, and a lot of reductionist basic research work does seem to be driven by understanding along the forms of "I have a mental model of this subsystem, if I do X, then I expect Y." However, a lot is also discovered via his "convexity" principles (which I agree most laypeople would label as "trial and error"). Biologists often discover functions by randomly mutating billions or trillions of individual microbes, screening for interesting phenotypes, and then sequencing to discover the supposedly causal mutations. Where the understanding approach really breaks down is in engineering systems -- which I believe requires an even higher level of understanding than the qualitative mental models bandied about in biology. We simply don't understand enough to be able to develop most drugs with that sort of rational approach, so reducing costs per attempt (while keeping all else equal, which is something often overlooked) would be beneficial. Unfortunately, things in the industry appear to be heading the other direction.
EricBurnett 8y ago
The expected value of an infinite series of coin flips centers on 0. Sure, you might get lucky, but you can't leverage that as a strategy to ratchet up value. That's the point about luck in the non convex case. To your mining example, that's where prospecting comes in. Use your knowledge to make checking candidate locations cheaper (increasing convexity) - knowledge of geology to approximate likelihood, improvements to technology to determine if gold exists, etc. Then go check as many candidate mines as possible. It's much more cost effective to have a lot of shallow mines than it is to extract every ounce all the way down to the crust from a single mine.
- adrianN 8y ago
  If heads is 1 and tails is 0 then the expected value is 0.5. The expectation really depends on what values your random variable can take.
  - EricBurnett 8y ago
    That's not the non convex case though (as e.g. 1 and -1 is). You're right that chance can be convex; the premise of the article is that that's the property of interest, and that randomness alone isn't sufficient.
skybrian 8y ago
Yeah, it's badly written in a way to provoke controversy, but fundamentally, I'm not sure there's any real disagreement here? In simple terms, he's just saying that it needs to be safe to take chances in order for it to be worthwhile to take chances. As you say, science is an example of a system where it's often safe to take chances, because you don't risk losing any knowledge from a failed experiment. (But that doesn't mean there are no costs! You can lose time and money. And I'll also point out that some experiments can be dangerous.) In any search, whether you can find something interesting is going to depend at least partly on the landscape, so understanding the landscape better will improve the search process, along with your estimates of whether it's worth doing at all. Calling this property of a desirable landscape "convexity" doesn't, in itself, help you understand the landscape, but it doesn't seem wrong?
- DougBTX 8y ago
  > Yeah, it's badly written in a way to provoke controversy I’m not sure it is worth my time to read something that is badly written to provoke controversy, even if it does make good clickbait.
  - milesvp 8y ago
    Yeah, I used to go out of my way to follow links to Taleb’s writing, but then I kept running into pieces like this one, where it wasn’t clear if he knew he was fundamentally (if not subtly) wrong. He seems to welcome controversy, and his recent childish attack on guy from 538 means I won’t give him more than a paragragh to convince me the rest of the piece is worth my time.
    - solveit 8y ago
      I hate it when smart, competent people become famous, and a few years later they become total loons. Happens way too often.
    - coldtea 8y ago
      >* but then I kept running into pieces like this one, where it wasn’t clear if he knew he was fundamentally (if not subtly) wrong* He is not "fundamentally" wrong here though. The parent comment misread what TFA says (as I replied above).
evrydayhustling 8y ago
I think another way to put this is that betting on convexity still requires a formal theory of costs and benefits, even if not of the risks involved. There are some cases where that model can itself seem pretty fragile (like estimating the opportunity cost on your hole digging), but others where it night feel reasonable. Pick your poison!
bchjam 8y ago
> No experiment could ever possibly hurt scientific knowledge what about one with a falsified result? fwiw I think convexity and understanding are relatively orthogonal, but how could one employ the former without the latter? However the author's position seems to be more that gaming systems works better than exploring their contexts. Sometimes you might make more money in less time, but the money is all you'll get out of it. In practice, maybe understanding and convex payoff functions are both useful at different scales.
- hannasanarion 8y ago
  Falsification is something that you do to hypotheses, not data. Experiments that Aristotle performed to demonstrate classical elemental theory are still valid and useful, his incorrect interpretation of the underlying mechanisms notwithstanding
  - yesenadam 8y ago
    I thought Aristotle didn't do experiments.
    - hannasanarion 8y ago
      Sure he did. He didn't follow the modern Bacon/Popper empirical method with testable hypotheses, but he still performed experiments and drew conclusions based on what he saw. All beside the point: his observations are not invalid, his conclusions are
      - mattkrause 8y ago
        So women do have fewer teeth?!
        coldtea 8y ago
        No, but not because Aristotle was against observation. Just because he either miscounted or trusted a wrong earlier observation. What he wrote is: "”Males have more teeth than females in the case of men, sheep, goats, and swine; in the case of other animals _observations have not yet been made_” Emphasis mine.
lolc 8y ago
I'd say the zero is where you don't do anything. There's no gain and no loss. Once you decide to do something, you have to spend to make it happen. The dice example is pointless when the roll is assumed to be zero effort and you get a random number for it. Where's the gain? > No experiment could ever possibly hurt scientific knowledge. Sure could. An experiment might yield a false result. You might get a false negative and abandon a promising discovery. You might get a false positive and waste more experiments on an impossible setup. It's all about opportunity cost. But overall I couldn't follow Taleb's writing. It's not accessible to me anymore.
buboard 8y ago
> This is an extremely flawed initial assumption. that was not an initial assumption but (presumably) the subject of the essay. its not clearly stated in the text though > you can certainly win on chance in the long run No you can't ; both casinos and insurance companies make choices that give them an assymetry in gains, and thus they benefit from the randomness of events. Chance is by definition centered around the mean. The question of scientific research directions is interesting , but i m not sure that research is a random walk. There are biases and "hunches" that guide scientists.
- User23 8y ago
  This is the concept of ergodicity. The fun fact is that even if casinos offered a game with 50% odds players without unlimited bankrolls would still always bust eventually[1]. It's similar to why betting $1 to make $1 million at million to one odds is a smart bet, but betting $1 million to make $1 at the equivalent odds isn't, unless you have an unlimited bankroll. [1] If they use the Kelly criterion they won't bust hard, but they'll still lose most of their bankroll.
  - klipt 8y ago
    But real world casinos also have limited bankrolls and can go bust given 50% odds. Which is why they never offer 50% odds.
  - buboard 8y ago
    thaat's only because they have "chosen" to not allow players with negative balances or margins.
bryanrasmussen 8y ago
You could conceivably have a number of related experiments that lead you to conclude that phlogiston exists.
quickthrower2 8y ago
I read this from the perspective of money. If an invention or discovery makes some organisation, company or government a lot of money, then that money can be ploughed back into research. If on average science and technology is making us more money than it costs it will be "antifragile".
mr_toad 8y ago
The first two paragraphs of the article are a straw man. I think most of us were biting our tongues at that point. The thesis of the rest of the article contradicts the initial premise. Your scenario of ramdomly digging holes are a perfect example of what the author is explaining: “Critically, convex payoffs benefit from uncertainty and disorder.“
- fujimotos 8y ago
  IMHO, the author tries to argue that we can get a better understanding by thinking in term of the structure of the payoff function, rather than focusing on the input itself. Yes, the try-and-error process and chance are important, but the convexity of the output function is exactly what makes them so rewarding.
coldtea 8y ago
>No experiment could ever possibly hurt scientific knowledge. (...) This is an extremely flawed initial assumption. There is no requirement for chance to be centered around zero. Consider rolling a dice: sometimes you'll get more than the mean, sometimes less, but you'll never roll a negative number. That's the convexity property Taleb argues must lie underneath research though. So, you're saying the same thing. What he means with the first paragraph is not that chance can't lead to gains -- it's that chance alone cannot lead to gains. There should be an additional property, and that's what the article is about. Consider your counter-argument: "You can certainly win on chance in the long run, that's the foundation of casinos". And yet, that's not the foundation of casinos. That's what the article speaks against. For if it was chance + long run alone the "foundation" then it would work for the players too. But players face ruin in the long run (unless they have an infinite supply of money), while casinos do not. The foundation of casinos is chance + resilience to chance events (a casino doesn't go under from this or that player winning) -- e.g. the exact convexity the author talks about.

by jjk166 8y ago

doovd 8y ago

> This is an extremely flawed initial assumption. There is no requirement for chance to be centered around zero. Consider rolling a dice: sometimes you'll get more than the mean, sometimes less, but you'll never roll a negative number. You can certainly win on chance in the long run, that's the foundation of casinos and insurance companies. It's hard to imagine a scenario where trial and error can possibly lead to knowledge regressing. Casinos and Insurance companies don't win on "chance". There is an expected value of the events that are in consideration here, and these firms price their services such that their return is higher than the expected value...very little down to chance. It's as if we played a die roll game where I paid you the value on the dice each time you rolled it and you paid me >3.5 units per roll to take your turn. That's the house edge.

iso1337 8y ago

My take was that he is using a slightly different definition of "chance" or "trial and error" than what is perhaps common. I don't quite grasp his definition enough to be able to do it justice. I don't know enough about geology to be able to comment intelligently on your digging holes in the ground point. However, from my own background in biology research, a lot of what he writes strikes true with some caveats. Biological systems are highly complex, and a lot of reductionist basic research work does seem to be driven by understanding along the forms of "I have a mental model of this subsystem, if I do X, then I expect Y." However, a lot is also discovered via his "convexity" principles (which I agree most laypeople would label as "trial and error"). Biologists often discover functions by randomly mutating billions or trillions of individual microbes, screening for interesting phenotypes, and then sequencing to discover the supposedly causal mutations. Where the understanding approach really breaks down is in engineering systems -- which I believe requires an even higher level of understanding than the qualitative mental models bandied about in biology. We simply don't understand enough to be able to develop most drugs with that sort of rational approach, so reducing costs per attempt (while keeping all else equal, which is something often overlooked) would be beneficial. Unfortunately, things in the industry appear to be heading the other direction.

EricBurnett 8y ago

The expected value of an infinite series of coin flips centers on 0. Sure, you might get lucky, but you can't leverage that as a strategy to ratchet up value. That's the point about luck in the non convex case. To your mining example, that's where prospecting comes in. Use your knowledge to make checking candidate locations cheaper (increasing convexity) - knowledge of geology to approximate likelihood, improvements to technology to determine if gold exists, etc. Then go check as many candidate mines as possible. It's much more cost effective to have a lot of shallow mines than it is to extract every ounce all the way down to the crust from a single mine.

adrianN 8y ago
If heads is 1 and tails is 0 then the expected value is 0.5. The expectation really depends on what values your random variable can take.
- EricBurnett 8y ago
  That's not the non convex case though (as e.g. 1 and -1 is). You're right that chance can be convex; the premise of the article is that that's the property of interest, and that randomness alone isn't sufficient.

skybrian 8y ago

Yeah, it's badly written in a way to provoke controversy, but fundamentally, I'm not sure there's any real disagreement here? In simple terms, he's just saying that it needs to be safe to take chances in order for it to be worthwhile to take chances. As you say, science is an example of a system where it's often safe to take chances, because you don't risk losing any knowledge from a failed experiment. (But that doesn't mean there are no costs! You can lose time and money. And I'll also point out that some experiments can be dangerous.) In any search, whether you can find something interesting is going to depend at least partly on the landscape, so understanding the landscape better will improve the search process, along with your estimates of whether it's worth doing at all. Calling this property of a desirable landscape "convexity" doesn't, in itself, help you understand the landscape, but it doesn't seem wrong?

DougBTX 8y ago
> Yeah, it's badly written in a way to provoke controversy I’m not sure it is worth my time to read something that is badly written to provoke controversy, even if it does make good clickbait.
- milesvp 8y ago
  Yeah, I used to go out of my way to follow links to Taleb’s writing, but then I kept running into pieces like this one, where it wasn’t clear if he knew he was fundamentally (if not subtly) wrong. He seems to welcome controversy, and his recent childish attack on guy from 538 means I won’t give him more than a paragragh to convince me the rest of the piece is worth my time.
  - solveit 8y ago
    I hate it when smart, competent people become famous, and a few years later they become total loons. Happens way too often.
  - coldtea 8y ago
    >* but then I kept running into pieces like this one, where it wasn’t clear if he knew he was fundamentally (if not subtly) wrong* He is not "fundamentally" wrong here though. The parent comment misread what TFA says (as I replied above).

evrydayhustling 8y ago

I think another way to put this is that betting on convexity still requires a formal theory of costs and benefits, even if not of the risks involved. There are some cases where that model can itself seem pretty fragile (like estimating the opportunity cost on your hole digging), but others where it night feel reasonable. Pick your poison!

bchjam 8y ago

> No experiment could ever possibly hurt scientific knowledge what about one with a falsified result? fwiw I think convexity and understanding are relatively orthogonal, but how could one employ the former without the latter? However the author's position seems to be more that gaming systems works better than exploring their contexts. Sometimes you might make more money in less time, but the money is all you'll get out of it. In practice, maybe understanding and convex payoff functions are both useful at different scales.

hannasanarion 8y ago
Falsification is something that you do to hypotheses, not data. Experiments that Aristotle performed to demonstrate classical elemental theory are still valid and useful, his incorrect interpretation of the underlying mechanisms notwithstanding
- yesenadam 8y ago
  I thought Aristotle didn't do experiments.
  - hannasanarion 8y ago
    Sure he did. He didn't follow the modern Bacon/Popper empirical method with testable hypotheses, but he still performed experiments and drew conclusions based on what he saw. All beside the point: his observations are not invalid, his conclusions are
    - mattkrause 8y ago
      So women do have fewer teeth?!
      - coldtea 8y ago
        No, but not because Aristotle was against observation. Just because he either miscounted or trusted a wrong earlier observation. What he wrote is: "”Males have more teeth than females in the case of men, sheep, goats, and swine; in the case of other animals _observations have not yet been made_” Emphasis mine.

lolc 8y ago

I'd say the zero is where you don't do anything. There's no gain and no loss. Once you decide to do something, you have to spend to make it happen. The dice example is pointless when the roll is assumed to be zero effort and you get a random number for it. Where's the gain? > No experiment could ever possibly hurt scientific knowledge. Sure could. An experiment might yield a false result. You might get a false negative and abandon a promising discovery. You might get a false positive and waste more experiments on an impossible setup. It's all about opportunity cost. But overall I couldn't follow Taleb's writing. It's not accessible to me anymore.

buboard 8y ago

> This is an extremely flawed initial assumption. that was not an initial assumption but (presumably) the subject of the essay. its not clearly stated in the text though > you can certainly win on chance in the long run No you can't ; both casinos and insurance companies make choices that give them an assymetry in gains, and thus they benefit from the randomness of events. Chance is by definition centered around the mean. The question of scientific research directions is interesting , but i m not sure that research is a random walk. There are biases and "hunches" that guide scientists.

User23 8y ago
This is the concept of ergodicity. The fun fact is that even if casinos offered a game with 50% odds players without unlimited bankrolls would still always bust eventually[1]. It's similar to why betting $1 to make $1 million at million to one odds is a smart bet, but betting $1 million to make $1 at the equivalent odds isn't, unless you have an unlimited bankroll. [1] If they use the Kelly criterion they won't bust hard, but they'll still lose most of their bankroll.
- klipt 8y ago
  But real world casinos also have limited bankrolls and can go bust given 50% odds. Which is why they never offer 50% odds.
- buboard 8y ago
  thaat's only because they have "chosen" to not allow players with negative balances or margins.

bryanrasmussen 8y ago

You could conceivably have a number of related experiments that lead you to conclude that phlogiston exists.

quickthrower2 8y ago

I read this from the perspective of money. If an invention or discovery makes some organisation, company or government a lot of money, then that money can be ploughed back into research. If on average science and technology is making us more money than it costs it will be "antifragile".

mr_toad 8y ago

The first two paragraphs of the article are a straw man. I think most of us were biting our tongues at that point. The thesis of the rest of the article contradicts the initial premise. Your scenario of ramdomly digging holes are a perfect example of what the author is explaining: “Critically, convex payoffs benefit from uncertainty and disorder.“

fujimotos 8y ago
IMHO, the author tries to argue that we can get a better understanding by thinking in term of the structure of the payoff function, rather than focusing on the input itself. Yes, the try-and-error process and chance are important, but the convexity of the output function is exactly what makes them so rewarding.

coldtea 8y ago

>No experiment could ever possibly hurt scientific knowledge. (...) This is an extremely flawed initial assumption. There is no requirement for chance to be centered around zero. Consider rolling a dice: sometimes you'll get more than the mean, sometimes less, but you'll never roll a negative number. That's the convexity property Taleb argues must lie underneath research though. So, you're saying the same thing. What he means with the first paragraph is not that chance can't lead to gains -- it's that chance alone cannot lead to gains. There should be an additional property, and that's what the article is about. Consider your counter-argument: "You can certainly win on chance in the long run, that's the foundation of casinos". And yet, that's not the foundation of casinos. That's what the article speaks against. For if it was chance + long run alone the "foundation" then it would work for the players too. But players face ruin in the long run (unless they have an infinite supply of money), while casinos do not. The foundation of casinos is chance + resilience to chance events (a casino doesn't go under from this or that player winning) -- e.g. the exact convexity the author talks about.