5 ms·

Scientists use big data to understand what separates winners from losers

63 points by juniusfree 7y ago

fsloth 7y ago
Well, I say... "move fast and break things" is now a scientifically validated methodology.
- GoToRO 7y ago
  Move fast, break things and succeed is the valid methodology. Just move fast, break things and put them in production, not so much.
- deleted 7y ago
  [deleted]
- marcosdumay 7y ago
  Not so much. It's validating "fail fast".
lifeisstillgood 7y ago
This sounds like it has immediate practical applications at any incubator / VC firm.
kekebo 7y ago
Why was the original headline changed to the less descriptive subheadline of the article?
- Dowwie 7y ago
  for those not yet aware of this title change tracker: https://hackernewstitles.netlify.com/ https://hackernewstitles.netlify.com/
reportgunner 7y ago
So did the scientists succeed or fail ? I couldn't really tell from the article and the paper, albeit I just skimmed it.
- keymon-o 7y ago
  If there were any conclusions made in the article, OP would probably make sure to dramatically acknowledge the subjective idea in the title. I guess we can only wait for the title "Here is 10 reasons why you are a loser, Harry..."..
itronitron 7y ago
>> The average number of failures for those who failed at least once before success was 2.03 for NIH, 1.5 for startups and 3.90 for terrorist groups. Regarding counter-terrorism, that would suggest that rather than more surveillance we instead need better reporting of and responding to failed terrorist incidents/attacks.
- perl4ever 7y ago
  This reminds me of (I think) Richard Feynman's comment about the Challenger, to the effect that when the o-rings were eroded by 1/3 or whatever, they called that a safety factor of 3, whereas they should have stopped and said "wait, it wasn't supposed to do that at all, something is seriously wrong". I'm not sure if it was him or someone else that pointed out when you do a post-mortem on a disaster, you generally find a history of near-misses, but people don't take near-misses as seriously as they should.
  - michelpp 7y ago
    You're right, it was part of his opinion filed with the Rogers Commission: https://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_of_Richard_Feynman https://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_...
  - typhonic 7y ago
    The book Apollo by Charles Murray and Catherine Bly Cox shows a similar concept. In the investigation after the fire in Apollo 1, they found many possible near misses. But when each of those items did not lead to a failure, they became accepted. They poor designs or installations were allowed to slide and the accumulation of trouble spots kept increasing.
- golemotron 7y ago
  Thank goodness startup founders aren't as determined as terrorists.
- ummwhat 7y ago
  There was once a mathematician who got caught at the airport with a bomb. They took him in for questioning. When asked why he had a bomb in his suitcase he stated "the odds of there being a bomb on your airplane are less than one in ten million. So I figured what are the odds of there being two bombs on my flight?" Do you see why this is similar? When an outcome (terrorism) becomes an input to its own predictor (past terrorism failures), the logic breaks.
Fernicia 7y ago
>the faster you fail, the better your chances of success, and the more time between attempts, the more likely you are to fail again This is confounded by the fact that the closer one is to success the more frequent their attempts tend to be. The article doesn't indicate the research took this into consideration. e.g. Golfers takes shots more frequently the closer they are to a hole. But telling a golfer from the start to take a series of frequent short shots is bad advice.
- keymon-o 7y ago
  Golf match should not be defined as a consecutive failures before a potential success, but more of a set of strategic actions before a potential success. But yes, it is not defined what they took into account. e.g. Calling the same potential client hundred times in a row, can get you a restraining order only. If qualitative input and output of an attempt would be taken in account, quantity of an attempt would not really be that significant.
- vannevar 7y ago
  It is also confounded by the element of ruin, the ability of a gambler to stay in the game based on their stake. It's reasonable to suppose that some of the lag in re-try times (particularly with regard to startups) is attributable to raising money, which will be much easier for someone who starts out wealthy to begin with, even if they're not directly investing their own funds.
sqba 7y ago
There's hope yet for the Donald.
o09rdk 7y ago
I can't speak to venture funding or terrorism, but something seems really misleading to me about the NIH grant applications model. My sense is that a grant, like an academic paper, is often basically well-received or not. So prior to going into it, if you're familiar with the details of the particular grant, you can get some sense of interest or not. The grant may have even been solicited in a certain sense by a program officer or something, so it's quasi-invited. Those grants that are quasi-invited, or well-received, will basically involve polishing on subsequent revisions. Those that are totally unsolicited and not well-received it doesn't matter how much polishing you do often, it will not go anywhere. I think where this becomes relevant to the paper is that this often has very little to do with the process of the grant revisions. It says little about "how someone responds to failure", and everything about connections with the grant agency, program officers, and luck. All that stuff that happens before the grant is even submitted plays into it, and this paper kind of ignores that. You could say that it does speak to learning, in that you could ask "why doesn't the other scientist adjust their strategy?" To which I'd say, there are enormous pressures to submit anyway, and for someone who doesn't get good mentoring about what to target, or what agency to target, or whose research doesn't jive with the priorities of the division head at that time, or whatever, they might just not get it, and it might easily be over the 5 year period of the study. Basically, at least with NIH grants I think this study is really misleading and potentially harmful, because it kind of suggests failed applicants just aren't learning, when I think what's really happening is that you have a mixture of two groups, one of which can learn something, and the other of whom it just doesn't matter if they learn or not, because the outcome has kind of been preordained. That is, the process they predict from is an indicator of which mixture class the applicant belongs in, not the cause of the outcome.

63 points by juniusfree 7y ago

fsloth 7y ago

Well, I say... "move fast and break things" is now a scientifically validated methodology.

GoToRO 7y ago
Move fast, break things and succeed is the valid methodology. Just move fast, break things and put them in production, not so much.
deleted 7y ago
[deleted]
marcosdumay 7y ago
Not so much. It's validating "fail fast".

lifeisstillgood 7y ago

This sounds like it has immediate practical applications at any incubator / VC firm.

kekebo 7y ago

Why was the original headline changed to the less descriptive subheadline of the article?

Dowwie 7y ago
for those not yet aware of this title change tracker: https://hackernewstitles.netlify.com/ https://hackernewstitles.netlify.com/

reportgunner 7y ago

So did the scientists succeed or fail ? I couldn't really tell from the article and the paper, albeit I just skimmed it.

keymon-o 7y ago
If there were any conclusions made in the article, OP would probably make sure to dramatically acknowledge the subjective idea in the title. I guess we can only wait for the title "Here is 10 reasons why you are a loser, Harry..."..

itronitron 7y ago

>> The average number of failures for those who failed at least once before success was 2.03 for NIH, 1.5 for startups and 3.90 for terrorist groups. Regarding counter-terrorism, that would suggest that rather than more surveillance we instead need better reporting of and responding to failed terrorist incidents/attacks.

perl4ever 7y ago
This reminds me of (I think) Richard Feynman's comment about the Challenger, to the effect that when the o-rings were eroded by 1/3 or whatever, they called that a safety factor of 3, whereas they should have stopped and said "wait, it wasn't supposed to do that at all, something is seriously wrong". I'm not sure if it was him or someone else that pointed out when you do a post-mortem on a disaster, you generally find a history of near-misses, but people don't take near-misses as seriously as they should.
- michelpp 7y ago
  You're right, it was part of his opinion filed with the Rogers Commission: https://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_of_Richard_Feynman https://en.wikipedia.org/wiki/Rogers_Commission_Report#Role_...
- typhonic 7y ago
  The book Apollo by Charles Murray and Catherine Bly Cox shows a similar concept. In the investigation after the fire in Apollo 1, they found many possible near misses. But when each of those items did not lead to a failure, they became accepted. They poor designs or installations were allowed to slide and the accumulation of trouble spots kept increasing.
golemotron 7y ago
Thank goodness startup founders aren't as determined as terrorists.
ummwhat 7y ago
There was once a mathematician who got caught at the airport with a bomb. They took him in for questioning. When asked why he had a bomb in his suitcase he stated "the odds of there being a bomb on your airplane are less than one in ten million. So I figured what are the odds of there being two bombs on my flight?" Do you see why this is similar? When an outcome (terrorism) becomes an input to its own predictor (past terrorism failures), the logic breaks.

Fernicia 7y ago

>the faster you fail, the better your chances of success, and the more time between attempts, the more likely you are to fail again This is confounded by the fact that the closer one is to success the more frequent their attempts tend to be. The article doesn't indicate the research took this into consideration. e.g. Golfers takes shots more frequently the closer they are to a hole. But telling a golfer from the start to take a series of frequent short shots is bad advice.

keymon-o 7y ago
Golf match should not be defined as a consecutive failures before a potential success, but more of a set of strategic actions before a potential success. But yes, it is not defined what they took into account. e.g. Calling the same potential client hundred times in a row, can get you a restraining order only. If qualitative input and output of an attempt would be taken in account, quantity of an attempt would not really be that significant.
vannevar 7y ago
It is also confounded by the element of ruin, the ability of a gambler to stay in the game based on their stake. It's reasonable to suppose that some of the lag in re-try times (particularly with regard to startups) is attributable to raising money, which will be much easier for someone who starts out wealthy to begin with, even if they're not directly investing their own funds.

sqba 7y ago

There's hope yet for the Donald.

o09rdk 7y ago

I can't speak to venture funding or terrorism, but something seems really misleading to me about the NIH grant applications model. My sense is that a grant, like an academic paper, is often basically well-received or not. So prior to going into it, if you're familiar with the details of the particular grant, you can get some sense of interest or not. The grant may have even been solicited in a certain sense by a program officer or something, so it's quasi-invited. Those grants that are quasi-invited, or well-received, will basically involve polishing on subsequent revisions. Those that are totally unsolicited and not well-received it doesn't matter how much polishing you do often, it will not go anywhere. I think where this becomes relevant to the paper is that this often has very little to do with the process of the grant revisions. It says little about "how someone responds to failure", and everything about connections with the grant agency, program officers, and luck. All that stuff that happens before the grant is even submitted plays into it, and this paper kind of ignores that. You could say that it does speak to learning, in that you could ask "why doesn't the other scientist adjust their strategy?" To which I'd say, there are enormous pressures to submit anyway, and for someone who doesn't get good mentoring about what to target, or what agency to target, or whose research doesn't jive with the priorities of the division head at that time, or whatever, they might just not get it, and it might easily be over the 5 year period of the study. Basically, at least with NIH grants I think this study is really misleading and potentially harmful, because it kind of suggests failed applicants just aren't learning, when I think what's really happening is that you have a mixture of two groups, one of which can learn something, and the other of whom it just doesn't matter if they learn or not, because the outcome has kind of been preordained. That is, the process they predict from is an indicator of which mixture class the applicant belongs in, not the cause of the outcome.