3 ms·

They're producing the wisdom of the crowd, which is a real and highly accurate piece of information. It's quite difficult and expensive to produce information a

by tinco 3y ago

They're producing the wisdom of the crowd, which is a real and highly accurate piece of information. It's quite difficult and expensive to produce information as fast and reliable by other means. And they don't cost much, it's mostly money being moved around. edit: I interpreted it as asking wether prediction markets in general produce value. In this specific case I'm 100% with you, they're absolutely useless in predicting wether this finding is going to replicate or not. BTW probably 100% useless is going to be better than trusting a single reply in a HN thread. Even averaging out a group of replies on HN is going to be pretty bad, probably worse than averaging out a group of replies on Reddit. The idea of wisdom of the crowd is based on the idea that knowledge about a topic (both false and true) is roughly normally distributed (as many things are in nature), so the averaged result of a large group of answers is likely to be close to the real answer, as long as there are no external factors pushing the whole distribution left or right. Also, the final result is not going to be the answer if it's gonna replicate, but more the odds of it replicating (i.e. the odds of a paper like this being legit). The odds could be 1 in a million, and it still wouldn't affect the reality of LK-99 being super conductive or not.

MLH6ft1 3y ago
"They're producing the wisdom of the crowd" Yeah we saw how wise was the crowd's wisdom with crypto.
- Regnore 3y ago
  This is correct - the overwhelming majority of people did not get involved with crypto. Even for people and companies who did most put a fraction of their money into it.
hutzlibu 3y ago
"They're producing the wisdom of the crowd, which is a real and highly accurate piece of information." I have strong doubts, that the wisdom of the crowd here is competent in judging whether a revolutionary new superconductor is real, or not.
- jonmumm 3y ago
  what's an alternative that is better?
civilitty 3y ago
There is literally zero wisdom in the crowd about a brand new just discovered material that’s only ever been produced by one small group by definition. This market fetishism is out of control.
- naasking 3y ago
  That's not correct. Condensed matter physicists will have a good handle on how plausible this is (but not certain). Other people will vote randomly so their votes cancel out, effectively leaving the final result as biased by the expert opinions. That's how the wisdom of the crowd works.
discreteevent 3y ago
The opinion of a crowd is "real and highly accurate"? The opinion of crowds is frequently completely disconnected from reality. Crowds are often an amplifier of individual delusion. As for accuracy, the only thing the opinion of a crowd is accurate about is the opinion of that particular crowd (not even "the crowd" - look at election polling)
- naasking 3y ago
  The wisdom of the crowds works given a large and diverse sample of independent predictors. People who don't know anything about a topic will vote randomly so their votes effectively cancel each other out, but people who know more about a particular topic will be biased towards correct answers.
nonethewiser 3y ago
So practically speaking, what can you do with the fact that X% of fans (because people betting are enthusiasts) think LK-99 will reproduce and Y% think it wont?
- tinco 3y ago
  You assume all the people betting are enthusiasts. The theory of prediction markets is that rational actors in the market will recognise that a portion of the betters is overhyped and adjust their bets to make use of their irrational behaviour. If the rational actors are actually effective at making such adjustments I don't know, I bet there's statistics out there on how well prediction markets correlate with reality. In any case, even if the market was perfect, it wouldn't tell us if LK-99 would reproduce, which I guess is the meat of your question. It would just tell us how likely it is that an experimental result made under those specific circumstances would reproduce. And what you could do with that information depends on what your answer to the question: "How would I be affected if LK-99 would reproduce?" would be. If you're a big energy business leader, and you want to filter what topics to spend your valuable time on maybe you could set a rule that you only want spend time reading scientific papers that have >10% odds of being legit. More realistically though, I think things like prediction markets are mostly useful to traders who are trying to arbitrage things like resource markets. What's the price of copper going to do when this turns out to be true? You could adjust your futures based on that.
gilleain 3y ago
Have you heard the one about the Emperor of China's nose? https://imaginatorium.org/stuff/nose.htm https://imaginatorium.org/stuff/nose.htm Basically, making an average of a large number of estimates of an unknown value will (of course) fail if most/none of the estimators have any idea of the actual value being estimated.
- ethbr0 3y ago
  > most/none of the estimators have any idea of the actual value being estimated. A subtle distinction is who is allowed to participate in a prediction market. "Everyone with $1" is a terrible answer, and produces the bad results people are pointing to. Financial markets avoid this because of their scale, where there's enough smart money to (usually) punish stupid money. Absent that scale, it's just stupid money muddling the decisions of smart money. Prediction markets with a knowledge barrier to entry would produce better results.
  - gilleain 3y ago
    How would you construct such a knowledge barrier? Another prediction market? Also, suggesting that there is such a thing as 'smart' money - presumably due to having more of it? - is amusing. As pointed out elsewhere in this discussion, there has been a lot of smart money acting particularly dumb over the last few years.
    - ethbr0 3y ago
      A "Do you understand what superconductivity is?" or "Do you have a physics or engineering degree?" barrier? And the relevant question isn't whether 'smart' money does dumb things: it's whether 'smart' money does dumb things less frequently than a random sample of money. No one is an oracle, and there are absolutely outlier events that specifically confound experts, but I can't believe that increased expertise is uncorrelated with increased accuracy.
      - kritiko 3y ago
        Tetlock’s Superforecasters performed better than experts, though: >In the Good Judgment Project, "the top forecasters... performed about 30 percent better than the average for intelligence community analysts who could read intercepts and other secret data"
throw0101b 3y ago
> They're producing the wisdom of the crowd, which is a real and highly accurate piece of information. Unless the group of people is not a crowd but rather a mob.
Ar-Curunir 3y ago
The opinion of a crowd is generally useless in highly technical matters. The people betting on this stuff generally do not have the background to evaluate any claims appropriately, and just react to what other people (who they believe to be more informed) are saying.

by tinco 3y ago

MLH6ft1 3y ago

"They're producing the wisdom of the crowd" Yeah we saw how wise was the crowd's wisdom with crypto.

Regnore 3y ago
This is correct - the overwhelming majority of people did not get involved with crypto. Even for people and companies who did most put a fraction of their money into it.

hutzlibu 3y ago

"They're producing the wisdom of the crowd, which is a real and highly accurate piece of information." I have strong doubts, that the wisdom of the crowd here is competent in judging whether a revolutionary new superconductor is real, or not.

jonmumm 3y ago
what's an alternative that is better?

civilitty 3y ago

There is literally zero wisdom in the crowd about a brand new just discovered material that’s only ever been produced by one small group by definition. This market fetishism is out of control.

naasking 3y ago
That's not correct. Condensed matter physicists will have a good handle on how plausible this is (but not certain). Other people will vote randomly so their votes cancel out, effectively leaving the final result as biased by the expert opinions. That's how the wisdom of the crowd works.

discreteevent 3y ago

The opinion of a crowd is "real and highly accurate"? The opinion of crowds is frequently completely disconnected from reality. Crowds are often an amplifier of individual delusion. As for accuracy, the only thing the opinion of a crowd is accurate about is the opinion of that particular crowd (not even "the crowd" - look at election polling)

naasking 3y ago
The wisdom of the crowds works given a large and diverse sample of independent predictors. People who don't know anything about a topic will vote randomly so their votes effectively cancel each other out, but people who know more about a particular topic will be biased towards correct answers.

nonethewiser 3y ago

So practically speaking, what can you do with the fact that X% of fans (because people betting are enthusiasts) think LK-99 will reproduce and Y% think it wont?

tinco 3y ago
You assume all the people betting are enthusiasts. The theory of prediction markets is that rational actors in the market will recognise that a portion of the betters is overhyped and adjust their bets to make use of their irrational behaviour. If the rational actors are actually effective at making such adjustments I don't know, I bet there's statistics out there on how well prediction markets correlate with reality. In any case, even if the market was perfect, it wouldn't tell us if LK-99 would reproduce, which I guess is the meat of your question. It would just tell us how likely it is that an experimental result made under those specific circumstances would reproduce. And what you could do with that information depends on what your answer to the question: "How would I be affected if LK-99 would reproduce?" would be. If you're a big energy business leader, and you want to filter what topics to spend your valuable time on maybe you could set a rule that you only want spend time reading scientific papers that have >10% odds of being legit. More realistically though, I think things like prediction markets are mostly useful to traders who are trying to arbitrage things like resource markets. What's the price of copper going to do when this turns out to be true? You could adjust your futures based on that.

gilleain 3y ago

Have you heard the one about the Emperor of China's nose? https://imaginatorium.org/stuff/nose.htm https://imaginatorium.org/stuff/nose.htm Basically, making an average of a large number of estimates of an unknown value will (of course) fail if most/none of the estimators have any idea of the actual value being estimated.

ethbr0 3y ago
> most/none of the estimators have any idea of the actual value being estimated. A subtle distinction is who is allowed to participate in a prediction market. "Everyone with $1" is a terrible answer, and produces the bad results people are pointing to. Financial markets avoid this because of their scale, where there's enough smart money to (usually) punish stupid money. Absent that scale, it's just stupid money muddling the decisions of smart money. Prediction markets with a knowledge barrier to entry would produce better results.
- gilleain 3y ago
  How would you construct such a knowledge barrier? Another prediction market? Also, suggesting that there is such a thing as 'smart' money - presumably due to having more of it? - is amusing. As pointed out elsewhere in this discussion, there has been a lot of smart money acting particularly dumb over the last few years.
  - ethbr0 3y ago
    A "Do you understand what superconductivity is?" or "Do you have a physics or engineering degree?" barrier? And the relevant question isn't whether 'smart' money does dumb things: it's whether 'smart' money does dumb things less frequently than a random sample of money. No one is an oracle, and there are absolutely outlier events that specifically confound experts, but I can't believe that increased expertise is uncorrelated with increased accuracy.
    - kritiko 3y ago
      Tetlock’s Superforecasters performed better than experts, though: >In the Good Judgment Project, "the top forecasters... performed about 30 percent better than the average for intelligence community analysts who could read intercepts and other secret data"

throw0101b 3y ago

> They're producing the wisdom of the crowd, which is a real and highly accurate piece of information. Unless the group of people is not a crowd but rather a mob.

Ar-Curunir 3y ago

The opinion of a crowd is generally useless in highly technical matters. The people betting on this stuff generally do not have the background to evaluate any claims appropriately, and just react to what other people (who they believe to be more informed) are saying.