3 ms·

Great writeup. I think prediction markets will break through when they figure out how to capitalize not only on collective judgment, but super-forecasting. As

Great writeup. I think prediction markets will break through when they figure out how to capitalize not only on collective judgment, but super-forecasting. As currently formulated, prediction market outputs are just a fancy opinion poll, where participants have some incentive for accuracy. To rise above the simple wisdom of the crowds, you would want to identify the subset of market participants that are constantly beating the market (because they have a more accurate mental model of the world). I think this necessitates both 1) long term tracking of bets and 2) likely withholding individual positions from the market to prevent follower effects. Similar to the title article, this raises the question of who the ultimate customer is prediction market is. Individuals can be incentivized to bet by winnings, but who else is the customer for aggregated data? I wonder about the extent to which current prediction markets have internal outputs and derivative statistics, and what they might do with it. If polymarket or similar companies put Trump vs Harris at 55-45, do they have internal statistics that that put the race at 80-20% among their most accurate betters? Was this data for sale?

JumpCrisscross 2y ago
There is a conflict of interest between the information prerogative, which requires facilitating the law of large numbers and factors stability, and the profitability of gambling, which favours encouraging larger bets and volatility.
- s1artibartfast 2y ago
  My point of curiosity is if these prerogatives can (or have been) reconciled. To that end, I'm not sure that these factors are mutually exclusive and would like to hear more of your thoughts. If I understand you correctly, I would think that a market could have both volume and depth. With respect to volatility-stability, what do you see as the drivers there? Is it that gamblers would need a significant upside to drive betting? Is this solved by the size of the bet? I suppose there is an internal conflict. If the line of a bet is 49-51% on a binary outcome, the risk of ruin is high, and the upside is low. You would need to aggregate outcomes over many distinct events to mitigate. I suppose this could hang on ability/inclination of professional forecasters to research and take several positions.
  - JumpCrisscross 2y ago
    > With respect to volatility-stability, what do you see as the drivers there? Given two prediction markets, one which varies wildly and one which is stable, the former will attract more users. Particularly the most profitable ones. Even if the underlying odds are unchanging, the volatile market is more “fit.”
    - s1artibartfast 2y ago
      I understand the claim, just not the reasoning for why that is the case. Is this driven by human psychology or economics of return? Why is a volatile market more fit?
      - JumpCrisscross 2y ago
        > Why is a volatile market more fit? More people get to feel like winners for longer [1]. And reward uncertainty makes gambling more additive [2]. For purposes of information discovery, volatility is bad. But for purposes of gambling, volatility is good. Running an information-discovery (or financial) platform is less profitable than running a gambling platform. Herego, operators will optimise their prediction markets for gamblers. [1] https://pmc.ncbi.nlm.nih.gov/articles/PMC10562822/ https://pmc.ncbi.nlm.nih.gov/articles/PMC10562822/ [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC3845016/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3845016/
        s1artibartfast 2y ago
        Thanks for clarifying. That line of thinking doesnt mean that a discovery planform can't exist. Even if one grants that gambling is more profitable than prediction, markets segment, saturate, and specialize all the time.
        JumpCrisscross 2y ago
        > Even if one grants that gambling is more profitable than prediction, markets segment, saturate, and specialize all the time It's difficult to see the niche for the academic market. More profit to the gambling platforms means more money for R&D, customer service, user retention and marketing. That means more liquidity. Gamblers means dumb money, which in turn attracts the smart money: if you're commissioning private polling to place more informed bets [1], you want to place a big bet against dumb money. [1] https://www.bloomberg.com/opinion/articles/2024-11-07/prediction-markets-are-a-thing-now?srnd=undefined https://www.bloomberg.com/opinion/articles/2024-11-07/predic...
ddp26 2y ago
(Author here). > To rise above the simple wisdom of the crowds, you would want to identify the subset of market participants that are constantly beating the market (because they have a more accurate mental model of the world). Identifying, and then working with, the top traders at Google (including one card-carrying superforecaster) was a great joy. And yes, they're sitting on some great data, on what the employee crowd tends to get right and wrong, who individually is good at forecasting what. Though one complication is that being a great trader is not the same as being a great forecaster.
- coderintherye 2y ago
  Great piece, if you write more would love to read your thoughts on what makes a great trader vs. what makes a great forecaster.
  - xrd 2y ago
    I second that! To the author: if you have a personal blog, can you post that here?
    - ddp26 2y ago
      That's very kind of you to ask. My personal blog is defunct (for now!). But some of my recent writings can be found on the research page [1] of my startup, FutureSearch. We're building an AI that can forecast accurately. We've written some pieces on topics like the problems with using crowds to forecast, and contesting recent papers' claims of good forecasts coming from simple LLMs. [1] https://futuresearch.ai/reports https://futuresearch.ai/reports
      - xrd 2y ago
        Thanks, this sounds so interesting and timely. There are some interesting jupyter to blog tools like quarto.org. Or, my Svelte based blogging tool: svekyll.com (I use it to blog about AI/ML because Svelte is the best visualization front end tool). It's a great time for you to start blogging again!
smokedetector1 2y ago
The problem is that if a prediction market exposed the fact that 80% of their most accurate betters are betting for YES, then wouldn't that skew the rest of the market? If I'm on there and I think it's about 50/50 chances, but I see that the super-betters are mostly saying YES, I'm probably going to vote YES too.
- s1artibartfast 2y ago
  I flagged that as implication #2. They would have to keep the market participants in the dark about it, and find a confidential use for the private forecast. If polymarket had an internal 80-20 model based on super-forecasters, they wouldn't disclose that, but they might have paying customers for it outside the prediction market itself. For example, stock traders might pay for the private model. If I could pay the NYSE for real time trading info on the buy/sell limits from warren buffet and other whales, I would.
ralegh 2y ago
Markets are what you described. Participants that regularly beat the market are rewarded with more money (and confidence) which lets them bet larger size and have more impact on the market. Uninformed bets should wash out as noise, and informed bettors should reverse uninformed moves so long as they are profitable.
- s1artibartfast 2y ago
  The distinction that I am drawing is the ability use the betting market generate forecasts with greater accuracy than the market itself. The stock market analogy would be the predictions you could make as an individual if you knew the internal limits and assessments of the best trading firms, and not just current market prices. If I could pay the NYSE for real time trading info on the buy/sell limits from warren buffet and other whales, I would.
redhed 2y ago
Many sports gambling companies do this, weighing the bets of "sharps" (people who are more accurate than the average) heavier than other bets. A good example of this was Mayweather vs McGregor where a lot of sharps were betting on Mayweather whereas the public was betting more on McGregor. Even with about 80% of people betting on McGregor, the house still had Mayweather as a favorite.
- s1artibartfast 2y ago
  Thats a different phenomenon than what I was discussing, but a really interesting one. Nate Silver discussed the topic with Tyler Cowen and said many platforms ban sharps, but then look to other platforms that allow them to benchmark their own odds. I think the analogous situation to what I was proposing would be if a platform had open betting and organic odds, but sold the sharp betting data to 3rd parties.

JumpCrisscross 2y ago

There is a conflict of interest between the information prerogative, which requires facilitating the law of large numbers and factors stability, and the profitability of gambling, which favours encouraging larger bets and volatility.

s1artibartfast 2y ago
My point of curiosity is if these prerogatives can (or have been) reconciled. To that end, I'm not sure that these factors are mutually exclusive and would like to hear more of your thoughts. If I understand you correctly, I would think that a market could have both volume and depth. With respect to volatility-stability, what do you see as the drivers there? Is it that gamblers would need a significant upside to drive betting? Is this solved by the size of the bet? I suppose there is an internal conflict. If the line of a bet is 49-51% on a binary outcome, the risk of ruin is high, and the upside is low. You would need to aggregate outcomes over many distinct events to mitigate. I suppose this could hang on ability/inclination of professional forecasters to research and take several positions.
- JumpCrisscross 2y ago
  > With respect to volatility-stability, what do you see as the drivers there? Given two prediction markets, one which varies wildly and one which is stable, the former will attract more users. Particularly the most profitable ones. Even if the underlying odds are unchanging, the volatile market is more “fit.”
  - s1artibartfast 2y ago
    I understand the claim, just not the reasoning for why that is the case. Is this driven by human psychology or economics of return? Why is a volatile market more fit?
    - JumpCrisscross 2y ago
      > Why is a volatile market more fit? More people get to feel like winners for longer [1]. And reward uncertainty makes gambling more additive [2]. For purposes of information discovery, volatility is bad. But for purposes of gambling, volatility is good. Running an information-discovery (or financial) platform is less profitable than running a gambling platform. Herego, operators will optimise their prediction markets for gamblers. [1] https://pmc.ncbi.nlm.nih.gov/articles/PMC10562822/ https://pmc.ncbi.nlm.nih.gov/articles/PMC10562822/ [2] https://pmc.ncbi.nlm.nih.gov/articles/PMC3845016/ https://pmc.ncbi.nlm.nih.gov/articles/PMC3845016/
      - s1artibartfast 2y ago
        Thanks for clarifying. That line of thinking doesnt mean that a discovery planform can't exist. Even if one grants that gambling is more profitable than prediction, markets segment, saturate, and specialize all the time.
        JumpCrisscross 2y ago
        > Even if one grants that gambling is more profitable than prediction, markets segment, saturate, and specialize all the time It's difficult to see the niche for the academic market. More profit to the gambling platforms means more money for R&D, customer service, user retention and marketing. That means more liquidity. Gamblers means dumb money, which in turn attracts the smart money: if you're commissioning private polling to place more informed bets [1], you want to place a big bet against dumb money. [1] https://www.bloomberg.com/opinion/articles/2024-11-07/prediction-markets-are-a-thing-now?srnd=undefined https://www.bloomberg.com/opinion/articles/2024-11-07/predic...

ddp26 2y ago

(Author here). > To rise above the simple wisdom of the crowds, you would want to identify the subset of market participants that are constantly beating the market (because they have a more accurate mental model of the world). Identifying, and then working with, the top traders at Google (including one card-carrying superforecaster) was a great joy. And yes, they're sitting on some great data, on what the employee crowd tends to get right and wrong, who individually is good at forecasting what. Though one complication is that being a great trader is not the same as being a great forecaster.

coderintherye 2y ago
Great piece, if you write more would love to read your thoughts on what makes a great trader vs. what makes a great forecaster.
- xrd 2y ago
  I second that! To the author: if you have a personal blog, can you post that here?
  - ddp26 2y ago
    That's very kind of you to ask. My personal blog is defunct (for now!). But some of my recent writings can be found on the research page [1] of my startup, FutureSearch. We're building an AI that can forecast accurately. We've written some pieces on topics like the problems with using crowds to forecast, and contesting recent papers' claims of good forecasts coming from simple LLMs. [1] https://futuresearch.ai/reports https://futuresearch.ai/reports
    - xrd 2y ago
      Thanks, this sounds so interesting and timely. There are some interesting jupyter to blog tools like quarto.org. Or, my Svelte based blogging tool: svekyll.com (I use it to blog about AI/ML because Svelte is the best visualization front end tool). It's a great time for you to start blogging again!

smokedetector1 2y ago

The problem is that if a prediction market exposed the fact that 80% of their most accurate betters are betting for YES, then wouldn't that skew the rest of the market? If I'm on there and I think it's about 50/50 chances, but I see that the super-betters are mostly saying YES, I'm probably going to vote YES too.

s1artibartfast 2y ago
I flagged that as implication #2. They would have to keep the market participants in the dark about it, and find a confidential use for the private forecast. If polymarket had an internal 80-20 model based on super-forecasters, they wouldn't disclose that, but they might have paying customers for it outside the prediction market itself. For example, stock traders might pay for the private model. If I could pay the NYSE for real time trading info on the buy/sell limits from warren buffet and other whales, I would.

ralegh 2y ago

Markets are what you described. Participants that regularly beat the market are rewarded with more money (and confidence) which lets them bet larger size and have more impact on the market. Uninformed bets should wash out as noise, and informed bettors should reverse uninformed moves so long as they are profitable.

s1artibartfast 2y ago
The distinction that I am drawing is the ability use the betting market generate forecasts with greater accuracy than the market itself. The stock market analogy would be the predictions you could make as an individual if you knew the internal limits and assessments of the best trading firms, and not just current market prices. If I could pay the NYSE for real time trading info on the buy/sell limits from warren buffet and other whales, I would.

redhed 2y ago

Many sports gambling companies do this, weighing the bets of "sharps" (people who are more accurate than the average) heavier than other bets. A good example of this was Mayweather vs McGregor where a lot of sharps were betting on Mayweather whereas the public was betting more on McGregor. Even with about 80% of people betting on McGregor, the house still had Mayweather as a favorite.

s1artibartfast 2y ago
Thats a different phenomenon than what I was discussing, but a really interesting one. Nate Silver discussed the topic with Tyler Cowen and said many platforms ban sharps, but then look to other platforms that allow them to benchmark their own odds. I think the analogous situation to what I was proposing would be if a platform had open betting and organic odds, but sold the sharp betting data to 3rd parties.