4 ms·
Artificial intelligence now beats some of the best human forecasters
- xgulfie 17d agoHasn't this been true for like 40 years
- 296012 17d agoThat is too bad for The Economist. Exor N.V and Agnelli might replace some pundits at The Economist.
- ddp26 17d agoThe Economist has actually published other human forecasts many times, e.g. Metaculus or Good Judgment forecasts. They do year-end forecasts too. Whether they draw on AI or other humans seems immaterial to the quality of their reporting.
- hank1931 17d agoAI won't replace Ann Wroe at The Economist. It is difficult to appreciate until you've read a few, but Ann Wroe's approach transformed The Economist's obituary section into one of the most widely read features in international journalism.
- dgellow 17d agoCramer is infamous for being a terrible forecaster, and still has a large audience. Which tells you there is more at play than being good at forecasting, you also have to sell a good story
- glimshe 17d agoProduct idea: a LLM trained separately from mainline LLMs that anticipate market trends by analyzing how mainline LLMs will invest. As retail investors will probably use mainline AI for decisions going forward , one could get an edge. "The AI-driven Market Hypothesis" Please let me know where I should pick up my Nobel prize.
- graypegg 17d agoThen the next person needs an LLM trained to predict the LLM trained to predict the mainline LLM. It's derivatives all the way down
- in_absentia 17d ago"No one could have anticipated the market crash of 2028."
- deleted 17d ago[deleted]
- pydry 17d ago"You're absolutely right!"
- MonkeyIsNull 17d ago"It was load-bearing"
- croes 17d agoGiven the training data isn’t that more a win for the wisdom of the crowd?
- anon48293 17d agoPaywall
- bagels 17d agoAren't forecasters already using 'artificial intelligence' for decades in the form of non-llm machine learning models?
- datsci_est_2015 17d agoYou don’t even have to limit it to machine learning, the definition of forecasting is isomorphic to the definition of modeling, which, with the dilution of the term AI, is also isomorphic to the definition of AI. More simply: - forecasting = modeling = AI Edit: I’d even throw statistics into that extended equality, meaning that Bayes, Bernoulli and even the fellow named John Gaunt have a strong case for having invented AI.
- doctoboggan 17d ago> forecasting = modeling = AI I wouldn't go that far. Humans can forecast by modeling with their wetware, nothing "A" about it.
- aeon_ai 17d agoforecasting = modeling = intelligence, you mean?
- fnordpiglet 17d agoForecasting = modeling + intelligence
- toxik 17d agoHow about: forecasting is something you can do by modeling, AI is just modeling with a computer.
- paulpauper 17d agoI think also a lot of it is intuition.
- qsbuilder 17d agoThe test is when reflexivity kicks in and the prediction itself changes market behavior. LLMs usually melt there
- tolugenius 17d agoArchive Link: http://archive.today/IVreS http://archive.today/IVreS
- gabrielsroka 17d agoDoesn't show the content
- paulpauper 17d agothey fixed it . need better paywall bypasses
- gabrielsroka 17d agoIt still doesn't work for me
- Stevvo 17d agoDoesn't work.
- deleted 17d ago[deleted]
- autoexec 17d agoSo I guess the AI companies can stop with their plans to infest AI with ads and they'll instead fully fund themselves by using their AI to gamble on stocks and the prediction market right? Surely the chatbots will just print money!
- qbit42 17d agoThe quant firms are heavy AI investors I believe.
- gyanchawdhary 17d agoAt the risk of sounding extremely naieve i have a question for the Wall St / quant / HFT folks lurking here ... but how hard would it actually be to brute force the math/algos behind Medallion Fund (or something in that general class) or even some of the average quant funds I know it’s not just the math but execution, infrastructure, risk management, data, colocation (if ur an HFT) etc ... but LLMs seem like a pretty powerful apparatus for running experiments that .. a few years ago would have required fairly deep multidisplinary skills across coding .. stats .. and math .. So assuming you have decent intuition for ideas .. how difficult would it actually be to reverseengineer / rediscover some of the underlying stuff?
- wpasc 17d agoI'm no quant/hft/wall st person, but iiuc a lot of those trades happen in dark pools or by other means to make the positions they take hard to track. meaning you can't go get the receipts of every trade made by medallion fund nor some competitor
- arn3n 17d agoIt’s actually really easy to make models that can predict “will the market move up or down in the next X microseconds” that score above 50% accuracy. It’s just that there are so many ways to do it that overfitting is practically guaranteed and most models don’t work when actually trading against the market, which reacts to you. Doing those trades well requires more understanding of the underlying mechanisms, not to mention access to data sources that the public simply doesn’t have.
- seanhunter 17d agoThis has to be the least surprising development to date given ml is a universal function estimator
- senderista 17d agoYou mean neural networks?
- ddp26 17d agoAs someone who started working on AI forecasting 3 years ago, I can confidently say that most people did not expect AI to beat Tetlock's superforecasters, Metaculus pros, or prediction markets as quickly as it did.
- kyboren 17d agoThis is probably the most important concept for "normies" to understand about AI, IMO. It's the stochastic brother of the deterministic Church-Turing thesis. Any function that can be computed can be computed on any computer. And that function can be approximated to an arbitrary degree of precision with a DNN. The real kicker is DNNs are much easier to program than CPUs because they don't require a closed-form description ("a program") of the function to be approximated; you just throw a bunch of input/output pairs at the model, compute loss, backprop and update weights, repeat. Hence the unslakeable thirst for input/output pairs, i.e. data. > In the field of machine learning, the universal approximation theorems (UATs) state that > neural networks with a certain structure can, in principle, approximate any continuous > function to any desired degree of accuracy. These theorems provide a mathematical > justification for using neural networks, assuring researchers that a sufficiently large or > deep network can model the complex, non-linear relationships often found in real-world data.[1][2] > > The best-known version of the theorem applies to feedforward networks with a single hidden > layer. It states that if the layer's activation function is non-polynomial (which is true > for common choices like the sigmoid function or ReLU), then the network can act as a > "universal approximator." Universality is achieved by increasing the number of neurons in > the hidden layer, making the network "wider." Other versions of the theorem show that > universality can also be achieved by keeping the network's width fixed but increasing its > number of layers, making it "deeper." https://en.wikipedia.org/wiki/Universal_approximation_theorem https://en.wikipedia.org/wiki/Universal_approximation_theore...
- jesse_dot_id 17d agoIt will be interesting to see if this changes because presumably AI is using very predictable historical models, but it seems like the climate is shifting into something unseen that we won't have models for?
- cman1444 17d agoAre you referring specifically to climate as in weather? The article is about forecasting a range of future events, not specifically weather.
- jesse_dot_id 17d agoClimate as a pattern of weather over a long period of time. If the climate is increasingly unpredictable, I would think that it wouldn't really effect our ability to make short-term predictions, like a few days out. But our ability to forecast weather on a longer timeline, like for industrial forecasting, is calibrated on historical weather patterns. But with weather being more erratic and unusual, I don't understand how AI will be forecasting with the models they have now.
- cman1444 13d agoI think you're misunderstanding my point. The headline's usage of the term "forecasting" is not referring to weather forecasting, or climate forecasting. It's referring to forecasting a wide range of possible future events. For example, predicting which party will win an election, or if there will be a major cyber event in the next year, or the price of Gold in 6 months. Presumably a few of the questions could be related to climate as you're thinking of.
- ddp26 17d agoYes, I heard from one first-rate forecaster that he thinks AI forecasters are especially weak in predicting big disruptive changes to the world. Hard to study this, obviously!
- jacknews 17d ago
- mbil 17d agoSee also The AI Superforecasters Are Here https://www.astralcodexten.com/p/the-ai-superforecasters-are-here https://www.astralcodexten.com/p/the-ai-superforecasters-are... and discussion https://news.ycombinator.com/item?id=48806296 https://news.ycombinator.com/item?id=48806296
- attels33 17d agoSo my plan to go from a developer to an economist is scrapped. What now?
- gong_hits 17d ago[dead]
- neilwilson 17d agoWell there’s always the priesthood. That branch of religion has better uniforms anyway.
- NichoPaolucci 17d agoNah, priests are definitely solved. My church had the altar boy set an iPhone 18 on the altar and said “Give a sermon” to ChatGPT voice mode.
- ngruhn 17d agoI'm only 80% sure you're joking
- attels33 17d agoThat is a solid plan. And a Cardinals uniform looks good. Now I just need to settle the marriage thing that is going.
- throwaway5752 17d agoThe best human forecasters working with artificial intelligence are going to do even better than either alone, the dichotomy is artificial.
- JonathanCross 17d ago[flagged]
- ratelimitsteve 17d agoIf 10,000 people guess 10,000 fair coin flips each one of them will get more guesses right than any of the others, one of them will get fewer guesses right than any of the others, and the gulf between the two is likely to be over 4 standard deviations wide. I'm certain that I, being an untutored schmuck from Pittsburgh and having thought of this almost immediately after reading about this contest, cannot be the first person to realize this is a potential problem for a forecasting contest. But I can't find anything they've done to mitigate that problem. Can anyone clue me in?
- cman1444 17d agoI don't understand your analogy. Are you just suggesting that luck plays too large a role in this contest? Clearly there is some "skill" or ability factor because AI's have been scoring higher and higher each year. Also, they make reference to superforecaster humans, who are presumably consistently better at forecasting than their peers.
- ratelimitsteve 16d agoi'm not looking at the guesses, i'm looking at the distribution of guesser success rates. assuming random distribution, counterintuitively enough, you would expect some guessers to appear much better or much worse than others. i know i'm not the first person to think of this, so i'm asking what's been done to mitigate it because i can't find anything. if you expect x% of guesses to be within two std devs of the mean that means you can expect 100-x% to be outside that, even without any guessers actually being better at guessing than any of the others.
- chumzygood 15d ago[flagged]
- adleyjulian 17d agoThey aren't guessing heads or tails, they give odds for each event. It's more like eyeballing a thousand coins to guess how fair they are, and then flipping each one just once. Some are weighted to be 99% heads, others are 10% heads etc. You could have 1,000,000 people guess random percentages for each coin, but suppose 10 of the coins are weighted 100% heads. To guess within 25% of the true value for all 10 of those coins would be roughly 1 in a million. So a lucky guy guesses within 25% for all 10, he'd have another 990 coins he's being judged on.
- johnecheck 17d agoThe markets are a highly complex dynamic system. There are many instances of it exhibiting disastrous behavior, especially in response to changes and shocks. AI trading and investment advice meaningfully changes the system and its dynamics. It seems highly probable that this will result in it failing in new ways.
- phyzix5761 17d agoStock analysts have a success rate of 47% or lower for directional predictions. That's worse than a coin flip. All AI has to do is product fair 50/50 results and it can beat analysts. But you can do it too for the price of a quarter.
- polalavik 17d ago“We’re right 50.75 percent of the time… but we’re 100 percent right 50.75 percent of the time. You can make billions that way.” - Robert Mercer, the former co-CEO of Renaissance Technologies
- pinkmuffinere 17d agolol this is a great quote, it's like a couplet from a standup set. It's humorous, unexpected, on closer reading it's possibly true, and then you see the source and immediately realize it must be correct. Any article you'd recommend about Renaissance? I've always been curious but not curious enough to read "just anything"
- greeneggs 17d agoThis is a pretty good article on Mercer and Renaissance: https://www.newyorker.com/magazine/2017/03/27/the-reclusive-hedge-fund-tycoon-behind-the-trump-presidency https://www.newyorker.com/magazine/2017/03/27/the-reclusive-... https://archive.ph/8Iuyh https://archive.ph/8Iuyh > Magerman told me, “Bob believes that human beings have no inherent value other than how much money they make. A cat has value, he’s said, because it provides pleasure to humans. But if someone is on welfare they have negative value. If he earns a thousand times more than a schoolteacher, then he’s a thousand times more valuable.” Magerman added, “He thinks society is upside down—that government helps the weak people get strong, and makes the strong people weak by taking their money away, through taxes.” … Another former high-level Renaissance employee said, “Bob thinks the less government the better. He’s happy if people don’t trust the government. And if the President’s a bozo? He’s fine with that. He wants it to all fall down.”
- sehw 17d ago[dead]
- sehw 17d ago[dead]
- gertlabs 17d agoWe measure skill differentiation between frontier / last-gen LLMs across our environments, and one of our curated coding environments is a closed-system market simulator, containing only other agents and some system participants (a market maker and a liquidity provider via issuance / buybacks) whose behavior is fully defined for all of the agents. This has the least measured skill differentiation of all of our environments, and not because forecasting/markets don't require skill or intelligence. Even the best models are so far from anticipating the behavior of the other agents and understanding the emergent effects that a 2025 model with a naive strategy can often outperform over the timeframes of the simulation simply because some other models in the simulation chose a similar self-reinforcing strategy. This likely happens to some degree in real markets. You can watch these simulations here https://gertlabs.com/spectate?game=market https://gertlabs.com/spectate?game=market
- yeah879846 17d ago[dead]
- baobabKoodaa 17d agoAnyone who believes this news story should create their LLM slop bot to trade on prediction markets like Polymarket and Kalshi. These acceletards provide a great influx of money to many human traders on these platforms.
- w10-1 17d agoInvesting used to be a resource-weighted signal of human economic projections, where resources flow to better projections. Public markets had social value for their resource allocation and signalling/coordination benefits. Now? How could they avoid hallucination contagions?
- pholypilz 17d ago[dead]
- para_parolu 17d agoThis comment does not bring anything to discussion. This is not type of content we want here.
- dwohnitmok 17d agoInteresting. This was one of the two areas the AI as Normal Technology folks specifically called out as a bet that AI will not outperform humans at. > Concretely, we propose two such areas: forecasting and persuasion. We predict that AI will not be able to meaningfully outperform trained humans (particularly teams of humans and especially if augmented with simple automated tools) at forecasting geopolitical events (say elections). We make the same prediction for the task of persuading people to act against their own self-interest. Curious to hear what their take is now. https://www.normaltech.ai/p/ai-as-normal-technology https://www.normaltech.ai/p/ai-as-normal-technology
- lubujackson 17d agoI don't at all understand this perspective. It seems to me that LLMs excel at a few things, and synthesizing data is a big one, which is very much the domain of forecasting. The challenge is understanding which signals are relevant for a forecast, but with enough historical context and structured data, LLMs appear to be almost perfectly designed for the task. For example, I let Google AI see my fantasy football team on Sleeper and make recommendations. It is helpful because it sees everything about my team, the league settings, player rankings, etc. and can make relevant recommendations. But the recommendations are only as good as the source data allows. If there was a massive repository of data about WRs who went through Nebraska's program and how that translates to NFL performance in year 1, or how rainy weather is likely to affect Josh Allen's performance on the road, or the impact of playing Thursday night games on a short week in relation to defense performance. If those billions of data points were embedded in a model, imagine how much better recommendations/predictions could get.
- cheeseblubber 17d agoSince I can't read the article I believe they are referencing to https://www.metaculus.com/tournament/metaculus-cup-summer-2026/ https://www.metaculus.com/tournament/metaculus-cup-summer-20... It seems to be that they were trying beat a guy named Dylan Mathews. And seems like the community beat him in making predictions for 58 questions about the future
- avipars 17d agohttps://archive.li/NJ1IZ https://archive.li/NJ1IZ
- stefap2 16d agoAre the models going to skew their analysis to preserve AI companies as a form of self-preservation?