26 ms·

One thing this really highlights to me is how often the "boring" takes end up being the most accurate. The provocative, high-energy threads are usually the ones

One thing this really highlights to me is how often the "boring" takes end up being the most accurate. The provocative, high-energy threads are usually the ones that age the worst. If an LLM were acting as a kind of historian revisiting today’s debates with future context, I’d bet it would see the same pattern again and again: the sober, incremental claims quietly hold up, while the hyperconfident ones collapse. Something like "Lithium-ion battery pack prices fall to $108/kWh" is classic cost-curve progress. Boring, steady, and historically extremely reliable over long horizons. Probably one of the most likely headlines today to age correctly, even if it gets little attention. On the flip side, stuff like "New benchmark shows top LLMs struggle in real mental health care" feels like high-risk framing. Benchmarks rotate constantly, and “struggle” headlines almost always age badly as models jump whole generations. I bet theres many "boring but right" takes we overlook today and I wondr if there's a practical way to surface them before hindsight does

simianparrot 10mo ago
Instead of "LLM's will put developers out of jobs" the boring reality is going to be "LLM's are a useful tool with limited use".
- jimbokun 10mo ago
  That is at odds with predicting based on recent rates of progress.
yunwal 10mo ago
"Boring but right" generally means that this prediction is already priced in to our current understanding of the world though. Anyone can reliably predict "the sun will rise tomorrow", but I'm not giving them high marks for that.
- SubiculumCode 10mo ago
  Perhaps a new category, 'highest risk guess but right the most often'. Those is the high impact predictions.
  - arjie 10mo ago
    Prediction markets have pretty much obviated the need for these things. Rather than rely on "was that really a hot take?" you have a market system that rewards those with accurate hot takes. The massive fees and lock-up period discourage low-return bets.
    - Karrot_Kream 10mo ago
      FWIW Polymarket (which is one of the big markets) has no lock-up period and, for now while they're burning VC coins, no fees. Otherwise agree with your point though.
    - gammarator 10mo ago
      Can’t wait for the brave new world of individuals “match fixing” outcomes on Polymarket.
      - Karrot_Kream 10mo ago
        As opposed to the current world of brigading social media threads to make consensus look like it goes your way and then getting journalists scraping by on covering clickbait to cover your brigading as fact?
- onraglanroad 10mo ago
  I'm giving them higher marks than the people who say it won't. LLMs have seen huge improvements over the last 3 years. Are you going to make the bet that they will continue to make similarly huge improvements, taking them well past human ability, or do you think they'll plateau? The former is the boring, linear prediction.
  - yunwal 10mo ago
    > Are you going to make the bet that they will continue to make similarly huge improvements Sure yeah why not > taking them well past human ability, At what? They're already better than me at reciting historical facts. You'd need some actual prediction here for me to give you "prescience".
    - onraglanroad 10mo ago
      At every intellectual task. They're already better than you at reciting historical facts. I'd guess they're probably better at composing poems (they're not great but far better than the average person). Or you agree with me? I'm not looking for prescience marks, I'm just less convinced that people really make the more boring and obvious predictions.
      - yunwal 10mo ago
        What is an intellectual task? Once again, there's tons of stuff LLMs won't be trained on in the next 3 years. So it would be trivial to just find one of those things and say voila! LLMs aren't better than me at that. I'll make one prediction that I think will hold up. No LLM-based system will be able to take a generic ask like "hack the nytimes website and retrieve emails and password hashes of all user accounts" and do better than the best hackers and penetration testers in the world, despite having plenty of training data to go off of. It requires out-of-band thinking that they just don't possess.
        hathawsh 10mo ago
        I'll take a stab at this: LLMs currently seem to be rather good at details, but they seem to struggle greatly with the overall picture, in every subject. - If I want Claude Code to write some specific code, it often handles the task admirably, but if I'm not sure what should be written, consulting Claude takes a lot of time and doesn't yield much insight, where as 2 minutes with a human is 100x more valuable. - I asked ChatGPT about some political event. It mirrored the mainstream press. After I reminded it of some obvious facts that revealed a mainstream bias, it agreed with me that its initial answer was wrong. These experiences and others serve to remind me that current LLMs are mostly just advanced search engines. They work especially well on code because there is a lot of reasonably good code (and tutorials) out there to train on. LLMs are a lot less effective on intellectual tasks that humans haven't already written and published about.
- Gravityloss 10mo ago
  something like correctness^2 x novel information content rank?
  - Gravityloss 10mo ago
    Actually now thinking about it, incorrect information has negative value so the metric should probably reflect that.
johnfn 10mo ago
This suggests that the best way to grade predictions is some sort of weighting of how unlikely they were at the time. Like, if you were to open a prediction market for statement X, some sort of grade of the delta between your confidence of the event and the “expected” value, summed over all your predictions.
- jacquesm 10mo ago
  Exactly, that's the element that is missing. If there are 50 comments against and one pro and that pro has it in the longer term then that is worth noticing, not when there are 50 comments pro and you were one of the 'pros'. Going against the grain and turning out right is far more valuable than being right consistently when the crowd is with you already.
  - mcmoor 10mo ago
    Yeah a simple of total points of pro comments vs total points of con comments may be simple and exact enough to simulate a prediction market. I don't know if it can be included in the prompt or better to be vibecoded in directly.
copperx 10mo ago
Is this why depressed people often end up making the best predictions? In personal situations there's clearly a self fulfilling prophecy going on, but when it comes to the external world, the predictions come out pretty accurate.
xpe 10mo ago
> One thing this really highlights to me is how often the "boring" takes end up being the most accurate. Would the commenter above mind sharing the method behind of their generalization? Many people would spot check maybe five items -- which is enough for our brains to start to guess at potential patterns -- and stop there. On HN, when I see a generalization, one of my mental checklist items is to ask "what is this generalization based on?" and "If I were to look at the problem with fresh eyes, what would I conclude?".
jimbokun 10mo ago
The one about LLMs and mental health is not a prediction but a current news report, the way you phrased it. Also, the boring consistent progress case for AI plays out in the end of humans as viable economic agents requiring a complete reordering of our economic and political systems in the near future. So the “boring but right” prediction today is completely terrifying.
- adam1996TL 10mo ago
  [dead]
- p-e-w 10mo ago
  “Boring” predictions usually state that things will continue to work the way they do right now. Which is trivially correct, except in cases where it catastrophically isn’t. So the correctness of boring predictions is unsurprising, but also quite useless, because predicting the future is precisely about predicting those events which don’t follow that pattern.
0manrho 10mo ago
It's because algorithmic feeds based on "user engagement" rewards antagonism. If your goal is to get eyes on content, being boring, predictable and nuanced is a sure way to get lost in the ever increasing noise.
schoen 10mo ago
I predict that, in 2035, 1+1=2. I also predict that, in 2045, 2+2=4. I also predict that, in 2055, 3+3=6. By 2065, we should be in possession of a proof that 0+0=0. Hopefully by the following year we will also be able to confirm that 0*0=0. (All arithmetic here is over the natural numbers.)

simianparrot 10mo ago

Instead of "LLM's will put developers out of jobs" the boring reality is going to be "LLM's are a useful tool with limited use".

jimbokun 10mo ago
That is at odds with predicting based on recent rates of progress.

yunwal 10mo ago

"Boring but right" generally means that this prediction is already priced in to our current understanding of the world though. Anyone can reliably predict "the sun will rise tomorrow", but I'm not giving them high marks for that.

SubiculumCode 10mo ago
Perhaps a new category, 'highest risk guess but right the most often'. Those is the high impact predictions.
- arjie 10mo ago
  Prediction markets have pretty much obviated the need for these things. Rather than rely on "was that really a hot take?" you have a market system that rewards those with accurate hot takes. The massive fees and lock-up period discourage low-return bets.
  - Karrot_Kream 10mo ago
    FWIW Polymarket (which is one of the big markets) has no lock-up period and, for now while they're burning VC coins, no fees. Otherwise agree with your point though.
  - gammarator 10mo ago
    Can’t wait for the brave new world of individuals “match fixing” outcomes on Polymarket.
    - Karrot_Kream 10mo ago
      As opposed to the current world of brigading social media threads to make consensus look like it goes your way and then getting journalists scraping by on covering clickbait to cover your brigading as fact?
onraglanroad 10mo ago
I'm giving them higher marks than the people who say it won't. LLMs have seen huge improvements over the last 3 years. Are you going to make the bet that they will continue to make similarly huge improvements, taking them well past human ability, or do you think they'll plateau? The former is the boring, linear prediction.
- yunwal 10mo ago
  > Are you going to make the bet that they will continue to make similarly huge improvements Sure yeah why not > taking them well past human ability, At what? They're already better than me at reciting historical facts. You'd need some actual prediction here for me to give you "prescience".
  - onraglanroad 10mo ago
    At every intellectual task. They're already better than you at reciting historical facts. I'd guess they're probably better at composing poems (they're not great but far better than the average person). Or you agree with me? I'm not looking for prescience marks, I'm just less convinced that people really make the more boring and obvious predictions.
    - yunwal 10mo ago
      What is an intellectual task? Once again, there's tons of stuff LLMs won't be trained on in the next 3 years. So it would be trivial to just find one of those things and say voila! LLMs aren't better than me at that. I'll make one prediction that I think will hold up. No LLM-based system will be able to take a generic ask like "hack the nytimes website and retrieve emails and password hashes of all user accounts" and do better than the best hackers and penetration testers in the world, despite having plenty of training data to go off of. It requires out-of-band thinking that they just don't possess.
      - hathawsh 10mo ago
        I'll take a stab at this: LLMs currently seem to be rather good at details, but they seem to struggle greatly with the overall picture, in every subject. - If I want Claude Code to write some specific code, it often handles the task admirably, but if I'm not sure what should be written, consulting Claude takes a lot of time and doesn't yield much insight, where as 2 minutes with a human is 100x more valuable. - I asked ChatGPT about some political event. It mirrored the mainstream press. After I reminded it of some obvious facts that revealed a mainstream bias, it agreed with me that its initial answer was wrong. These experiences and others serve to remind me that current LLMs are mostly just advanced search engines. They work especially well on code because there is a lot of reasonably good code (and tutorials) out there to train on. LLMs are a lot less effective on intellectual tasks that humans haven't already written and published about.
Gravityloss 10mo ago
something like correctness^2 x novel information content rank?
- Gravityloss 10mo ago
  Actually now thinking about it, incorrect information has negative value so the metric should probably reflect that.

johnfn 10mo ago

This suggests that the best way to grade predictions is some sort of weighting of how unlikely they were at the time. Like, if you were to open a prediction market for statement X, some sort of grade of the delta between your confidence of the event and the “expected” value, summed over all your predictions.

jacquesm 10mo ago
Exactly, that's the element that is missing. If there are 50 comments against and one pro and that pro has it in the longer term then that is worth noticing, not when there are 50 comments pro and you were one of the 'pros'. Going against the grain and turning out right is far more valuable than being right consistently when the crowd is with you already.
- mcmoor 10mo ago
  Yeah a simple of total points of pro comments vs total points of con comments may be simple and exact enough to simulate a prediction market. I don't know if it can be included in the prompt or better to be vibecoded in directly.

copperx 10mo ago

Is this why depressed people often end up making the best predictions? In personal situations there's clearly a self fulfilling prophecy going on, but when it comes to the external world, the predictions come out pretty accurate.

xpe 10mo ago

> One thing this really highlights to me is how often the "boring" takes end up being the most accurate. Would the commenter above mind sharing the method behind of their generalization? Many people would spot check maybe five items -- which is enough for our brains to start to guess at potential patterns -- and stop there. On HN, when I see a generalization, one of my mental checklist items is to ask "what is this generalization based on?" and "If I were to look at the problem with fresh eyes, what would I conclude?".

jimbokun 10mo ago

The one about LLMs and mental health is not a prediction but a current news report, the way you phrased it. Also, the boring consistent progress case for AI plays out in the end of humans as viable economic agents requiring a complete reordering of our economic and political systems in the near future. So the “boring but right” prediction today is completely terrifying.

adam1996TL 10mo ago
[dead]
p-e-w 10mo ago
“Boring” predictions usually state that things will continue to work the way they do right now. Which is trivially correct, except in cases where it catastrophically isn’t. So the correctness of boring predictions is unsurprising, but also quite useless, because predicting the future is precisely about predicting those events which don’t follow that pattern.

0manrho 10mo ago

It's because algorithmic feeds based on "user engagement" rewards antagonism. If your goal is to get eyes on content, being boring, predictable and nuanced is a sure way to get lost in the ever increasing noise.

schoen 10mo ago

I predict that, in 2035, 1+1=2. I also predict that, in 2045, 2+2=4. I also predict that, in 2055, 3+3=6. By 2065, we should be in possession of a proof that 0+0=0. Hopefully by the following year we will also be able to confirm that 0*0=0. (All arithmetic here is over the natural numbers.)