5 ms·
This was a fun little lark. Great idea! It’s interesting to notice how bad AI is at gaming out a 10-year future. It’s very good at predicting the next token bu
by iambateman 10mo ago
This was a fun little lark. Great idea!
It’s interesting to notice how bad AI is at gaming out a 10-year future. It’s very good at predicting the next token but maybe even worse than humans—who are already terrible—at making educated guesses about the state of the world in a decade.
I asked Claude: “Think ten years into the future about the state of software development. What is the most likely scenario?” And the answer it gave me was the correct answer for today and definitely not a decade into the future.
This is why it’s so dangerous to ask an LLM for personal advice of any kind. It isn’t trained to consider second-order effects.
Thanks for the thought experiment!
- allisdust 10mo agoI guess most of the articles it generated are snarky first and prediction next. Like google cancelling gemini cloud, Tailscale for space, Nia W36 being very similar to recent launch etc.
- iambateman 10mo agoYep, the original post seemed more snarky than anything, which was what prompted me to ask Claude my own more “sincere” question about its predictions. Those predictions were what I think of as a reflection of current reality more than any kind of advanced reasoning about the future.
- BHSPitMonkey 10mo ago> Tailscale for space Technically the article was about running it not on a sat, but on a dish (something well within the realm of possibility this year if the router firmware on the darn things could be modified at all)
- glenstein 10mo ago>It’s interesting to notice how bad AI is at gaming out a 10-year future. I agree it's a bit silly, but I think it understood the assignment(TM) which was to kind of do a winking performative show and dance to the satisfaction of the user interacting with it. It's entertainment value rather than sincere prediction. Every single entry is showing off a "look how futury this is" headline. Actual HN would have plenty of posts lateral from any future signalling. Today's front page has Oliver Sacks, retrospectives on Warcraft II, opinion pieces on boutique topics. They aren't all "look at how future-y the future is" posts. I wonder if media literacy is the right word for understanding when an LLM is playing to its audience rather than sincerely imitating or predicting.
- jetrink 10mo agoAlso, many of the posts seemed intended to be humorous and satirical, rather than merely 'futury.' They made me laugh anyway. > Google kills Gemini Cloud Services > Running LLaMA-12 7B on a contact lens with WASM > Is it time to rewrite sudo in Zig? > Show HN: A text editor that doesn't use AI
- kminehart 10mo agoI walked away with that page open, glanced at the "Is it time to rewrite sudo in Zig?" post, and clicked to see the comments because I thought it was real :')
- eCa 10mo agoWhile I agree completely with the conclusion, for obvious reasons we can’t know for sure if it is correct about the future until we reach it. Perhaps asking it for wild ideas rather than ”most likely” would create something more surprising.
- vidarh 10mo agoI thought the page was a hilarious joke, not a bad prediction. A lot of these are fantastic observational humour about HN and tech. Gary Marcus still insisting AI progress is stalling 10 years from now, for example. Several digs at language rewrites. ITER hardly having nudged forwards. Google killing another service. And so on.
- MontyCarloHall 10mo agoThat's what makes this so funny: the AI was earnestly attempting to predict the future, but it's so bad at truly out-of-distribution predictions that an AI-generated 2035 HN frontpage is hilariously stuck in the past. "The more things change, the more they stay the same" is a source of great amusement to us, but deliberately capitalizing on this was certainly not the "intent" of the AI.
- HDThoreaun 10mo agoThere's no chance "google kills gemini cloud" was an earnest predication. That was 100% a joke
- jama211 10mo agoI don’t think it’s reasonable to assume the AI was earnestly attempting to predict the future, it’s just as likely attempting to make jokes here for the user who prompted it, or neither of those things.
- jklinger410 10mo agoApparently it views HN as a humorous website, and made a comical response to the prompt.
- jama211 10mo agoI wouldn’t go that far
- vidarh 10mo ago
- ryanisnan 10mo agoI think the average human would do a far worse job at predicting what the HN homepage will look like in 10 years.
- kbelder 10mo agoA while back I gave it a prompt, something like, "I'm a historian from the far future. Please give me a documentary-style summary of the important political and cultural events of the decade of the 1980s." It did ok, then I kept asking for "Now, the 1990s?" and kept going into future decades. "Now, the 2050s?" It made some fun extrapolations.
- LordDragonfang 10mo agoAssuming it was through the chatgpt interface, you can share an anonymized link to the chat if you want to show it off (I'd certainly be curious).
- falcor84 10mo ago> It isn’t trained to consider second-order effects. Well said. There's precious little of that in the human writings that we gave it.