7 ms·

Why Anthropic's Claude still hasn't beaten Pokémon

53 points by Workaccount2 2y ago

barotalomey 2y ago
I very much feel nostalgic now, thinking about the time when Twitch plays pokemon was something we all enjoyed.
roxolotl 2y ago
I wonder how much help probably having tons of guides in its training material is. This really does feel like the sort of test that’s actually interesting for evaluating how general these AI can be especially compared to the existing tests.
- dezgeg 2y ago
  It's also a detriment to some extent, as Claude's now believing things that apply only to later generations of the game (doesn't help that those beliefs would apply to the FireRed remake). Also it made Claude try to perform quests out of order and waste lots of time trying to do things he could only do later in the game (the thirsty guards blocking some routes-sidequest).
  - morcus 2y ago
    It's interesting to me though because I can see a human getting confused similarly. IIRC when I played Pokemon a while back someone told me you could get a valuable item (Leftovers) by checking the garbage can on one ship in one game, and as a result when I played I checked every garbage can in case there was a hidden valuable item in there. I'm not even sure if Leftovers was obtained in the same way in the game I was playing.
rsynnott 2y ago
> OpenAI is quietly seeding expectations for a "PhD-level" AI agent that could operate autonomously at the level of a "high-income knowledge worker" in the near future. Elon Musk says that "we'll have AI smarter than any one human probably" by the end of 2025. Anthropic CEO Dario Amodei thinks it might take a bit longer but similarly says it's plausible that AI will be "better than humans at almost everything" by the end of 2027. I am baffled that anyone is still buying this complete nonsense. Like, come on.
- readthenotes1 2y ago
  Will AI be better at hyping AI than certain humans? Seems improbable...
- d3nj4l 2y ago
  Yeah, the people with a vested interest in AI say AI is good, news at 9.
  - rsynnott 2y ago
    I mean I think this is actually kind of abnormal; you do not get this level of absolute nonsense promises in most industries, including most parts of the tech industry.
- zurfer 2y ago
  It's an interesting world where both things can be true: 1 AI surpassing humans even in PhD level tasks in a couple of years and 2 current AI not being able to successfully play a childrens game. One way to square that is that AIs have a big memory advantage that allows them to store and access information much faster than humans. However they still lack visual understanding and some forms of reasoning and live learning. Pokemon doesn't require PhD level knowledge and rather just all the other qualities that make up intelligence. Now if the big labs make progress on visual reasoning etc. the moment AI can solve Pokemon it might be able to solve everything else as well.
  - nprateem 2y ago
    Yeah right. It can't draw on enough of its training data to be useful. That's why it gets stuck going down one particular alley without considering alternatives. See the whole using glue to keep pepperoni on your pizza. Even looping a la Claude extended doesn't help. It still keeps going in circles.
    - alabastervlog 2y ago
      There's so incredibly much knowledge in non-digitized documents and trapped in human brains, yet. Plus in various proprietary databases. It's 2025 and that stuff hasn't already made it onto the "repository of all human knowledge" (LO fucking L) World Wide Web still, so... I wouldn't hold my breath.
skybrian 2y ago
Interesting analysis. Would anyone care to bet on whether Claude will beat Pokémon in a couple of years? It seems like a good question for a prediction market. My guess is that image understanding will improve.
- amelius 2y ago
  If that bet existed, then who decides which benchmark is used?
- cratermoon 2y ago
  > My guess is that image understanding will improve. The "it will get better" assertion fails on what's known as the "first step fallacy". The analogy I like to use is the idea of a ladder to the moon. We can build tall self-supporting ladders, and ladder technology has advanced considerably since they were first used at least 10,000 years ago. More recently, in 1862, John H. Balsley made the step ladder safer by changing the rounded rungs to flat steps. Henry Quackenbush patented the extension ladder in 1867. Ladder technology continues to progress, will we someday build a ladder tall enough to climb to the moon? Do you see the fallacy? https://thebullshitmachines.com/lesson-16-the-first-step-fallacy/index.html https://thebullshitmachines.com/lesson-16-the-first-step-fal...
  - someothherguyy 2y ago
    No one is attempting to build ladders to the moon though?
    - namaria 2y ago
      No, but some people claim that compressing text into vector spaces and using the patterns to extend bits of text will somehow do more than extend texts in a sometimes plausible, sometimes nonsensical way if we just: add enough data, and process it for long enough.
      - someothherguyy 2y ago
0xbadcafebee 2y ago
Anyone who has used AI recently knows how much bullshit the claims are about replacing any human for virtually any task. AI still can't do super advanced things like tell me the correct height of my truck from the manufacturer's tech specs PDF that it has in its databank. Even when I tell it what the correct height is, it'll randomly change the height throughout a session. I have to constantly correct it because thankfully I know enough about a given subject that I know it's bullshitting. Once I correct it, it suddenly admits, oh yeah, actually it was this other thing, here's the right info. It's an amazing search engine, and has really cool suggestions/ideas. It's certainly very useful. But replace a human? Get real. All these stocks are going to tank once the media starts running honest articles instead of PR fluff.
- nprateem 2y ago
  Exactly. It was a stroke of genius to popularise the word 'hallucinate' instead of 'bullshit' or 'wrong'. Since it has no fundamental understanding of anything, it's either a sycophant or arrogant idiot. When I told chatgpt it didn't understand something it obnoxiously told me it understood perfectly. When I told it why, it did a 180 and carried on where I'd left off. I don't use it any more.
- Henchman21 2y ago
  Do you have an ETA for when the media will start running honest articles instead of PR fluff? Asking for the entire western world…
  - deleted 2y ago
    [deleted]
  - cratermoon 2y ago
    About the time they stop depending on targeted advertising revenue.
- alabastervlog 2y ago
  The things feel closer to using a fine-grained search algo with some randomness and a fairly large corpus, than to interacting with something intelligent. And if you read how they work... that's because that's exactly what they are. There's no thinking going on. When using them, that they've been programmed and prompted to have some kind of tone or personality and to fit within some kind of parameters of "behavior" as if they're a real being reminds me of Flash-based site navigation in the early '00s: flashy bullshit that's impressive for about one minute and then just annoying and inconvenient forever. As for programming with them, writing the prompts feels more like just another kind of programming than instructing a human. I'm skeptical this entire approach is more than one small part of what might become AGI, given several more somewhat-unrelated breakthroughs, including probably in hardware. Like a lot of us, I've gotten sucked into building products with these things because every damn company on the planet, tech or not, has decided to do that for usually-very-bad reasons, and I'm a little worried this is going to crash so hard that having been involved in it at all will be a (minor) black mark on my résumé.
blainm 2y ago
So when it really struggled to get around (kept just walking into obstacles), they gave Claude the ability to navigate by adding pathfinding and awareness of its position and map ID. However, it still struggles, particularly in open-ended areas. This suggests a fundamental issue beyond just navigation. While accessing more RAM data or using external tools using said data could improve consistency or get it further, that approach reduces the extent to which Claude is independently playing and reasoning. A more effective solution would enhance its decision-making without relying on direct RAM access or any kind of fine tuning. I'm sure it's possible. There has to be a better approach, and also in a way that's not relying on reading values from RAM or any kind of fine tuning.
- OgsyedIE 2y ago
  It can't do a good job of reasoning about higher-level abstractions in its long term memory without making poor decisions about which memory items to retain and which to forget. Would a mixture-of-experts paradigm, where each expert weights the value of short-term memories differently to the weight of long-term memoried, do noticeably better at overcoming that one category of roadblocks?
- rtkwe 2y ago
  Seems like the 200k context window is a huge issue and it's summarization deletes important information leading it to revisit solved areas even when it's working properly or simply forget things it needs to progress.
Recursing 2y ago
This Sunday there's going to be a hackaton at https://lu.ma/poke https://lu.ma/poke to improve the Pokemon-playing agent. I think most hackatons don't achieve much, but maybe someone from HN can go improve the scaffolding and save Claude
kibwen 2y ago
> Claude will readily notice when the game tells it that an attack from an electric-type Pokémon is “not very effective” against a rock-type opponent, for instance. Claude will then squirrel that factoid away in a massive written knowledge base for future reference later in the run. But these models already know all this information??? Surely it's ingested Bulbapedia, along with a hundred zillion terabytes of every other Pokemon resource on the internet, so why does it need to squirrel this information away? What's the point of ruining the internet with all this damn parasitic crawling if the models can't even recall basic facts like "thunderbolt is an electric-type move", "geodude is a rock-type pokemon", "electric-type moves are ineffective against rock-type pokemon"?
- gbear605 2y ago
  Claude definitely does know it (I just checked). Plausibly they’re giving that as an example of other things that it actually needs to save, but yeah, it’s a bad example.
  - stonemetal12 2y ago
    It has trained on multiple guides and walkthroughs and whatnot for the game from the internet. Theoretically it knows the game inside and out, it can recall all of that if you ask about it. It lacks the ability to turn that into something other than text. To me that shines a light on the claim that there is a real conceptual space limited by the output being generated text.
- bob1029 2y ago
  > ruining the internet with all this damn parasitic crawling I feel like we may be conflating the performance implications of crawling with the intellectual property ones.
  - devnullbrain 2y ago
    PC's complaint may be related to this: https://x.com/GergelyOrosz/status/1905000078839845303 https://x.com/GergelyOrosz/status/1905000078839845303
jerf 2y ago
LLMs are, to put it in human brain terms, super, super, super hypertrophied speech centers. The human brain is completely capable of having grown a hypertrophied speech center if that was the path to intelligence. It did not, because that is not the path to general intelligence. Of course LLMs can't play Pokemon long term. Used this way, they're the AI equivalent of someone who has the beginnings of dementia, who knows it, and is compensating by metaphorically sitting in a corner rocking to themselves repeating things so they don't forget them, because all they can remember is what they said. It's amazing what they can do. I'm not denying anything they have actually done, because things that have been concretely been done, are done. But the AIs that will actually fulfill the promises of this generation of AI are the ones yet to come, that incorporate some sort of memory-analog that isn't just "I repeated something to myself inside my token window", and some sort of symbolic manipulation capability. In both cases, I'm using "some sort of" very broadly. I don't know what that looks like any more than anyone else. For instance, to obtain "symbolic manipulation capability" I don't necessarily mean hooking up a symbolic manipulation package. Humans can manipulate symbols but we clearly do it in a somewhat inefficient and often incorrect way with our neural nets. Even so, we get a great deal of benefit from it. We don't have integrated symbolic manipulation packages, and using the ones that exist is actually a rather rare and difficult skill. But what LLMs do is really quite different from either what humans or software packages do. However I think it's pretty clear that LLMs aren't really going to get much farther than they have now in terms of raw capability. People will continue to find clever ways to use them, of course, but the raw capabilities of LLMs qua LLMs are probably pretty close to tapped out. I expect in the future that what we today call "LLMs" will become a component of a system that parses text into vectors and then feeds those vectors into what we consider the "real" AI as those vectors, and the AI will in turn emit vectors back into the LLM module that will be converted to human speech. And I rather suspect those LLMs, since they won't be the thing trying to do the "real work", will actually be quite a bit smaller than they are today, with the bulk of the computational power being allocated to the "real" AI. We won't be hypertrophying the LLM layer to try to provide a poor and tiny memory and maintain the state of whatever is happening because the "real" AI layer will be doing that. The future will probably laugh at us sitting here trying to make the language center bigger and bigger and bigger when obviously the correct answer was to... do whatever it is they did in their past but our future.
boomboomsubban 2y ago
I'm surprised it hasn't indexed some speed runners notes and is able to recreate them. I wonder how it'd do if you asked it to compete in the "any%" speed run category.
- YurgenJurgensen 2y ago
  Any% Pokemon Red is under a minute and a half long and beats the game using memory corruption and needs specific inputs in a four-frame window. Claude has no concept of time. What do you think is going to happen?
smjburton 2y ago
> But despite recent advances in AI image processing, Hershey said Claude still struggles to interpret the low-resolution, pixelated world of a Game Boy screenshot as well as a human can. > We built the text side of it first, and the text side is definitely... more powerful. How these models can reason about images is getting better, but I think it's a decent bit behind. This seems to be the main issue: using an AI model predominantly trained on text-based reasoning to play through a graphical video game challenge. Given this context, the image processing for this model is like an underdeveloped skill compared to its text-based reasoning. Even though it spent an excessive amount of time navigating through Mt. Moon or getting trapped in small areas of the map, Claude will likely only get better at playing Pokemon or other games as it's trained on more image-related tasks and the model balances out its capabilities.
disambiguation 2y ago
We all know the shortcomings of LLMs but its been interesting to observe Agency as a system and the flaws that emerge. For example, does Claude have any reason to get from point A to B quickly and efficiently? Running into walls, running in circles, back tracking, etc. Clearly it doesn't have a sense of urgency, but does it matter as long as it eventually gets there? But here's what strikes me as odd about the whole approach. Everyone knows it's easier for an LLM to write a program that counts the number of R's in "strawberry" than it is to count it directly, yet no one is leveraging this fact. Instead of ever more elaborate prompts and contexts, why not ask Claude to write a program for mapping and path finding? Hell if that's too much to ask, maybe make the tools before hand and see if it's at least smart enough to use them effectively. My personal wishlist is things like fact tables, goal graphs, and a word model - where things are and when things happened. Strategies and hints. All these things can be turned into formal systems. Something as simple as a battle calculator should be a no-brainer. My last hair brained idea - I would like to see LLMs managing an ontology in prolog as a proxy for reasoning. This all theory and even if implemented wouldn't solve everything, but I'm tired of watching them throw prompts at the wall in hopes that the LLM can be tricked into being smarter than it is.
- bigmadshoe 2y ago
  I worked for one of a few "game playing AI as a service" startups for several years, and we did in fact leverage the strong programming skills of LLMs to play games. When we pivoted from training directly on human gameplay data to simply having the LLM write modular code to play the game, we managed to solve many long-standing problems in our demo levels. Even on unseen games this works well, since LLMs already have a good understanding of how video games work generally.
  - disambiguation 2y ago
    That's great that someone out there is solving this stuff, but it begs the question why we're watching Claude fumbling around and not seeing this service in action?
    - 2y ago
cruffle_duffle 2y ago
These things are not general purpose AI tools. The are Large Language tools. There are dudes on YouTube who get millions of views doing basic reinforcement learning to train weird shapes to navigate obstacle course, win races, learn to walk, etc. But they do this by making a custom model with inputs and outputs that are directly mapped into the “physical world” which these creatures live. Until these LLM’s have input and output parameters that specifically wire into “front distance sensor or leg pressure sensor” and “leg muscles or engine speed” they are never going to be truly good at tasks requiring such interaction. Any such attempt that lacks such inputs and outputs and somehow manages to have passable results will be in spite of the model not because of it. They’ll always get their ass kicked by specialized models trained for such tasks on every dimension including efficiency, power, memory use, compute and size. And that is the thing, despite their incredible capabilities, LLM’s are not AGI and they are not general purpose models either! And they never will be. And that is just fine.

53 points by Workaccount2 2y ago

I very much feel nostalgic now, thinking about the time when Twitch plays pokemon was something we all enjoyed.

I wonder how much help probably having tons of guides in its training material is. This really does feel like the sort of test that’s actually interesting for evaluating how general these AI can be especially compared to the existing tests.

dezgeg 2y ago
It's also a detriment to some extent, as Claude's now believing things that apply only to later generations of the game (doesn't help that those beliefs would apply to the FireRed remake). Also it made Claude try to perform quests out of order and waste lots of time trying to do things he could only do later in the game (the thirsty guards blocking some routes-sidequest).
- morcus 2y ago
  It's interesting to me though because I can see a human getting confused similarly. IIRC when I played Pokemon a while back someone told me you could get a valuable item (Leftovers) by checking the garbage can on one ship in one game, and as a result when I played I checked every garbage can in case there was a hidden valuable item in there. I'm not even sure if Leftovers was obtained in the same way in the game I was playing.

rsynnott 2y ago

> OpenAI is quietly seeding expectations for a "PhD-level" AI agent that could operate autonomously at the level of a "high-income knowledge worker" in the near future. Elon Musk says that "we'll have AI smarter than any one human probably" by the end of 2025. Anthropic CEO Dario Amodei thinks it might take a bit longer but similarly says it's plausible that AI will be "better than humans at almost everything" by the end of 2027. I am baffled that anyone is still buying this complete nonsense. Like, come on.

readthenotes1 2y ago
Will AI be better at hyping AI than certain humans? Seems improbable...
d3nj4l 2y ago
Yeah, the people with a vested interest in AI say AI is good, news at 9.
- rsynnott 2y ago
  I mean I think this is actually kind of abnormal; you do not get this level of absolute nonsense promises in most industries, including most parts of the tech industry.
zurfer 2y ago
It's an interesting world where both things can be true: 1 AI surpassing humans even in PhD level tasks in a couple of years and 2 current AI not being able to successfully play a childrens game. One way to square that is that AIs have a big memory advantage that allows them to store and access information much faster than humans. However they still lack visual understanding and some forms of reasoning and live learning. Pokemon doesn't require PhD level knowledge and rather just all the other qualities that make up intelligence. Now if the big labs make progress on visual reasoning etc. the moment AI can solve Pokemon it might be able to solve everything else as well.
- nprateem 2y ago
  Yeah right. It can't draw on enough of its training data to be useful. That's why it gets stuck going down one particular alley without considering alternatives. See the whole using glue to keep pepperoni on your pizza. Even looping a la Claude extended doesn't help. It still keeps going in circles.
  - alabastervlog 2y ago
    There's so incredibly much knowledge in non-digitized documents and trapped in human brains, yet. Plus in various proprietary databases. It's 2025 and that stuff hasn't already made it onto the "repository of all human knowledge" (LO fucking L) World Wide Web still, so... I wouldn't hold my breath.

skybrian 2y ago

Interesting analysis. Would anyone care to bet on whether Claude will beat Pokémon in a couple of years? It seems like a good question for a prediction market. My guess is that image understanding will improve.

amelius 2y ago
If that bet existed, then who decides which benchmark is used?
cratermoon 2y ago
> My guess is that image understanding will improve. The "it will get better" assertion fails on what's known as the "first step fallacy". The analogy I like to use is the idea of a ladder to the moon. We can build tall self-supporting ladders, and ladder technology has advanced considerably since they were first used at least 10,000 years ago. More recently, in 1862, John H. Balsley made the step ladder safer by changing the rounded rungs to flat steps. Henry Quackenbush patented the extension ladder in 1867. Ladder technology continues to progress, will we someday build a ladder tall enough to climb to the moon? Do you see the fallacy? https://thebullshitmachines.com/lesson-16-the-first-step-fallacy/index.html https://thebullshitmachines.com/lesson-16-the-first-step-fal...
- someothherguyy 2y ago
  No one is attempting to build ladders to the moon though?
  - namaria 2y ago
    No, but some people claim that compressing text into vector spaces and using the patterns to extend bits of text will somehow do more than extend texts in a sometimes plausible, sometimes nonsensical way if we just: add enough data, and process it for long enough.
    - someothherguyy 2y ago

0xbadcafebee 2y ago

Anyone who has used AI recently knows how much bullshit the claims are about replacing any human for virtually any task. AI still can't do super advanced things like tell me the correct height of my truck from the manufacturer's tech specs PDF that it has in its databank. Even when I tell it what the correct height is, it'll randomly change the height throughout a session. I have to constantly correct it because thankfully I know enough about a given subject that I know it's bullshitting. Once I correct it, it suddenly admits, oh yeah, actually it was this other thing, here's the right info. It's an amazing search engine, and has really cool suggestions/ideas. It's certainly very useful. But replace a human? Get real. All these stocks are going to tank once the media starts running honest articles instead of PR fluff.

nprateem 2y ago
Exactly. It was a stroke of genius to popularise the word 'hallucinate' instead of 'bullshit' or 'wrong'. Since it has no fundamental understanding of anything, it's either a sycophant or arrogant idiot. When I told chatgpt it didn't understand something it obnoxiously told me it understood perfectly. When I told it why, it did a 180 and carried on where I'd left off. I don't use it any more.
Henchman21 2y ago
Do you have an ETA for when the media will start running honest articles instead of PR fluff? Asking for the entire western world…
- deleted 2y ago
  [deleted]
- cratermoon 2y ago
  About the time they stop depending on targeted advertising revenue.
alabastervlog 2y ago
The things feel closer to using a fine-grained search algo with some randomness and a fairly large corpus, than to interacting with something intelligent. And if you read how they work... that's because that's exactly what they are. There's no thinking going on. When using them, that they've been programmed and prompted to have some kind of tone or personality and to fit within some kind of parameters of "behavior" as if they're a real being reminds me of Flash-based site navigation in the early '00s: flashy bullshit that's impressive for about one minute and then just annoying and inconvenient forever. As for programming with them, writing the prompts feels more like just another kind of programming than instructing a human. I'm skeptical this entire approach is more than one small part of what might become AGI, given several more somewhat-unrelated breakthroughs, including probably in hardware. Like a lot of us, I've gotten sucked into building products with these things because every damn company on the planet, tech or not, has decided to do that for usually-very-bad reasons, and I'm a little worried this is going to crash so hard that having been involved in it at all will be a (minor) black mark on my résumé.

blainm 2y ago

So when it really struggled to get around (kept just walking into obstacles), they gave Claude the ability to navigate by adding pathfinding and awareness of its position and map ID. However, it still struggles, particularly in open-ended areas. This suggests a fundamental issue beyond just navigation. While accessing more RAM data or using external tools using said data could improve consistency or get it further, that approach reduces the extent to which Claude is independently playing and reasoning. A more effective solution would enhance its decision-making without relying on direct RAM access or any kind of fine tuning. I'm sure it's possible. There has to be a better approach, and also in a way that's not relying on reading values from RAM or any kind of fine tuning.

OgsyedIE 2y ago
It can't do a good job of reasoning about higher-level abstractions in its long term memory without making poor decisions about which memory items to retain and which to forget. Would a mixture-of-experts paradigm, where each expert weights the value of short-term memories differently to the weight of long-term memoried, do noticeably better at overcoming that one category of roadblocks?
rtkwe 2y ago
Seems like the 200k context window is a huge issue and it's summarization deletes important information leading it to revisit solved areas even when it's working properly or simply forget things it needs to progress.

Recursing 2y ago

This Sunday there's going to be a hackaton at https://lu.ma/poke https://lu.ma/poke to improve the Pokemon-playing agent. I think most hackatons don't achieve much, but maybe someone from HN can go improve the scaffolding and save Claude

kibwen 2y ago

> Claude will readily notice when the game tells it that an attack from an electric-type Pokémon is “not very effective” against a rock-type opponent, for instance. Claude will then squirrel that factoid away in a massive written knowledge base for future reference later in the run. But these models already know all this information??? Surely it's ingested Bulbapedia, along with a hundred zillion terabytes of every other Pokemon resource on the internet, so why does it need to squirrel this information away? What's the point of ruining the internet with all this damn parasitic crawling if the models can't even recall basic facts like "thunderbolt is an electric-type move", "geodude is a rock-type pokemon", "electric-type moves are ineffective against rock-type pokemon"?

gbear605 2y ago
Claude definitely does know it (I just checked). Plausibly they’re giving that as an example of other things that it actually needs to save, but yeah, it’s a bad example.
- stonemetal12 2y ago
  It has trained on multiple guides and walkthroughs and whatnot for the game from the internet. Theoretically it knows the game inside and out, it can recall all of that if you ask about it. It lacks the ability to turn that into something other than text. To me that shines a light on the claim that there is a real conceptual space limited by the output being generated text.
bob1029 2y ago
> ruining the internet with all this damn parasitic crawling I feel like we may be conflating the performance implications of crawling with the intellectual property ones.
- devnullbrain 2y ago
  PC's complaint may be related to this: https://x.com/GergelyOrosz/status/1905000078839845303 https://x.com/GergelyOrosz/status/1905000078839845303

jerf 2y ago

LLMs are, to put it in human brain terms, super, super, super hypertrophied speech centers. The human brain is completely capable of having grown a hypertrophied speech center if that was the path to intelligence. It did not, because that is not the path to general intelligence. Of course LLMs can't play Pokemon long term. Used this way, they're the AI equivalent of someone who has the beginnings of dementia, who knows it, and is compensating by metaphorically sitting in a corner rocking to themselves repeating things so they don't forget them, because all they can remember is what they said. It's amazing what they can do. I'm not denying anything they have actually done, because things that have been concretely been done, are done. But the AIs that will actually fulfill the promises of this generation of AI are the ones yet to come, that incorporate some sort of memory-analog that isn't just "I repeated something to myself inside my token window", and some sort of symbolic manipulation capability. In both cases, I'm using "some sort of" very broadly. I don't know what that looks like any more than anyone else. For instance, to obtain "symbolic manipulation capability" I don't necessarily mean hooking up a symbolic manipulation package. Humans can manipulate symbols but we clearly do it in a somewhat inefficient and often incorrect way with our neural nets. Even so, we get a great deal of benefit from it. We don't have integrated symbolic manipulation packages, and using the ones that exist is actually a rather rare and difficult skill. But what LLMs do is really quite different from either what humans or software packages do. However I think it's pretty clear that LLMs aren't really going to get much farther than they have now in terms of raw capability. People will continue to find clever ways to use them, of course, but the raw capabilities of LLMs qua LLMs are probably pretty close to tapped out. I expect in the future that what we today call "LLMs" will become a component of a system that parses text into vectors and then feeds those vectors into what we consider the "real" AI as those vectors, and the AI will in turn emit vectors back into the LLM module that will be converted to human speech. And I rather suspect those LLMs, since they won't be the thing trying to do the "real work", will actually be quite a bit smaller than they are today, with the bulk of the computational power being allocated to the "real" AI. We won't be hypertrophying the LLM layer to try to provide a poor and tiny memory and maintain the state of whatever is happening because the "real" AI layer will be doing that. The future will probably laugh at us sitting here trying to make the language center bigger and bigger and bigger when obviously the correct answer was to... do whatever it is they did in their past but our future.

boomboomsubban 2y ago

I'm surprised it hasn't indexed some speed runners notes and is able to recreate them. I wonder how it'd do if you asked it to compete in the "any%" speed run category.

YurgenJurgensen 2y ago
Any% Pokemon Red is under a minute and a half long and beats the game using memory corruption and needs specific inputs in a four-frame window. Claude has no concept of time. What do you think is going to happen?

smjburton 2y ago

> But despite recent advances in AI image processing, Hershey said Claude still struggles to interpret the low-resolution, pixelated world of a Game Boy screenshot as well as a human can. > We built the text side of it first, and the text side is definitely... more powerful. How these models can reason about images is getting better, but I think it's a decent bit behind. This seems to be the main issue: using an AI model predominantly trained on text-based reasoning to play through a graphical video game challenge. Given this context, the image processing for this model is like an underdeveloped skill compared to its text-based reasoning. Even though it spent an excessive amount of time navigating through Mt. Moon or getting trapped in small areas of the map, Claude will likely only get better at playing Pokemon or other games as it's trained on more image-related tasks and the model balances out its capabilities.

disambiguation 2y ago

We all know the shortcomings of LLMs but its been interesting to observe Agency as a system and the flaws that emerge. For example, does Claude have any reason to get from point A to B quickly and efficiently? Running into walls, running in circles, back tracking, etc. Clearly it doesn't have a sense of urgency, but does it matter as long as it eventually gets there? But here's what strikes me as odd about the whole approach. Everyone knows it's easier for an LLM to write a program that counts the number of R's in "strawberry" than it is to count it directly, yet no one is leveraging this fact. Instead of ever more elaborate prompts and contexts, why not ask Claude to write a program for mapping and path finding? Hell if that's too much to ask, maybe make the tools before hand and see if it's at least smart enough to use them effectively. My personal wishlist is things like fact tables, goal graphs, and a word model - where things are and when things happened. Strategies and hints. All these things can be turned into formal systems. Something as simple as a battle calculator should be a no-brainer. My last hair brained idea - I would like to see LLMs managing an ontology in prolog as a proxy for reasoning. This all theory and even if implemented wouldn't solve everything, but I'm tired of watching them throw prompts at the wall in hopes that the LLM can be tricked into being smarter than it is.

bigmadshoe 2y ago
I worked for one of a few "game playing AI as a service" startups for several years, and we did in fact leverage the strong programming skills of LLMs to play games. When we pivoted from training directly on human gameplay data to simply having the LLM write modular code to play the game, we managed to solve many long-standing problems in our demo levels. Even on unseen games this works well, since LLMs already have a good understanding of how video games work generally.
- disambiguation 2y ago
  That's great that someone out there is solving this stuff, but it begs the question why we're watching Claude fumbling around and not seeing this service in action?
  - 2y ago

cruffle_duffle 2y ago

These things are not general purpose AI tools. The are Large Language tools. There are dudes on YouTube who get millions of views doing basic reinforcement learning to train weird shapes to navigate obstacle course, win races, learn to walk, etc. But they do this by making a custom model with inputs and outputs that are directly mapped into the “physical world” which these creatures live. Until these LLM’s have input and output parameters that specifically wire into “front distance sensor or leg pressure sensor” and “leg muscles or engine speed” they are never going to be truly good at tasks requiring such interaction. Any such attempt that lacks such inputs and outputs and somehow manages to have passable results will be in spite of the model not because of it. They’ll always get their ass kicked by specialized models trained for such tasks on every dimension including efficiency, power, memory use, compute and size. And that is the thing, despite their incredible capabilities, LLM’s are not AGI and they are not general purpose models either! And they never will be. And that is just fine.