4 ms·

The AI can write a chess bot program that will beat you. You're thinking about this the wrong way. The system is built and delivered as it is because that's ho

by echelon 10d ago

The AI can write a chess bot program that will beat you. You're thinking about this the wrong way. The system is built and delivered as it is because that's how the providers make the most money. If they cared to have it perform well in chess games, you'd see a different shape and behavior. We shouldn't ask the multibillion dollar automated software generation system to play games with us any more than we should ask a Boeing's flight guidance system to do so.

minraws 10d ago
So AGI needs to be trained on something to work well on it. Lovely reasoning we have right here. Delusion runs deep in HN circles. I say that as someone heavily invested in AI startups and projects and as someone working in the field. I think most people on HN should touch grass and find real human contact. Lmao Incredible reasoning all around here.
- echelon 10d ago
  I'm stating that certain folks are trying to use the software-generating product as an AGI/ASI and then complaining when it doesn't play chess very well. People are holding it wrong, deliberately or not. Some are inventing bad faith measures so they can claim AI sucks.
  - minraws 10d ago
    Then why respond at all for the sake of responding? We all know AI can code, but the question it all stemmed from what if it's AGI or GM level in chess on it's own. You can't just back pedal from the statement that apparently being able to code a chess engine is the same as being good at chess. I can write a chess engine that beats Magnus Carlson without AI that alone neither makes me GM level or AGI or any of the other claims the above comments seem to be making?
    - sdf32dsf 10d ago
      He keeps posting with a particular type of tone. He definitely needs to touch grass.
      - echelon 10d ago
        Try to embrace hacker ethos and stop hating. Y'all seem to miss the point of this forum. Building and hacking and science and engineering. I swear there's a whole lot of you who just like to look down instead of up. There's a whole universe up there.
    - bigstrat2003 10d ago
      > We all know AI can code... We know no such thing. LLMs are quite bad at generating code, worse than any capable human.
  - modulus1 10d ago
    I agree w/ this perspective. An agent with a harness that can run programs can solve a lot more than one without the harness. The AI system includes the harness, and it's not clear to me that AGI requires more than LLMs + code generation & execution are capable of.
    - minraws 10d ago
      So AI is AGI in fields where code can't solve anything? Is code omnipotent, I have been in software all my life and I would hard agree here. Sure stuff LLMs can do with being good at parts of code reproduction is incredible. And honestly it's the new way to do a lot of things but I have not see an iota of proof that it can scale across the board. For instance Maths is just code with different symbols and slightly less universally legible concepts. AI is the best invention at figuring out or walking the search space and directionally doing logically computation over general software adjacent stuff. But that's it, I am certain a bunch of companies will make a lot of money despite no AGI. I think people either don't understand AGI or don't understand how real world works. Until an LLM can bow it's head take responsibility for mistakes made and ensure they aren't repeated again with 100% confidence to the leadership it's inarguably a tool a rather questionable one at that.
      - simianwords 10d ago
        > AI is the best invention at figuring out or walking the search space and directionally doing logically computation over general software adjacent stuff. So.. like chess? Anyway, do you have any prediction on what LLM's can or can't do in a few years?
  - Yizahi 10d ago
    It's not even a "software-generating product". It's only half of it. Most of the heavy lifting is done by absolutely not-AI compilers, analyzers and the like. If not for these programs, written well before AI boom, them LLMs would be no better at programming than they are are at pure LLM based calculations or writing.
- diehunde 10d ago
  AI bros: the LLM beats humans at solving Navier-Stokes and some old cypher. We are close to AGI Also AI bros: LLM can’t beat an avg chess player. But that doesn’t mean anything. It doesn’t count
  - hackinthebochs 10d ago
    >LLM can’t beat an avg chess player. Why should that matter?
    - janalsncm 10d ago
      If something has general intelligence it should be able to read the rules of a game and follow them. Therefore an artificial general intelligence (AGI) should be able to do this. So we have a situation where very powerful and influential people are saying we will have AGI in 6 months (if we don’t already), yet the facts on the ground are so clearly pointing in the opposite direction.
      - hackinthebochs 10d ago
        I would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than what I responded to.
        minraws 10d ago
        I can write you a benchmark to prove it even with a heavy handed system prompt Astra will make an illegal move during the course of the games first few moves are generally ok since it's just throwing out learned moves.
        hackinthebochs 10d ago
        I'd genuinely like to see the results of that.
        10d ago
- Gregkion 10d ago
  An AGI doesn't stand for 'perfect intelligence' it stands for artificial general intelligence. And no an AGI system doesn't need to play chess on a certain level to be disruptive to you and me and whole industries. It only needs to be as good as a person and cheaper. Just because you define AGI as something it doesn't has to be,doesn't mean i need to touch grass. This chess comparision is one of the most ignorant and stupid arguments i have heard after the parrot thing
  - tsimionescu 10d ago
    Do you know what the "General" in "Artificial General Intelligence" means? It specifically means that the AGI adapts to novel domains that it hasn't been trained on - its training generalizes to real world problems. That doesn't mean it has to be extraordinary at these things. But to be AGI, it has to have some level of competency when used on problems outside its training set. In particular, it the LLMs were to install a known chess engine and run that to get the moves when asked to play chess, that would qualify for more AGI-like behavior. But really, chess is such a simplistic game that they should be able to do decently well at it even without even needing that. At the very least, they should be able to consistently play without making illegal moves - something that many 7-year olds manage quite well.
  - rsfern 10d ago
    On the contrary, I think the chess comparison is on point. We’re discussing observations that even the strongest models devolve into making invalid moves without scaffolding. For me that raises the question of whether these models are learning the rules and generalizing from them, or of they’re just pattern matching and flailing on this task. Maybe the reality is somewhere in between, but the benchmarks don’t seem to directly measure conceptual generalization, they measure task completion. They can disrupt a lot of people and industries by pattern matching and flailing without being AGI. I’m sure these models know the rules and can explain them when prompted, but that doesn’t seem to be the way they actually complete this task. Will they get there? Maybe
striking 10d ago
It's not quite the same, but the in-flight chess game provided by Delta was known to be absurdly hard: https://news.ycombinator.com/item?id=46593395 https://news.ycombinator.com/item?id=46593395
- willmarch 10d ago
  I believe I remember reading it was based on Glaurung's code (which eventually evolved into what we now know as the juggernaut Stockfish).
what 10d ago
I can write a chess bot program that will beat you. Does that mean I’m good at chess? >If they cared to have it perform well in chess games, you'd see a different shape and behavior. So the things they claim are on the verge of AGI actually aren’t? They need to be trained for specific tasks?
- phoghed 10d ago
  They’ll never be AGI simply because the definition will be constantly updated to be some steps ahead of them.
  - fc417fc802 10d ago
    I'm pretty sure "competent at chess without external aids" has been on the standard AGI checklist since before personal computers were a thing. How can you claim an intelligence is general if it can't make sense of such a highly constrained board game? This is solidly table stakes.
    - phoghed 10d ago
      Because they’ll train it to be good at chess and then everyone will say yeah but playing chess doesn’t mean you’re AGI, it can’t even ____ It can’t even count the R’s in strawberry It can’t even add numbers It can’t even solve a millennium puzzle It’s not even a chess GM It’s not even beyond human capability in Go It can’t even drive a car It can’t even self replicate It can’t even build weapons It doesn’t even have feelings So how could someone conceivably convince everyone that some system is AGI when there are still tasks that some human or group of humans can do that the system cannot? This will only happen, in my opinion, when the model/system can self-improve at a rate that scares people.
      - fc417fc802 10d ago
        > and then everyone will say yeah but playing chess doesn’t mean you’re AGI, it can’t even One, you're not addressing what I wrote above and two, yes, that's absolutely correct. Doing X doesn't qualify something as AGI. If you can't X you can't be AGI. The inverse doesn't hold though. Notably, if you have to retrain the model in order to X then it can't possibly be AGI since if it were _general_ it would be capable of figuring X out on its own having never seen it before.
jibal 10d ago
First, you're moving the goalposts. Second, it's not actually true that any existing frontier AI can write a chess bot program that can beat a 1600 player ... not unless the program is derived from Stockfish or some other leading engine that has been in development for decades. > The system is built and delivered as it is because that's how the providers make the most money. If they cared to have it perform well in chess games, you'd see a different shape and behavior. These comments indicate a complete failure to understand the technology. I won't respond again.
zahlman 10d ago
> The system is built and delivered as it is because that's how the providers make the most money. If they cared to have it perform well in chess games, you'd see a different shape and behavior. This argument is fundamentally incompatible with all the breathless rhetoric about "AGI" coming from the providers' general direction.
- echelon 10d ago
  It's really not. The labs frequently apply their raw models to problems that do not make economic sense for their customers but that demonstrate the power and capability of their systems. These experiments can cost millions of dollars. That's not customer-shaped. They're not going to give you access to that. It's not a product. The government might have an interest in this, but that's not something you'd be privileged to know about. And when these labs do develop "AGI", they more than likely won't be selling it to end users. They've pretty much already said this.
- anthonyrstevens 10d ago
  >> the breathless rhetoric about "AGI" coming from the providers' general direction So many commenters here see it as their ... duty? to argue against the most optimistic/unhinged (take your pick) arguments from "the other side" and then treat everybody who disagrees as a shill or an idiot. Why is "being good at chess" a proxy for whatever AGI strawmen you want to argue against? Maybe step back from your black-and-white ledge and think about discussing what's actually under discussion? For example, why or why not would an LLM be good at chess? Will they be good at chess? What technical limitations might preclude that?

by echelon 10d ago

minraws 10d ago

So AGI needs to be trained on something to work well on it. Lovely reasoning we have right here. Delusion runs deep in HN circles. I say that as someone heavily invested in AI startups and projects and as someone working in the field. I think most people on HN should touch grass and find real human contact. Lmao Incredible reasoning all around here.

echelon 10d ago
I'm stating that certain folks are trying to use the software-generating product as an AGI/ASI and then complaining when it doesn't play chess very well. People are holding it wrong, deliberately or not. Some are inventing bad faith measures so they can claim AI sucks.
- minraws 10d ago
  Then why respond at all for the sake of responding? We all know AI can code, but the question it all stemmed from what if it's AGI or GM level in chess on it's own. You can't just back pedal from the statement that apparently being able to code a chess engine is the same as being good at chess. I can write a chess engine that beats Magnus Carlson without AI that alone neither makes me GM level or AGI or any of the other claims the above comments seem to be making?
  - sdf32dsf 10d ago
    He keeps posting with a particular type of tone. He definitely needs to touch grass.
    - echelon 10d ago
      Try to embrace hacker ethos and stop hating. Y'all seem to miss the point of this forum. Building and hacking and science and engineering. I swear there's a whole lot of you who just like to look down instead of up. There's a whole universe up there.
  - bigstrat2003 10d ago
    > We all know AI can code... We know no such thing. LLMs are quite bad at generating code, worse than any capable human.
- modulus1 10d ago
  I agree w/ this perspective. An agent with a harness that can run programs can solve a lot more than one without the harness. The AI system includes the harness, and it's not clear to me that AGI requires more than LLMs + code generation & execution are capable of.
  - minraws 10d ago
    So AI is AGI in fields where code can't solve anything? Is code omnipotent, I have been in software all my life and I would hard agree here. Sure stuff LLMs can do with being good at parts of code reproduction is incredible. And honestly it's the new way to do a lot of things but I have not see an iota of proof that it can scale across the board. For instance Maths is just code with different symbols and slightly less universally legible concepts. AI is the best invention at figuring out or walking the search space and directionally doing logically computation over general software adjacent stuff. But that's it, I am certain a bunch of companies will make a lot of money despite no AGI. I think people either don't understand AGI or don't understand how real world works. Until an LLM can bow it's head take responsibility for mistakes made and ensure they aren't repeated again with 100% confidence to the leadership it's inarguably a tool a rather questionable one at that.
    - simianwords 10d ago
      > AI is the best invention at figuring out or walking the search space and directionally doing logically computation over general software adjacent stuff. So.. like chess? Anyway, do you have any prediction on what LLM's can or can't do in a few years?
- Yizahi 10d ago
  It's not even a "software-generating product". It's only half of it. Most of the heavy lifting is done by absolutely not-AI compilers, analyzers and the like. If not for these programs, written well before AI boom, them LLMs would be no better at programming than they are are at pure LLM based calculations or writing.
diehunde 10d ago
AI bros: the LLM beats humans at solving Navier-Stokes and some old cypher. We are close to AGI Also AI bros: LLM can’t beat an avg chess player. But that doesn’t mean anything. It doesn’t count
- hackinthebochs 10d ago
  >LLM can’t beat an avg chess player. Why should that matter?
  - janalsncm 10d ago
    If something has general intelligence it should be able to read the rules of a game and follow them. Therefore an artificial general intelligence (AGI) should be able to do this. So we have a situation where very powerful and influential people are saying we will have AGI in 6 months (if we don’t already), yet the facts on the ground are so clearly pointing in the opposite direction.
    - hackinthebochs 10d ago
      I would bet a lot of money that Astra can follow the rules of chess (perhaps if repeated within the context window). Also, this is a different argument than what I responded to.
      - minraws 10d ago
        I can write you a benchmark to prove it even with a heavy handed system prompt Astra will make an illegal move during the course of the games first few moves are generally ok since it's just throwing out learned moves.
        hackinthebochs 10d ago
        I'd genuinely like to see the results of that.
      - 10d ago
Gregkion 10d ago
An AGI doesn't stand for 'perfect intelligence' it stands for artificial general intelligence. And no an AGI system doesn't need to play chess on a certain level to be disruptive to you and me and whole industries. It only needs to be as good as a person and cheaper. Just because you define AGI as something it doesn't has to be,doesn't mean i need to touch grass. This chess comparision is one of the most ignorant and stupid arguments i have heard after the parrot thing
- tsimionescu 10d ago
  Do you know what the "General" in "Artificial General Intelligence" means? It specifically means that the AGI adapts to novel domains that it hasn't been trained on - its training generalizes to real world problems. That doesn't mean it has to be extraordinary at these things. But to be AGI, it has to have some level of competency when used on problems outside its training set. In particular, it the LLMs were to install a known chess engine and run that to get the moves when asked to play chess, that would qualify for more AGI-like behavior. But really, chess is such a simplistic game that they should be able to do decently well at it even without even needing that. At the very least, they should be able to consistently play without making illegal moves - something that many 7-year olds manage quite well.
- rsfern 10d ago
  On the contrary, I think the chess comparison is on point. We’re discussing observations that even the strongest models devolve into making invalid moves without scaffolding. For me that raises the question of whether these models are learning the rules and generalizing from them, or of they’re just pattern matching and flailing on this task. Maybe the reality is somewhere in between, but the benchmarks don’t seem to directly measure conceptual generalization, they measure task completion. They can disrupt a lot of people and industries by pattern matching and flailing without being AGI. I’m sure these models know the rules and can explain them when prompted, but that doesn’t seem to be the way they actually complete this task. Will they get there? Maybe

striking 10d ago

It's not quite the same, but the in-flight chess game provided by Delta was known to be absurdly hard: https://news.ycombinator.com/item?id=46593395 https://news.ycombinator.com/item?id=46593395

willmarch 10d ago
I believe I remember reading it was based on Glaurung's code (which eventually evolved into what we now know as the juggernaut Stockfish).

what 10d ago

I can write a chess bot program that will beat you. Does that mean I’m good at chess? >If they cared to have it perform well in chess games, you'd see a different shape and behavior. So the things they claim are on the verge of AGI actually aren’t? They need to be trained for specific tasks?

phoghed 10d ago
They’ll never be AGI simply because the definition will be constantly updated to be some steps ahead of them.
- fc417fc802 10d ago
  I'm pretty sure "competent at chess without external aids" has been on the standard AGI checklist since before personal computers were a thing. How can you claim an intelligence is general if it can't make sense of such a highly constrained board game? This is solidly table stakes.
  - phoghed 10d ago
    Because they’ll train it to be good at chess and then everyone will say yeah but playing chess doesn’t mean you’re AGI, it can’t even ____ It can’t even count the R’s in strawberry It can’t even add numbers It can’t even solve a millennium puzzle It’s not even a chess GM It’s not even beyond human capability in Go It can’t even drive a car It can’t even self replicate It can’t even build weapons It doesn’t even have feelings So how could someone conceivably convince everyone that some system is AGI when there are still tasks that some human or group of humans can do that the system cannot? This will only happen, in my opinion, when the model/system can self-improve at a rate that scares people.
    - fc417fc802 10d ago
      > and then everyone will say yeah but playing chess doesn’t mean you’re AGI, it can’t even One, you're not addressing what I wrote above and two, yes, that's absolutely correct. Doing X doesn't qualify something as AGI. If you can't X you can't be AGI. The inverse doesn't hold though. Notably, if you have to retrain the model in order to X then it can't possibly be AGI since if it were _general_ it would be capable of figuring X out on its own having never seen it before.

jibal 10d ago

First, you're moving the goalposts. Second, it's not actually true that any existing frontier AI can write a chess bot program that can beat a 1600 player ... not unless the program is derived from Stockfish or some other leading engine that has been in development for decades. > The system is built and delivered as it is because that's how the providers make the most money. If they cared to have it perform well in chess games, you'd see a different shape and behavior. These comments indicate a complete failure to understand the technology. I won't respond again.

zahlman 10d ago

> The system is built and delivered as it is because that's how the providers make the most money. If they cared to have it perform well in chess games, you'd see a different shape and behavior. This argument is fundamentally incompatible with all the breathless rhetoric about "AGI" coming from the providers' general direction.

echelon 10d ago
It's really not. The labs frequently apply their raw models to problems that do not make economic sense for their customers but that demonstrate the power and capability of their systems. These experiments can cost millions of dollars. That's not customer-shaped. They're not going to give you access to that. It's not a product. The government might have an interest in this, but that's not something you'd be privileged to know about. And when these labs do develop "AGI", they more than likely won't be selling it to end users. They've pretty much already said this.
anthonyrstevens 10d ago
>> the breathless rhetoric about "AGI" coming from the providers' general direction So many commenters here see it as their ... duty? to argue against the most optimistic/unhinged (take your pick) arguments from "the other side" and then treat everybody who disagrees as a shill or an idiot. Why is "being good at chess" a proxy for whatever AGI strawmen you want to argue against? Maybe step back from your black-and-white ledge and think about discussing what's actually under discussion? For example, why or why not would an LLM be good at chess? Will they be good at chess? What technical limitations might preclude that?