3 ms·

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for

There's been substantial effort put into designing and running benchmarks for these models. You know which one they don't report any more? Chess ELO. Watt for watt, every one of these models loses to stockfish. There is no possible scenario where we need to be afraid of "rogue superintelligence" when the intelligence in question cannot reason or plan well enough to play chess. Lecun has the correct approach. Mock these people relentlessly for their attention-seeking doomerism.

Gregkion 20d ago
It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data. Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other. And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat. It doesn't make sense to ask a random LLM for a next chess move. Would you ask a random human to beat a chess master? We have reached peak learning. Its probalby now cheaper to teach 1 LLM something new than teaching it to 100.000 people. Blender support got a lot better just this month. So much better that its quesitonable if anyone new/young wants to learn blender today.
- deleted 20d ago
  [deleted]
- carodgers 20d ago
  "And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat." Always claims and promises about what they "can" do. This pattern is constant across AI hype. No one can prove that an LLM will forever fail to solve problem X, so the claim goes uncontested, and humans are biased to believe unrebutted claims. I reject this fallacious line of reasoning. I will instead require evidence of what they "have" done. No LLM to date has written a chess engine that beats stockfish. More importantly, no LLM has ever written any chess engine without heavily copying code and patterns from humans.
  - pixl97 20d ago
    Then why do you care so much to repeatedly post in this thread? If they are no danger and will never do anything, just let them get banned, or don't. If you are correct (which you are not) then they won't affect your world in any way.
    - carodgers 20d ago
      I am an American taxpayer. The money to pay the regulators that Dario wants to permanently hire will come out of my paycheck and will take from me time I could spend interacting with family, traveling, or pursuing hobbies. Dario and others demanding this can get bent.
      - Gregkion 20d ago
        If you are an american taxpayer, you should be happy. The AI investment from all these companies are good for you as a taxpayer. You do understand that the USA would be in a recession without AI right? And that these billions of invest are driipling down into your economy right?
        carodgers 20d ago
        I am happy. AI is great. I use it. I'm glad that the U.S. is building datacenters. I work for a company involved in siting and building datacenters. But AI is not as scary or dangerous as Dario claims.
        Gregkion 19d ago
        Its a mix probably between a new potential and the progress. Taking it seriuos and then figuring out its not that dangerious is probably a 1000 times better than the opposite. And the last Ubuntu LTS patch had the most CVEs fixed i ever seen. After Snowden, you can assume a lot more of what i thought would be 'tin foil hat' might just have happened like a massive exploit scheme of CVEs. Or things like Stuxnet. You always had to be an expert somehow, now you can throw compute at it. Just imagine a USA, China, russian or israely hackergroup having their uncensored models running in secret datacenters with very basic prompts because they don't care. It would be weird tbh if state actors/hackers aren't doing that yet which already shows us one big problem with ai: AI is technology you have to use now and you have to invest. If you are not, someone else will break into your systems.
ACCount39 20d ago
If chess ELO is the best proxy for general intelligence, then surely, even Deep Blue was smarter than any human alive? You can also claim that the best proxy for general intelligence is being able to multiply large numbers. In which case both humans and LLMs lose resoundingly to an old Casio. Pick a piss poor metric - get a piss poor result. Also, LeCun is a fool and his "world model" approach has consistently failed to yield any advantages over either the "pure imitation learning" or modern "imitation learning annealed by RL", but that's an aside.
- carodgers 20d ago
  Chess ELO is not a proxy for general intelligence because many intelligent people choose to learn things other than chess. But any intelligent person can, if they choose, learn chess and play it well. LLMs cannot do this. They still play very poorly relative to human pros, and even modern LLMs still understand the rules so poorly that they request illegal moves. Anyone claiming that we should be afraid of a text generator that can't understand the rules of chess is being ridiculous.
  - ACCount39 20d ago
    You picked a piss poor metric and you are getting piss poor results. You can absolutely train an LLM-based model to be superhuman at chess. We just don't care enough to do so. LLMs being even as good at implicitly tracking chess board states as they currently are is an emergent capability. The difficulty wasn't in "understanding rules of chess" in a long while now.
    - carodgers 20d ago
      "You picked a piss poor metric and you are getting piss poor results." Well, we agree that the results are piss poor, at least.

Gregkion 20d ago

It doesn't really matter if it will be an LLM or Lecuns world model. Lecuns world model, if its the right architecture, will be retrained by google, anthropic and co with the massive amount of compute they have and the massive amount of data. Also i'm pretty sure Lecun didn't get his money from thin air so big companies are invested in this one way or the other. And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat. It doesn't make sense to ask a random LLM for a next chess move. Would you ask a random human to beat a chess master? We have reached peak learning. Its probalby now cheaper to teach 1 LLM something new than teaching it to 100.000 people. Blender support got a lot better just this month. So much better that its quesitonable if anyone new/young wants to learn blender today.

deleted 20d ago
[deleted]
carodgers 20d ago
"And an LLM can easily, today, just rebuild a chess engine and beat whatever they need to beat." Always claims and promises about what they "can" do. This pattern is constant across AI hype. No one can prove that an LLM will forever fail to solve problem X, so the claim goes uncontested, and humans are biased to believe unrebutted claims. I reject this fallacious line of reasoning. I will instead require evidence of what they "have" done. No LLM to date has written a chess engine that beats stockfish. More importantly, no LLM has ever written any chess engine without heavily copying code and patterns from humans.
- pixl97 20d ago
  Then why do you care so much to repeatedly post in this thread? If they are no danger and will never do anything, just let them get banned, or don't. If you are correct (which you are not) then they won't affect your world in any way.
  - carodgers 20d ago
    I am an American taxpayer. The money to pay the regulators that Dario wants to permanently hire will come out of my paycheck and will take from me time I could spend interacting with family, traveling, or pursuing hobbies. Dario and others demanding this can get bent.
    - Gregkion 20d ago
      If you are an american taxpayer, you should be happy. The AI investment from all these companies are good for you as a taxpayer. You do understand that the USA would be in a recession without AI right? And that these billions of invest are driipling down into your economy right?
      - carodgers 20d ago
        I am happy. AI is great. I use it. I'm glad that the U.S. is building datacenters. I work for a company involved in siting and building datacenters. But AI is not as scary or dangerous as Dario claims.
        Gregkion 19d ago
        Its a mix probably between a new potential and the progress. Taking it seriuos and then figuring out its not that dangerious is probably a 1000 times better than the opposite. And the last Ubuntu LTS patch had the most CVEs fixed i ever seen. After Snowden, you can assume a lot more of what i thought would be 'tin foil hat' might just have happened like a massive exploit scheme of CVEs. Or things like Stuxnet. You always had to be an expert somehow, now you can throw compute at it. Just imagine a USA, China, russian or israely hackergroup having their uncensored models running in secret datacenters with very basic prompts because they don't care. It would be weird tbh if state actors/hackers aren't doing that yet which already shows us one big problem with ai: AI is technology you have to use now and you have to invest. If you are not, someone else will break into your systems.

ACCount39 20d ago

If chess ELO is the best proxy for general intelligence, then surely, even Deep Blue was smarter than any human alive? You can also claim that the best proxy for general intelligence is being able to multiply large numbers. In which case both humans and LLMs lose resoundingly to an old Casio. Pick a piss poor metric - get a piss poor result. Also, LeCun is a fool and his "world model" approach has consistently failed to yield any advantages over either the "pure imitation learning" or modern "imitation learning annealed by RL", but that's an aside.

carodgers 20d ago
Chess ELO is not a proxy for general intelligence because many intelligent people choose to learn things other than chess. But any intelligent person can, if they choose, learn chess and play it well. LLMs cannot do this. They still play very poorly relative to human pros, and even modern LLMs still understand the rules so poorly that they request illegal moves. Anyone claiming that we should be afraid of a text generator that can't understand the rules of chess is being ridiculous.
- ACCount39 20d ago
  You picked a piss poor metric and you are getting piss poor results. You can absolutely train an LLM-based model to be superhuman at chess. We just don't care enough to do so. LLMs being even as good at implicitly tracking chess board states as they currently are is an emergent capability. The difficulty wasn't in "understanding rules of chess" in a long while now.
  - carodgers 20d ago
    "You picked a piss poor metric and you are getting piss poor results." Well, we agree that the results are piss poor, at least.