3 ms·

Here's a good thread over 1+ month, as each model comes out https://bsky.app/profile/pekka.bsky.social/post/3meokmizvt22v https://bsky.app/profile/pekka.bsky.s

Here's a good thread over 1+ month, as each model comes out https://bsky.app/profile/pekka.bsky.social/post/3meokmizvt22v https://bsky.app/profile/pekka.bsky.social/post/3meokmizvt22... tl;dr - Pekka says Arc-AGI-2 is now toast as a benchmark

Aperocky 9mo ago
If you look at the problem space it is easy to see why it's toast, maybe there's intelligence in there, but hardly general.
- verdverm 9mo ago
  the best way I've seen this describes is "spikey" intelligence, really good at some points, those make the spikes humans are the same way, we all have a unique spike pattern, interests and talents ai are effectively the same spikes across instances, if simplified. I could argue self driving vs chatbots vs world models vs game playing might constitute enough variation. I would not say the same of Gemini vs Claude vs ... (instances), that's where I see "spikey clones"
  - Aperocky 9mo ago
    You can get more spiky with AIs, whereas with human brain we are more hard wired. So maybe we are forced to be more balanced and general whereas AI don't have to.
    - verdverm 9mo ago
      I suspect the non-spikey part is the more interesting comparison Why is it so easy for me to open the car door, get in, close the door, buckle up. You can do this in the dark and without looking. There are an infinite number of little things like this you think zero about, take near zero energy, yet which are extremely hard for Ai
      - gowld 9mo ago
        You are asking a robotics question, not an AI question. Robotics is more and less than AI. Boston Dynamics robots are getting quite near your benchmark.
        idiotsecant 9mo ago
        Boston dynamics is missing just about all the degrees of freedom involved in the scenario op mentions.
      - pixl97 9mo ago
        >Why is it so easy for me to open the car door Because this part of your brain has been optimized for hundreds of millions of years. It's been around a long ass time and takes an amazingly low amount of energy to do these things. On the other hand the 'thinking' part of your brain, that is your higher intelligence is very new to evolution. It's expensive to run. It's problematic when giving birth. It's really slow with things like numbers, heck a tiny calculator and whip your butt in adding. There's a term for this, but I can't think of it at the moment.
        te7447 9mo ago
        > There's a term for this, but I can't think of it at the moment. Moravec's paradox: https://epoch.ai/gradient-updates/moravec-s-paradox https://epoch.ai/gradient-updates/moravec-s-paradox
        pixl97 9mo ago
        Thanks, I can never quite remember that.
- tasuki 9mo ago
  > maybe there's intelligence in there, but hardly general. Of course. Just as our human intelligence isn't general.

Aperocky 9mo ago

If you look at the problem space it is easy to see why it's toast, maybe there's intelligence in there, but hardly general.

verdverm 9mo ago
the best way I've seen this describes is "spikey" intelligence, really good at some points, those make the spikes humans are the same way, we all have a unique spike pattern, interests and talents ai are effectively the same spikes across instances, if simplified. I could argue self driving vs chatbots vs world models vs game playing might constitute enough variation. I would not say the same of Gemini vs Claude vs ... (instances), that's where I see "spikey clones"
- Aperocky 9mo ago
  You can get more spiky with AIs, whereas with human brain we are more hard wired. So maybe we are forced to be more balanced and general whereas AI don't have to.
  - verdverm 9mo ago
    I suspect the non-spikey part is the more interesting comparison Why is it so easy for me to open the car door, get in, close the door, buckle up. You can do this in the dark and without looking. There are an infinite number of little things like this you think zero about, take near zero energy, yet which are extremely hard for Ai
    - gowld 9mo ago
      You are asking a robotics question, not an AI question. Robotics is more and less than AI. Boston Dynamics robots are getting quite near your benchmark.
      - idiotsecant 9mo ago
        Boston dynamics is missing just about all the degrees of freedom involved in the scenario op mentions.
    - pixl97 9mo ago
      >Why is it so easy for me to open the car door Because this part of your brain has been optimized for hundreds of millions of years. It's been around a long ass time and takes an amazingly low amount of energy to do these things. On the other hand the 'thinking' part of your brain, that is your higher intelligence is very new to evolution. It's expensive to run. It's problematic when giving birth. It's really slow with things like numbers, heck a tiny calculator and whip your butt in adding. There's a term for this, but I can't think of it at the moment.
      - te7447 9mo ago
        > There's a term for this, but I can't think of it at the moment. Moravec's paradox: https://epoch.ai/gradient-updates/moravec-s-paradox https://epoch.ai/gradient-updates/moravec-s-paradox
        pixl97 9mo ago
        Thanks, I can never quite remember that.
tasuki 9mo ago
> maybe there's intelligence in there, but hardly general. Of course. Just as our human intelligence isn't general.