11 ms·

> defining AGI as matching the cognitive versatility and proficiency of a well-educated adult I don't think people really realize how extraordinary accomplishm

by flkiwi 1y ago

> defining AGI as matching the cognitive versatility and proficiency of a well-educated adult I don't think people really realize how extraordinary accomplishment it would be to have an artificial system matching the cognitive versatility and proficiency of an uneducated child, much less a well-educated adult. Hell, AI matching the intelligence of some nonhuman animals would be an epoch-defining accomplishment.

cbdevidal 1y ago
Have any benchmarks been made that use this paper’s definition? I follow the ARC prize and Humanity’s Last Exam, but I don’t know how closely they would map to this paper’s methods. Edit: Probably not, since it was published less than a week ago :-) I’ll be watching for benchmarks.
surgical_fire 1y ago
There are some sycophants that claim that LLMs can operate at Junior Enginee level. Try to reconcile that with your ideas (that I think are correct for that matter)
- ben_w 1y ago
  I'll simultaneously call all current ML models "stupid" and also say that SOTA LLMs can operate at junior (software) engineer level. This is because I use "stupidity" as the number of examples some intelligence needs in order to learn from, while performance is limited to the quality of the output. LLMs *partially* make up for being too stupid to live (literally: no living thing could survive if it needed so many examples) by going through each example faster than any living thing ever could — by as many orders of magnitude as there are between jogging and continental drift.
  - ACCount37 1y ago
    Data-efficiency matters, but compute-efficiency matters too. LLMs have a reasonable learning rate at inference time (in-context learning is powerful), but a very poor learning rate in pretraining. And one issue with that is that we have an awful lot of cheap data to pretrain those LLMs with. We don't know how much compute human brain uses to do what it does. And if we could pretrain with the same data-efficiency as humans, but at the cost of using x10000 the compute for it? It would be impossible to justify doing that for all but the most expensive, hard-to-come-by gold-plated datasets - ones that are actually worth squeezing every drop of performance gains out from.
    - noir_lord 1y ago
      We do know how much energy a human brain uses to do whatever it does though. That it takes vast power to train the LLM’s (and run them) to not get intelligence is pretty bad when you compare the energy inputs to the outcomes.
      - ben_w 1y ago
        Energy is even weirder. Global electricity supply is about 3 TW/8 billion people, 375 W/person, vs the 100-124 W/person of our metabolism. Given how much cheaper electricity is than food, AI can be much worse Joules for the same outcome, while still being good enough to get all the electricity.
        sudosysgen 1y ago
        Rice is 45 cents per kg in bulk, and contains the equivalent of 4kWh. Electricity is not actually much cheaper than food, if at all.
  - card_zero 1y ago
    (10 orders of magnitude, it works out neatly as 8km/h for a fast jogger against 0.0008 mm/h for the East African Rift.)
  - JumpCrisscross 1y ago
    If you’re a shop that churns through juniors, LLMs may match that. If you retain them for more than a year, you rapidly see the difference. Both personally and in the teams that develop an LLM addiction versus those who use it to turbocharge innate advantages.
    - ben_w 1y ago
      For good devs, sure. Even for okay devs. I have had the unfortunate experience of having to work with people who have got a lot more than one year experience who are still worse than last year's LLMs, who didn't even realise they were bad at what they did.
ben_w 1y ago
Or even to come up with a definition of cognitive versatility and proficiency that is good enough to not get argued away once we have an AI which technically passes that specific definition. The Turing Test was great until something that passed it (with an average human as interrogator) turned out to also not be able to count letters in a word — because only a special kind of human interrogator (the "scientist or QA" kind) could even think to ask that kind of question.
- lumost 1y ago
  Or that this system would fail to adapt in anyway to changes of circumstance. The adaptive intelligence of a live human is truly incredible. Even in cases where the weights are updatable, We watch AI make the same mistake thousands of times in an RL loop before attempting a different strategy.
- alganet 1y ago
  Absolute definitions are weak. They won't settle anything. We know what we need right now, the next step. That step is a machine that, when it fails, it fails in a human way. Humans also make mistakes, and hallucinate. But we do it as humans. When a human fails, you think "damn, that's a mistake perhaps me or my friend could have done". LLMs on the other hand, fail in a weird way. When they hallucinate, they demonstrate how non-human they are. It has nothing to do with some special kind of interrogator. We must assume the best human interrogator possible. This next step I described work even with the most skeptic human interrogator possible. It also synergizes with the idea of alignment in ways other tests don't. When that step is reached, humans will or will not figure out another characteristic that makes it evident that "subject X" is a machine and not a human, and a way to test it. Moving the goalpost is the only way forward. Not all goalpost moves are valid, but the valid next move is a goalpost move. It's kind of obvious.
  - AstroBen 1y ago
    This makes sense if we're trying to recreate a human mind artifically, but I don't think that's the goal? There's no reason an equivalent or superior general intelligence needs to be similar to us at all
NedF 1y ago
[dead]
ninetyninenine 1y ago
AI is highly educated. It's a different sort of artifact we're dealing with where it can't tell truth from fiction. What's going on is AI fatigue. We see it everywhere, we use it all the time. It's becoming generic and annoying and we're getting bored of it EVEN though the accomplishment is through the fucking roof. If elon musk makes interstellar car that can reach the nearest star in 1 second and priced it at 1k, I guarantee within a year people will be bored of it and finding some angle to criticize it. So what happens is we get fatigued, and then we have such negative emotions about it that we can't possibly classify it as the same thing as human intelligence. We magnify the flaws and until it takes up all the space and we demand a redefinition of what agi is because it doesn't "feel" right. We already had a definition of AGI. We hit it. We moved the goal posts because we weren't satisfied. This cycle is endless. The definition of AGI will always be changing. Take LLMs as they exist now and only allow 10% of the population to access it. Then the opposite effect will happen. The good parts will be over magnified and the bad parts will be acknowledged and then subsequently dismissed. Think about it. All the AI slop we see on social media are freaking masterpieces works of art produced in minutes what most humans can't even hope to come close to. Yet we're annoyed and unimpressed by them. That's how it's always going to go down.
- poopiokaka 1y ago
  [dead]
- Forgeties79 1y ago
  > EVEN though the accomplishment is through the fucking roof. I agree with this but also, the output is almost entirely worthless if you can’t vet it with your own knowledge and experience because it routinely gives you large swaths of incorrect info. Enough that you can’t really use the output unless you can find the inevitable issues. If I had to put a number to it, I would say 30% of what an LLM spits out at any given time to me is completely bullshit or at best irrelevant. 70% is very impressive, but still, it presents major issues. That’s not boredom, that’s just acknowledging the limitations. It’s like designing an engine or power source that has incredible efficiency but doesn’t actually move or affect anything (not saying LLM’s are worthless but bear with me). It just outputs with no productive result. I can be impressed with the achievement while also acknowledging it has severe limitations
zulban 1y ago
Why don't you think people realize that? I must have heard this basic talking point a hundred times.
- frank_nitti 1y ago
  For me, it would be because the term AGI gets bandied about a lot more frequently in discussions involving Gen AI, as if that path takes us any closer to AGI than other threads in the AI field have.
- dsjoerg 1y ago
  Their people are different from your people.
  - shermantanktop 1y ago
    It turns out that all our people are different, and each of us belongs to some other people’s people.
- VerminOctopus1 1y ago
  Because the amount of people stating that AGI is just around the corner is staggering. These people have no conception of what they are talking about.
  - suprjami 1y ago
    But they do. They're not talking about AGI, they're talking about venture capital funding.
    - pankajdoharey 1y ago
      Exactly. It sure is around the corner, because they are talking about AGI (Actually Getting Investments).
andy99 1y ago
I think the bigger issue is people confusing impressive but comparatively simpler achievements (everything current LLMs do) with anything remotely near the cognitive versatility of any human.
- mikepurvis 1y ago
  But the big crisis right now is that for an astonishing number of tasks that a normal person could come up with, chatgpt.com is actually a good at or better than a typical human. If you took the current state of affairs back to the 90s you’d quickly convince most people that we’re there. Given that we’re actually not, we’re now have to come up with new goalposts.
  - p1esk 1y ago
    Exactly. Five years ago I posted here on HN that AI will pass Turing Test in the next 3 years (I was impressed by Facebook chatbot progress at the time). I was laughed at and downvoted into oblivion. TT was seen by many as a huge milestone, incredibly difficult task, “maybe in my lifetime” possibility.
    - photonthug 1y ago
      Turing test isn't actually a good test of much, but even so, we're not there yet. Anyone that thinks we've passed it already should experiment a bit a with counter-factuals. Ask your favorite SOTA model to assume something absurd and then draw the next logical conclusions based on that. "Green is yellow and yellow is green. What color is a banana?" They may get the first question(s) right, but will trip up within a few exchanges. Might be a new question, but often they are very happy to just completely contradict their own previous answers. You could argue that this is hitting alignment and guard-rails against misinformation.. but whatever the cause, it's a clear sign it's a machine and look, no em-dashes. Ironically it's also a failure of the turing test that arises from a failure in reasoning at a really basic level, which I would not have expected. Makes you wonder about the secret sauce for winning IMO competitions. Anyway, unlike other linguistic puzzles that attempt to baffle with ambiguous reference or similar, simple counterfactuals with something like colors are particular interesting because they would NOT trip up most ESL students or 3-5 year olds.
Grimblewald 1y ago
I always laugh these, why are people always jumping to defining AGI when they clearly don't have a functional definition for the I part yet? More to the point, once you have the I part you get the G part, it is a fundamental part of it.
interstice 1y ago
What I think is being skipped in the current conversation is that versatility keyword is hiding a lot of unknowns - even now. We don't seem to have a true understanding of the breadth or depth of our own unconscious thought processes, therefore we don't have much that is concrete to start with.
hopelite 1y ago
I’m more surprised and equally concerned that the majority of people’s understanding of intelligence and their definition of AGI. Not only does the definition “… matching the cognitive versatility and proficiency of a well-educated adult.”, by definition violate the “general” in AGI, by the “well educated” part; but it also implies that only the “well-educated” (presumably by a specific curriculum) qualifies one as intelligent and by definition also once you depart from the “well” of the “educated” you exponentially diverge from “intelligent”. It all seems rather unimpressive intelligence. In other words; in one question; is the current AI not already well beyond the “…cognitive versatility and proficiency of an uneducated child”? And when you consider that in many places like Africa, they didn’t even have a written language until European evangelists created it and taught it to them in the late 19th century, and they have far less “education” than even some of the most “uneducated” avg., European and even many American children, does that not mean that AI is well beyond them at least? Frankly, as it seems things are going, there Is at the very least going to be a very stark shift in “intelligence” that even exceeds that which has happened in the last 50 or so years that have brought us stark drops in memory, literary knowledge, mathematics, and even general literacy, not to mention the ability to write. What does it mean that kids now will not even have to feign acting like they’re selling out sources, vetting them, contradicting a story or logical sequence, forming ideas, messages, and stories, etc.? I’m not trying to be bleak, but I don’t see tons simply resulting in net positive outcomes, and most of the negative impacts will also be happening below the surface to the point that people won’t realize what is being lost.
pankajdoharey 1y ago
People are specialists not generalists, creating a AI that is generalist and claiming it to have cognitive abilities the same as an "well-educated" adult is an oxymoron. And if such system could ever be made My guess is it wont be more than a few (under 5) Billion Parameter model that is very good at looking up stuff online, forgetting stuff when not in use , planning and creating or expanding the knowledge in its nodes. Much like a human adult would. It will be highly sa mple efficient, It wont know 30 languages (although it has been seen that models generalize better with more languages), it wont know entire wikipedia by heart , it even wont remember minor details of programming languages and stuff. Now that is my definition of an AGI.
grantcas 1y ago
[dead]

by flkiwi 1y ago

cbdevidal 1y ago

Have any benchmarks been made that use this paper’s definition? I follow the ARC prize and Humanity’s Last Exam, but I don’t know how closely they would map to this paper’s methods. Edit: Probably not, since it was published less than a week ago :-) I’ll be watching for benchmarks.

surgical_fire 1y ago

There are some sycophants that claim that LLMs can operate at Junior Enginee level. Try to reconcile that with your ideas (that I think are correct for that matter)

ben_w 1y ago
I'll simultaneously call all current ML models "stupid" and also say that SOTA LLMs can operate at junior (software) engineer level. This is because I use "stupidity" as the number of examples some intelligence needs in order to learn from, while performance is limited to the quality of the output. LLMs *partially* make up for being too stupid to live (literally: no living thing could survive if it needed so many examples) by going through each example faster than any living thing ever could — by as many orders of magnitude as there are between jogging and continental drift.
- ACCount37 1y ago
  Data-efficiency matters, but compute-efficiency matters too. LLMs have a reasonable learning rate at inference time (in-context learning is powerful), but a very poor learning rate in pretraining. And one issue with that is that we have an awful lot of cheap data to pretrain those LLMs with. We don't know how much compute human brain uses to do what it does. And if we could pretrain with the same data-efficiency as humans, but at the cost of using x10000 the compute for it? It would be impossible to justify doing that for all but the most expensive, hard-to-come-by gold-plated datasets - ones that are actually worth squeezing every drop of performance gains out from.
  - noir_lord 1y ago
    We do know how much energy a human brain uses to do whatever it does though. That it takes vast power to train the LLM’s (and run them) to not get intelligence is pretty bad when you compare the energy inputs to the outcomes.
    - ben_w 1y ago
      Energy is even weirder. Global electricity supply is about 3 TW/8 billion people, 375 W/person, vs the 100-124 W/person of our metabolism. Given how much cheaper electricity is than food, AI can be much worse Joules for the same outcome, while still being good enough to get all the electricity.
      - sudosysgen 1y ago
        Rice is 45 cents per kg in bulk, and contains the equivalent of 4kWh. Electricity is not actually much cheaper than food, if at all.
- card_zero 1y ago
  (10 orders of magnitude, it works out neatly as 8km/h for a fast jogger against 0.0008 mm/h for the East African Rift.)
- JumpCrisscross 1y ago
  If you’re a shop that churns through juniors, LLMs may match that. If you retain them for more than a year, you rapidly see the difference. Both personally and in the teams that develop an LLM addiction versus those who use it to turbocharge innate advantages.
  - ben_w 1y ago
    For good devs, sure. Even for okay devs. I have had the unfortunate experience of having to work with people who have got a lot more than one year experience who are still worse than last year's LLMs, who didn't even realise they were bad at what they did.

ben_w 1y ago

Or even to come up with a definition of cognitive versatility and proficiency that is good enough to not get argued away once we have an AI which technically passes that specific definition. The Turing Test was great until something that passed it (with an average human as interrogator) turned out to also not be able to count letters in a word — because only a special kind of human interrogator (the "scientist or QA" kind) could even think to ask that kind of question.

lumost 1y ago
Or that this system would fail to adapt in anyway to changes of circumstance. The adaptive intelligence of a live human is truly incredible. Even in cases where the weights are updatable, We watch AI make the same mistake thousands of times in an RL loop before attempting a different strategy.
alganet 1y ago
Absolute definitions are weak. They won't settle anything. We know what we need right now, the next step. That step is a machine that, when it fails, it fails in a human way. Humans also make mistakes, and hallucinate. But we do it as humans. When a human fails, you think "damn, that's a mistake perhaps me or my friend could have done". LLMs on the other hand, fail in a weird way. When they hallucinate, they demonstrate how non-human they are. It has nothing to do with some special kind of interrogator. We must assume the best human interrogator possible. This next step I described work even with the most skeptic human interrogator possible. It also synergizes with the idea of alignment in ways other tests don't. When that step is reached, humans will or will not figure out another characteristic that makes it evident that "subject X" is a machine and not a human, and a way to test it. Moving the goalpost is the only way forward. Not all goalpost moves are valid, but the valid next move is a goalpost move. It's kind of obvious.
- AstroBen 1y ago
  This makes sense if we're trying to recreate a human mind artifically, but I don't think that's the goal? There's no reason an equivalent or superior general intelligence needs to be similar to us at all

NedF 1y ago

[dead]

ninetyninenine 1y ago

AI is highly educated. It's a different sort of artifact we're dealing with where it can't tell truth from fiction. What's going on is AI fatigue. We see it everywhere, we use it all the time. It's becoming generic and annoying and we're getting bored of it EVEN though the accomplishment is through the fucking roof. If elon musk makes interstellar car that can reach the nearest star in 1 second and priced it at 1k, I guarantee within a year people will be bored of it and finding some angle to criticize it. So what happens is we get fatigued, and then we have such negative emotions about it that we can't possibly classify it as the same thing as human intelligence. We magnify the flaws and until it takes up all the space and we demand a redefinition of what agi is because it doesn't "feel" right. We already had a definition of AGI. We hit it. We moved the goal posts because we weren't satisfied. This cycle is endless. The definition of AGI will always be changing. Take LLMs as they exist now and only allow 10% of the population to access it. Then the opposite effect will happen. The good parts will be over magnified and the bad parts will be acknowledged and then subsequently dismissed. Think about it. All the AI slop we see on social media are freaking masterpieces works of art produced in minutes what most humans can't even hope to come close to. Yet we're annoyed and unimpressed by them. That's how it's always going to go down.

poopiokaka 1y ago
[dead]
Forgeties79 1y ago
> EVEN though the accomplishment is through the fucking roof. I agree with this but also, the output is almost entirely worthless if you can’t vet it with your own knowledge and experience because it routinely gives you large swaths of incorrect info. Enough that you can’t really use the output unless you can find the inevitable issues. If I had to put a number to it, I would say 30% of what an LLM spits out at any given time to me is completely bullshit or at best irrelevant. 70% is very impressive, but still, it presents major issues. That’s not boredom, that’s just acknowledging the limitations. It’s like designing an engine or power source that has incredible efficiency but doesn’t actually move or affect anything (not saying LLM’s are worthless but bear with me). It just outputs with no productive result. I can be impressed with the achievement while also acknowledging it has severe limitations

zulban 1y ago

Why don't you think people realize that? I must have heard this basic talking point a hundred times.

frank_nitti 1y ago
For me, it would be because the term AGI gets bandied about a lot more frequently in discussions involving Gen AI, as if that path takes us any closer to AGI than other threads in the AI field have.
dsjoerg 1y ago
Their people are different from your people.
- shermantanktop 1y ago
  It turns out that all our people are different, and each of us belongs to some other people’s people.
VerminOctopus1 1y ago
Because the amount of people stating that AGI is just around the corner is staggering. These people have no conception of what they are talking about.
- suprjami 1y ago
  But they do. They're not talking about AGI, they're talking about venture capital funding.
  - pankajdoharey 1y ago
    Exactly. It sure is around the corner, because they are talking about AGI (Actually Getting Investments).

andy99 1y ago

I think the bigger issue is people confusing impressive but comparatively simpler achievements (everything current LLMs do) with anything remotely near the cognitive versatility of any human.

mikepurvis 1y ago
But the big crisis right now is that for an astonishing number of tasks that a normal person could come up with, chatgpt.com is actually a good at or better than a typical human. If you took the current state of affairs back to the 90s you’d quickly convince most people that we’re there. Given that we’re actually not, we’re now have to come up with new goalposts.
- p1esk 1y ago
  Exactly. Five years ago I posted here on HN that AI will pass Turing Test in the next 3 years (I was impressed by Facebook chatbot progress at the time). I was laughed at and downvoted into oblivion. TT was seen by many as a huge milestone, incredibly difficult task, “maybe in my lifetime” possibility.
  - photonthug 1y ago
    Turing test isn't actually a good test of much, but even so, we're not there yet. Anyone that thinks we've passed it already should experiment a bit a with counter-factuals. Ask your favorite SOTA model to assume something absurd and then draw the next logical conclusions based on that. "Green is yellow and yellow is green. What color is a banana?" They may get the first question(s) right, but will trip up within a few exchanges. Might be a new question, but often they are very happy to just completely contradict their own previous answers. You could argue that this is hitting alignment and guard-rails against misinformation.. but whatever the cause, it's a clear sign it's a machine and look, no em-dashes. Ironically it's also a failure of the turing test that arises from a failure in reasoning at a really basic level, which I would not have expected. Makes you wonder about the secret sauce for winning IMO competitions. Anyway, unlike other linguistic puzzles that attempt to baffle with ambiguous reference or similar, simple counterfactuals with something like colors are particular interesting because they would NOT trip up most ESL students or 3-5 year olds.

Grimblewald 1y ago

I always laugh these, why are people always jumping to defining AGI when they clearly don't have a functional definition for the I part yet? More to the point, once you have the I part you get the G part, it is a fundamental part of it.

interstice 1y ago

What I think is being skipped in the current conversation is that versatility keyword is hiding a lot of unknowns - even now. We don't seem to have a true understanding of the breadth or depth of our own unconscious thought processes, therefore we don't have much that is concrete to start with.

hopelite 1y ago

I’m more surprised and equally concerned that the majority of people’s understanding of intelligence and their definition of AGI. Not only does the definition “… matching the cognitive versatility and proficiency of a well-educated adult.”, by definition violate the “general” in AGI, by the “well educated” part; but it also implies that only the “well-educated” (presumably by a specific curriculum) qualifies one as intelligent and by definition also once you depart from the “well” of the “educated” you exponentially diverge from “intelligent”. It all seems rather unimpressive intelligence. In other words; in one question; is the current AI not already well beyond the “…cognitive versatility and proficiency of an uneducated child”? And when you consider that in many places like Africa, they didn’t even have a written language until European evangelists created it and taught it to them in the late 19th century, and they have far less “education” than even some of the most “uneducated” avg., European and even many American children, does that not mean that AI is well beyond them at least? Frankly, as it seems things are going, there Is at the very least going to be a very stark shift in “intelligence” that even exceeds that which has happened in the last 50 or so years that have brought us stark drops in memory, literary knowledge, mathematics, and even general literacy, not to mention the ability to write. What does it mean that kids now will not even have to feign acting like they’re selling out sources, vetting them, contradicting a story or logical sequence, forming ideas, messages, and stories, etc.? I’m not trying to be bleak, but I don’t see tons simply resulting in net positive outcomes, and most of the negative impacts will also be happening below the surface to the point that people won’t realize what is being lost.

pankajdoharey 1y ago

People are specialists not generalists, creating a AI that is generalist and claiming it to have cognitive abilities the same as an "well-educated" adult is an oxymoron. And if such system could ever be made My guess is it wont be more than a few (under 5) Billion Parameter model that is very good at looking up stuff online, forgetting stuff when not in use , planning and creating or expanding the knowledge in its nodes. Much like a human adult would. It will be highly sa mple efficient, It wont know 30 languages (although it has been seen that models generalize better with more languages), it wont know entire wikipedia by heart , it even wont remember minor details of programming languages and stuff. Now that is my definition of an AGI.

grantcas 1y ago

[dead]