3 ms·

Ask 100 people what their definition of AGI is and you'll probably get 100 different answers. My definition is closer to DeepMind's "can do any [computer-based

Ask 100 people what their definition of AGI is and you'll probably get 100 different answers. My definition is closer to DeepMind's "can do any [computer-based] task that a human can do", and certainly a human intern can learn on the job and do better on day #2 than day #1. Learning is basically THE hallmark of intelligence. Being able to learn from experience and use that to do better in the future. If you could somehow take everything the LLM learnt today and "put it into" an update for tomorrow that would be better than nothing, but the intern on the job is probably learning dozens of things per day that stack upon each other, so that's going to be a S-L-O-W learning AI intern ! Leaving aside the technical issue how you could store a days learning, or what that would even mean, you've also got the privacy and merge issues if this were to be done in the cloud without every customer having their own personalized model. For real continual learning we need to get past gradient descent-based batch training and develop a new learning algorithm so the model itself learns incrementally as it runs (animal-like predict, observe, learn cycle), rather than being reliant on some external alien to come in, equipped with it's entire learning curriculum, and program its weights.

pixl97 2mo ago
OK, let's say we get continuous learning today, what does tomorrow look like? I won't argue that AI can do some learning as context sizes are still terribly small and expensive to iterate. What does the world like in a week? A month? A year? How long before these models drift to their own languages? To their own set of morals? To their own alignment? None of these questions are answered and I'd rather stay at AGI-lite until they are as having a billion agents going off in their own directions seems like a recipe for disaster. With current models and their general knowledge self learning just seems like you'd have a few ASIs crop up really quick.
- jaggederest 2mo ago
  I also think it's moving the goalpost significantly. If you asked me what AGI looks like in 2024, I would have said "smarter on tasks that can be done via text than the average random human you'd meet in a NY bodega", and by that standard, it's long in the rear view mirror. Now I think people are asking a different question, which is "better than any human at any task that can be done via text", which imo is superhuman, not artificial general intelligence.
  - HarHarVeryFunny 2mo ago
    It's not moving the goalposts - it's just having a different goal in the first place. This has been Shane Legg and Demis Hassabis' definition of AGI since DeepMind was founded, or before (way before OpenAI appeared), and also for me the goal of AI has always been human level (really a human brain, complete with emotions, etc, but we can start with basic learning/prediction).
    - jaggederest 2mo ago
      I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes. I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.
      - HarHarVeryFunny 1mo ago
        I'm not sure there is anything to disagree on here - just different definitions of what we personally associate the term "AGI" with. There is no magic to the name, and however you choose to define it there will be less and more capable levels of AI that precede and follow it. Separate from what you choose to call any given set of AI capabilities, learning is generally more than retaining change over time. Learning isn't the same as memorization - it's the difference between memorizing a stack of how-to books and going out and practicing the skill for yourself, and all that entails. A learnt skill needs to be internalized such that in the future it's what is driving behavior, which means it needs to be in the weights. Imagine a robot trying to learn to play tennis where rather than updating it's connections between perception and action it's trying to store everything as a bunch of notes! It's not just physical tasks where this applies - the same would be true to trying to teach a multimodal model to recognize different types of mushrooms from photos, or even learning a new language. If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors.
- HarHarVeryFunny 2mo ago
  Yeah, I'm fascinated by AGI, but not very enthusiastic about it. I've yet to see anyone, notably not even the people trying to build it, articulate a future where it seems to be a net positive for society. If it's a choice between everyone living on food stamps or reducing the infinite prime gap to 140, then I guess I'll be happy with a prime gap of whatever it was yesterday. If/when we do eventually build a more animal-like true human level AGI - build a brain, not just a language model, then potentially we'd be able to build in some of the safeguards that millions of years of co-evolution have built into ourselves, but I expect that is still decades away, and then it'd be capable of doing even more jobs! The best AI outcome I could wish for is where it is extremely heavily regulated, and AI replacing any human job paying under $1M/yr is banned.

pixl97 2mo ago

OK, let's say we get continuous learning today, what does tomorrow look like? I won't argue that AI can do some learning as context sizes are still terribly small and expensive to iterate. What does the world like in a week? A month? A year? How long before these models drift to their own languages? To their own set of morals? To their own alignment? None of these questions are answered and I'd rather stay at AGI-lite until they are as having a billion agents going off in their own directions seems like a recipe for disaster. With current models and their general knowledge self learning just seems like you'd have a few ASIs crop up really quick.

jaggederest 2mo ago
I also think it's moving the goalpost significantly. If you asked me what AGI looks like in 2024, I would have said "smarter on tasks that can be done via text than the average random human you'd meet in a NY bodega", and by that standard, it's long in the rear view mirror. Now I think people are asking a different question, which is "better than any human at any task that can be done via text", which imo is superhuman, not artificial general intelligence.
- HarHarVeryFunny 2mo ago
  It's not moving the goalposts - it's just having a different goal in the first place. This has been Shane Legg and Demis Hassabis' definition of AGI since DeepMind was founded, or before (way before OpenAI appeared), and also for me the goal of AI has always been human level (really a human brain, complete with emotions, etc, but we can start with basic learning/prediction).
  - jaggederest 2mo ago
    I guess I fundamentally disagree, because on general knowledge they beat any human, and on processing speed they beat any human, and on problem solving likewise. I just don't see a domain besides "retain change over time" that they fail on, and that is something you can do in the system rather than the model - coding agents can and do build their own memory system and use it, like an amnestic human writing notes. I think what I'm objecting to is the idea that if it can't do every task that every human can do better than any human, it's not AGI - I think we have to allow that this is a fundamentally different form of cognition, so requiring a 100% match feels uncharitable. Every human has cognitive gaps that AI doesn't have.
    - HarHarVeryFunny 1mo ago
      I'm not sure there is anything to disagree on here - just different definitions of what we personally associate the term "AGI" with. There is no magic to the name, and however you choose to define it there will be less and more capable levels of AI that precede and follow it. Separate from what you choose to call any given set of AI capabilities, learning is generally more than retaining change over time. Learning isn't the same as memorization - it's the difference between memorizing a stack of how-to books and going out and practicing the skill for yourself, and all that entails. A learnt skill needs to be internalized such that in the future it's what is driving behavior, which means it needs to be in the weights. Imagine a robot trying to learn to play tennis where rather than updating it's connections between perception and action it's trying to store everything as a bunch of notes! It's not just physical tasks where this applies - the same would be true to trying to teach a multimodal model to recognize different types of mushrooms from photos, or even learning a new language. If a system can truly learn for itself then it shouldn't need to be pre-trained with language, but rather should be able to learn by immersion and practice the same way we do, and the same way a dog learns to follow commands. You can't do this by leaving notes to yourself - the learning mechanism needs to be animal-like and update the systems weights in response to prediction failure - a runtime version of the way SGD-based pre-training updates weights in response to prediction errors.
HarHarVeryFunny 2mo ago
Yeah, I'm fascinated by AGI, but not very enthusiastic about it. I've yet to see anyone, notably not even the people trying to build it, articulate a future where it seems to be a net positive for society. If it's a choice between everyone living on food stamps or reducing the infinite prime gap to 140, then I guess I'll be happy with a prime gap of whatever it was yesterday. If/when we do eventually build a more animal-like true human level AGI - build a brain, not just a language model, then potentially we'd be able to build in some of the safeguards that millions of years of co-evolution have built into ourselves, but I expect that is still decades away, and then it'd be capable of doing even more jobs! The best AI outcome I could wish for is where it is extremely heavily regulated, and AI replacing any human job paying under $1M/yr is banned.