7 ms·

For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what m

For me, the entire AGI conversation is hyperbolic / hype. How can we infer intelligence to something when we, ourselves, have such a poor (none) grasp of what makes us conscience? I'm associating intelligence with consciousness - because it seems correlated. Are we really ready to associate "AGI" with solving math problems ("new Q algo.")? That seems incredibly naive & reinforces my opinion that LLM's are much more like crypto, than actual progress.

poulsbohemian 3y ago
Completely agree, and while we are at it... look I'm just a guy, not an expert, but I can't understand why there's so much focus on AGI. It feels like there are so many niche areas where we could apply some kind of analytical augmentation and by solving problems in the small, might learn something that would help figure the larger question of intelligence. I don't need the AI to replace everything I do, I need it to solve 10,000 micro problems I solve every day - each of which is a business opportunity for someone.
- sgregnt 3y ago
  Many of the seemingly small problems do require a good model of the world for context and edge case solving, so they still get very close to general intelegence.
  - pixl97 3y ago
    Yep, at least in my eyes you'll never be able to "solve" self driving without solving the G in AGI. You require a world model for predictions in order to have enough time to avoid many bad outcomes. Avoiding an empty soda can and avoiding a brick are similar problems, but one can easily lead to critical failures if you miss it.
- Avicebron 3y ago
  I've thought about this a bit as well, and I think it's almost like this toxic concoction of incentive (how can "we" hype this until and make boatloads of money off of it, coupled with a genuine (if sub-conscious) desire to be seen as a visionary/great engineer who "created artificial life." I mean, at least on HN, I see lots of this aspirational attitude for living the sci-fi future circa. Star trek, ex-machina, etc, while couching their language in professions of expertise now that the firehose of cash has turned on. Also there is the general hubris in all this to only look at the new and shiny, I remember when there was that pizza robot (some multi-dimension axis hand thing) that cost whatever in building and research, when the costco pizza "robot" is pretty darn good, but doesn't sell as "futuristic/cool" because its a spigot on a servo.
- tivert 3y ago
  > ...but I can't understand why there's so much focus on AGI. Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy. It's interesting to see that in people who often view themselves as hyper-rational.
  - tim333 3y ago
    AGI as in computer intelligence that out does humans would be a huge deal in practical terms. Chat GPT and similar are kind of like handy toys. With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. It's a different category of thing. And if you think that's just sci-fi I think you'll get a surprise at some point during your life.
    - tivert 3y ago
      > With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when you have my hyperdrive ready.
      - tim333 3y ago
        I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.
        tivert 3y ago
        > I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.
- pixl97 3y ago
  >I need it to solve 10,000 micro problems I solve every day - each of which is a business opportunity for someone. Because you have to solve 10,000 different problems. And a huge number of those problems are going to have significant overlap, but sharing lessons between them is going to be difficult unless you have a generalized algorithm. Hence AGI is the trillion dollar question.
deleted 3y ago
[deleted]
RGamma 3y ago
A(G)I models don't need higher order thinking or somesuch to be impactful. For that they just need to increase productivity with or without job loss (be Good Enough), which they are on a good track for. The real impacts will come when they are properly integrated into the current computational fabric, which everyone is racing to do as we write this.
nyrikki 3y ago
A particular subset of Connectivism have a philosophical belief that the mind IS a neutral net, not that it is a reductive practical model. Hinton is one of these individuals and with no definition of what intelligence is it is an understandable of dogmatic position. This whole problem of not being able to define what intelligence is pretty much allows us all to pick and choose. In my mind BPP is the complexity class solvable by ANNs and it is a safe and educated guess that most likely BPP=P. BPP being one of the largest practical complexity classes makes work in this area valuable. But due to many reasons that I won't enumerate again AGI simply isn't possible and requires a dogmatic position to believe in for people who have even a basic understanding of how they work and the limits from the work of Gödel etc... But many of the top scientists in history have been believers of numerology etc... Associating math with LLMs is a useful too to avoid wasted effort by those who don't believe AGI is close, but it won't convince those who are true believers. LLM's are very useful for searching very large dimensional spaces and for those problems that are ergotic with the Markov property they can find real answers. But for most of what is popular in the press will almost certainly be a dead end for generalized use of the systems are not extremely error tolerant. Unfortunately it may take another AI winter to break the hype train but I hope not. IMHO it will have a huge impact but overconfident claims will cause real pain and misapplication for the foreseeable future.
- blovescoffee 3y ago
  You should state what AGI is and then state why it’s nog possible.
upghost 3y ago
Couldn't agree more. How about this -- I think we've already reached AGI. Let me know if this tracks: Pick a set of tasks that can be considered AGI tasks. Provided the task sequences can be compared as closer to AGI or further from AGI, we can create a reward model using the same techniques as were used by ChatGPT via RLHF. Thus, for any definition of AGI that is meaningful and selectable, even if subjectively selectable or arbitrarily preferential, we can create a reward model for it. You might say, well thats not AGI, AGI must also do such and such. Well, we can get arbitrarily close to that definition as well via RLHF. Another objection might be: well, if thats the definition of AGI, that seems really underwhelming compared to the hype train. This says nothing about autonomy, sentience, free will -- exactly. Those concepts can or should be orthogonal to doing productive work.l, IMHO. So, there it is. We can now make a reward model for folding socks, and use gradient descent with RL to do the motion planning. Maybe thats AGI and maybe its not, but I'd really love it if we had a golden period between now and total enshittification that involved laundry folding robots.
- AnimalMuppet 3y ago
  Well... humans have different mental "spaces" (not intended as a technical term). Let's say I'm deep in a coding problem. A co-worker comes by and says "How did your team do in the game yesterday?". I say, "Um, uh... sorry, my head's not there right now." It takes us time to swap between mental "spaces". So, if I have an AGI (defined as having a trained model for almost everything, even if that turns out to be a large number of different models), if it has to load the right model before it can reason on that topic, then that's pretty human-like. (As long as it can figure out which model to load...) The one thing missing is that (at least some) humans can figure out linkages between different mental "spaces", to turn them into a more coherent holistic mental space, even if they don't have (all of) each space at front-of-mind at any moment. I'm not sure if this flavor of an AGI could do that - could see the connections between different models.
  - pixl97 3y ago
    The power of analogy is one of the most important things that humans seem to have. Humans typically use the toolset they've seen along the way to solve problems (hence if you have a hammer all problems become nails statement). When you get people that are multi-disciplinary they commonly can solve a complex problem in one field by bringing parts of solutions from other fields. Hence if you have more life experiences (especially positive/learning ones) you are typically better off then a person who does not. Also I think this is where a lot of interest in Q* learning after the OpenAI thing occurred, as this would be a means of allowing an AI to explore problem spaces and enlist specialist AI and tools for it to do so.
corethree 3y ago
It's not hype. It's a language problem that makes people like you think this way. The problem is consciousness is a vocabulary word that establishes a hard boundary where such a boundary doesn't exit. The language makes you think either something is conscious or it is not when the reality is that these two concepts are actually extreme endpoints on a gradient. The vocabulary makes the concept seem binary and makes it seem more profound then it actually is. Thus we have no problem identifying things at the extreme. A rock is not conscious. That's obvious. A human IS conscious, that's also obvious. But only because these two objects are defined at the extremes of this gradient. For something fuzzy like chatGPT, we get confused. We think the problem is profound, but in actuality it's just poorly defined vocabulary. The word consciousness, again, assumes the world is binary that something is either/or, but, again, the reality is a gradient. When we have debates about whether something is "conscious" or not we are just arguing about where the line of demarcation is drawn along the gradient. Does it need a body to be conscious? Does it need to be able to do math? Where you draw this line is just a definition of vocabulary. So arguments about whether LLMs are conscious are arguments about vocabulary. We as humans are biased and we blindly allow the vocabulary to mold our thinking. Is chatGPT conscious? It's a loaded question based on a world view manipulated by the vocabulary. It doesn't even matter. That boundary is fuzzy, and any vocab attempting to describe this gradient is just arbitrary. But hear me out. chatGPT and DALL-E is NOT hype. Why? Because along that gradient it's leaps and bounds further than anything we had just even a decade ago. It's the closest we ever been to the extreme endpoint. Whichever side you are on in the great debate both sides can very much agree with this logic.
- nprateem 3y ago
  > A rock is not conscious. That's obvious. But it's not obvious at all. It may possess consciousness in a way we can't relate to or communicate with. This is the whole problem with consciousness and has been discussed by philosophers for centuries. We each appear to be conscious but can't be certain anything else is or isn't.
  - corethree 3y ago
    You're not reading my argument. You responded to one part of it. As a whole my argument is not about the problem you're describing. My argument is saying that the problem is a sham. An illusion. The problem doesn't even exist. Read my whole write up.
StimpsonJHoek 3y ago
[flagged]

poulsbohemian 3y ago

Completely agree, and while we are at it... look I'm just a guy, not an expert, but I can't understand why there's so much focus on AGI. It feels like there are so many niche areas where we could apply some kind of analytical augmentation and by solving problems in the small, might learn something that would help figure the larger question of intelligence. I don't need the AI to replace everything I do, I need it to solve 10,000 micro problems I solve every day - each of which is a business opportunity for someone.

sgregnt 3y ago
Many of the seemingly small problems do require a good model of the world for context and edge case solving, so they still get very close to general intelegence.
- pixl97 3y ago
  Yep, at least in my eyes you'll never be able to "solve" self driving without solving the G in AGI. You require a world model for predictions in order to have enough time to avoid many bad outcomes. Avoiding an empty soda can and avoiding a brick are similar problems, but one can easily lead to critical failures if you miss it.
Avicebron 3y ago
I've thought about this a bit as well, and I think it's almost like this toxic concoction of incentive (how can "we" hype this until and make boatloads of money off of it, coupled with a genuine (if sub-conscious) desire to be seen as a visionary/great engineer who "created artificial life." I mean, at least on HN, I see lots of this aspirational attitude for living the sci-fi future circa. Star trek, ex-machina, etc, while couching their language in professions of expertise now that the firehose of cash has turned on. Also there is the general hubris in all this to only look at the new and shiny, I remember when there was that pizza robot (some multi-dimension axis hand thing) that cost whatever in building and research, when the costco pizza "robot" is pretty darn good, but doesn't sell as "futuristic/cool" because its a spigot on a servo.
tivert 3y ago
> ...but I can't understand why there's so much focus on AGI. Lots of software engineers have spent their lives reading sci-fi that features AGI, and they're excited by/lost in that fantasy. It's interesting to see that in people who often view themselves as hyper-rational.
- tim333 3y ago
  AGI as in computer intelligence that out does humans would be a huge deal in practical terms. Chat GPT and similar are kind of like handy toys. With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. It's a different category of thing. And if you think that's just sci-fi I think you'll get a surprise at some point during your life.
  - tivert 3y ago
    > With proper AGI you could link it to a robot body and tell it to go off, design a better version of itself, make a billion more robots and take over the world. ^^^^^ literally a plot ripped from the pages of science fiction used to reason about the real world. > And if you think that's just sci-fi I think you'll get a surprise at some point during your life. Such faith that fantasy can be made real. Wake me up when you have my hyperdrive ready.
    - tim333 3y ago
      I've never actually read that in fiction. It's just logical really. I mean I'm sure it is in fiction somewhere because the idea is obvious.
      - tivert 3y ago
        > I've never actually read that in fiction. I find that hard to believe. Ever watch Terminator? But even if true, that science-fictional plot is so pervasive it would be easy to pick up from the millions who have the software engineer's blurry line between fantasy and reality. > It's just logical really. OK, then. You're a GI, go off and build an army of better yous and take over the world.
pixl97 3y ago
>I need it to solve 10,000 micro problems I solve every day - each of which is a business opportunity for someone. Because you have to solve 10,000 different problems. And a huge number of those problems are going to have significant overlap, but sharing lessons between them is going to be difficult unless you have a generalized algorithm. Hence AGI is the trillion dollar question.

deleted 3y ago

[deleted]

RGamma 3y ago

A(G)I models don't need higher order thinking or somesuch to be impactful. For that they just need to increase productivity with or without job loss (be Good Enough), which they are on a good track for. The real impacts will come when they are properly integrated into the current computational fabric, which everyone is racing to do as we write this.

nyrikki 3y ago

A particular subset of Connectivism have a philosophical belief that the mind IS a neutral net, not that it is a reductive practical model. Hinton is one of these individuals and with no definition of what intelligence is it is an understandable of dogmatic position. This whole problem of not being able to define what intelligence is pretty much allows us all to pick and choose. In my mind BPP is the complexity class solvable by ANNs and it is a safe and educated guess that most likely BPP=P. BPP being one of the largest practical complexity classes makes work in this area valuable. But due to many reasons that I won't enumerate again AGI simply isn't possible and requires a dogmatic position to believe in for people who have even a basic understanding of how they work and the limits from the work of Gödel etc... But many of the top scientists in history have been believers of numerology etc... Associating math with LLMs is a useful too to avoid wasted effort by those who don't believe AGI is close, but it won't convince those who are true believers. LLM's are very useful for searching very large dimensional spaces and for those problems that are ergotic with the Markov property they can find real answers. But for most of what is popular in the press will almost certainly be a dead end for generalized use of the systems are not extremely error tolerant. Unfortunately it may take another AI winter to break the hype train but I hope not. IMHO it will have a huge impact but overconfident claims will cause real pain and misapplication for the foreseeable future.

blovescoffee 3y ago
You should state what AGI is and then state why it’s nog possible.

upghost 3y ago

Couldn't agree more. How about this -- I think we've already reached AGI. Let me know if this tracks: Pick a set of tasks that can be considered AGI tasks. Provided the task sequences can be compared as closer to AGI or further from AGI, we can create a reward model using the same techniques as were used by ChatGPT via RLHF. Thus, for any definition of AGI that is meaningful and selectable, even if subjectively selectable or arbitrarily preferential, we can create a reward model for it. You might say, well thats not AGI, AGI must also do such and such. Well, we can get arbitrarily close to that definition as well via RLHF. Another objection might be: well, if thats the definition of AGI, that seems really underwhelming compared to the hype train. This says nothing about autonomy, sentience, free will -- exactly. Those concepts can or should be orthogonal to doing productive work.l, IMHO. So, there it is. We can now make a reward model for folding socks, and use gradient descent with RL to do the motion planning. Maybe thats AGI and maybe its not, but I'd really love it if we had a golden period between now and total enshittification that involved laundry folding robots.

AnimalMuppet 3y ago
Well... humans have different mental "spaces" (not intended as a technical term). Let's say I'm deep in a coding problem. A co-worker comes by and says "How did your team do in the game yesterday?". I say, "Um, uh... sorry, my head's not there right now." It takes us time to swap between mental "spaces". So, if I have an AGI (defined as having a trained model for almost everything, even if that turns out to be a large number of different models), if it has to load the right model before it can reason on that topic, then that's pretty human-like. (As long as it can figure out which model to load...) The one thing missing is that (at least some) humans can figure out linkages between different mental "spaces", to turn them into a more coherent holistic mental space, even if they don't have (all of) each space at front-of-mind at any moment. I'm not sure if this flavor of an AGI could do that - could see the connections between different models.
- pixl97 3y ago
  The power of analogy is one of the most important things that humans seem to have. Humans typically use the toolset they've seen along the way to solve problems (hence if you have a hammer all problems become nails statement). When you get people that are multi-disciplinary they commonly can solve a complex problem in one field by bringing parts of solutions from other fields. Hence if you have more life experiences (especially positive/learning ones) you are typically better off then a person who does not. Also I think this is where a lot of interest in Q* learning after the OpenAI thing occurred, as this would be a means of allowing an AI to explore problem spaces and enlist specialist AI and tools for it to do so.

corethree 3y ago

It's not hype. It's a language problem that makes people like you think this way. The problem is consciousness is a vocabulary word that establishes a hard boundary where such a boundary doesn't exit. The language makes you think either something is conscious or it is not when the reality is that these two concepts are actually extreme endpoints on a gradient. The vocabulary makes the concept seem binary and makes it seem more profound then it actually is. Thus we have no problem identifying things at the extreme. A rock is not conscious. That's obvious. A human IS conscious, that's also obvious. But only because these two objects are defined at the extremes of this gradient. For something fuzzy like chatGPT, we get confused. We think the problem is profound, but in actuality it's just poorly defined vocabulary. The word consciousness, again, assumes the world is binary that something is either/or, but, again, the reality is a gradient. When we have debates about whether something is "conscious" or not we are just arguing about where the line of demarcation is drawn along the gradient. Does it need a body to be conscious? Does it need to be able to do math? Where you draw this line is just a definition of vocabulary. So arguments about whether LLMs are conscious are arguments about vocabulary. We as humans are biased and we blindly allow the vocabulary to mold our thinking. Is chatGPT conscious? It's a loaded question based on a world view manipulated by the vocabulary. It doesn't even matter. That boundary is fuzzy, and any vocab attempting to describe this gradient is just arbitrary. But hear me out. chatGPT and DALL-E is NOT hype. Why? Because along that gradient it's leaps and bounds further than anything we had just even a decade ago. It's the closest we ever been to the extreme endpoint. Whichever side you are on in the great debate both sides can very much agree with this logic.

nprateem 3y ago
> A rock is not conscious. That's obvious. But it's not obvious at all. It may possess consciousness in a way we can't relate to or communicate with. This is the whole problem with consciousness and has been discussed by philosophers for centuries. We each appear to be conscious but can't be certain anything else is or isn't.
- corethree 3y ago
  You're not reading my argument. You responded to one part of it. As a whole my argument is not about the problem you're describing. My argument is saying that the problem is a sham. An illusion. The problem doesn't even exist. Read my whole write up.

StimpsonJHoek 3y ago

[flagged]