3 ms·

>1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? I think that an AI, a "generalized" intelligence with a generali

>1) Do you think that AI achieving superhuman cognitive abilities is unlikely/really far away? I think that an AI, a "generalized" intelligence with a generalized understanding, capable of performing every intellectual task, and adapting to new intellectual tasks, that a human being can do is certainly far away. >2) Do you believe that cognitive superiority is not a threat in general, or specifically when not embodied? Personally, I believe that "cognitive superiority" in a general sense is not what makes a real threat on an individual or on a global scale in either case. When that is achieved, so to speak. We already have so-called cognitive superiority in specialized tasks in many cases. It sure as shit is not generalized, and it still can kill you on an individual level when you put it into a car or strap a gun to it. But it can't adapt, and it can't generalize. Is there a global risk right now? No. The global catastrophic risk comes from human beings. Will there be a global catastrophic risk from a superior intelligence in the future? Mostly the scenarios are "the lights come on and AI does machiavellian shit and copies itself to everyone's smart phones and launches the nukes and mails anthrax to everybody. QED." so I doubt it. 3) Do you think we can trivially and indefinitely keep AI systems under control/"aligned"? We can't even keep our infrastructure safe from the next solar flare. Personally, I think we have bigger fish to fry. Speaking frankly for unembodied: you just pull the plug. Just walk away from the screen and close your eyes. For embodied intelligence, I don't think there's actually X-risk or whatever. Hardware is expensive. As far as an extinction risk, no. I don't personally believe that. It's overblown. We have better ways to destroy ourselves, and there are much more likelier ways that we'll be destroyed without invoking some intelligence of our own creation that subjugates and destroys us.

myrmidon 3y ago
> I think that an AI, a "generalized" intelligence with a generalized understanding [...] is certainly far away. I think this is where we disagree most; GPT-3 and ChatGPT have convinced me that the main difference between human and artificial cognitive capabilities are quantitative now, and unlikely to ever change in our favor... I do agree with you that it is very difficult to predict when this will switch, and how. I personally believe that AI with superhuman capability is an inevitable matter of time now, and I also think that the most likely risk to us, as species, is that we slowly become irrelevant and worthless, just like weavers during the industrial revolution, and that this leads to huge problems for our society. AI completely dominating humankind is a less likely secondary concern IMO, but the potential consequences to our species are unprecedented.
- srslack 3y ago
  Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet. It produces plausible language and code. It's the future of natural language agents, that's for sure, even though it has no business being the future, today (because of "prompt injection"). These failures are an aside to whether a generalized AI actually carries a substantial global catastrophic risk to us. In that case, if it were actually possible, I don't believe that it's a catastrophic risk either. Let's define the risk to be, at least in the context of LLMs, when junior developers expose internal APIs and databases and access to email accounts to these easily socially-engineered NLP front-ends. It's a very local risk. As far as the future, I can't see 70 years into the future, so anything is possible, but is it likely? I, personally, don't believe so.
  - BizarroLand 3y ago
    I don't think it's there or even necessarily close to being GAI. Within our own human brains, we have many sections dedicated to different tasks and processes that all must work together with years and decades of real world interaction to produce what we consider to be a generally intelligent human being, and even in the case of a certain percentage of us humans, even a small amount of damage to a single part of the brain can cause us to become functionally subhuman in our intelligence, barely able to move or eat on our own. A human that has been lobotomized can still sometimes speak full sentences after all. The current models seem to be able to imagine an image or video and show that imagination to us, and they can parrot words to us with a large vocabulary and with many references, but I find myself feeling like these are similar to the sections of our brains that can compute words and imagine pictures. Doesn't quite equal a human yet. These systems need "statefulness", short and long-term memory that can be referenced by multiple discreet interconnected AI systems to take a worthwhile step towards GAI. These systems need an overseer AI that manages and shepherds the LLAMAs and CHATGPTs and StableDiffusions to all work together towards some goal, one that can manage statefulness and manage a limited pool of computational resources (because any GAI would automatically assume that the entire world would provide it with every available resource, because why wouldn't you, right?) Until there is an AI system that has multiple AI systems under its subconscious control, to the point where it can surprise itself with the products it produces, has a memory bank that can be referred back to repeatedly by all of those processes, and that can accept that even if its reasoning is perfect it has been born into an imperfect world that is run by imperfect creatures and so it cannot have or do everything that it might could do even under the best of circumstances, we will not have a GAI.

myrmidon 3y ago

> I think that an AI, a "generalized" intelligence with a generalized understanding [...] is certainly far away. I think this is where we disagree most; GPT-3 and ChatGPT have convinced me that the main difference between human and artificial cognitive capabilities are quantitative now, and unlikely to ever change in our favor... I do agree with you that it is very difficult to predict when this will switch, and how. I personally believe that AI with superhuman capability is an inevitable matter of time now, and I also think that the most likely risk to us, as species, is that we slowly become irrelevant and worthless, just like weavers during the industrial revolution, and that this leads to huge problems for our society. AI completely dominating humankind is a less likely secondary concern IMO, but the potential consequences to our species are unprecedented.

srslack 3y ago
Have you used it outside of the training set? And have you actually used it for generalized tasks? Let's be kind to the LLM and define generalized to mean "do anything if it can be hammered into a textual interface." It fails at pretty much everything that's not on the internet. It produces plausible language and code. It's the future of natural language agents, that's for sure, even though it has no business being the future, today (because of "prompt injection"). These failures are an aside to whether a generalized AI actually carries a substantial global catastrophic risk to us. In that case, if it were actually possible, I don't believe that it's a catastrophic risk either. Let's define the risk to be, at least in the context of LLMs, when junior developers expose internal APIs and databases and access to email accounts to these easily socially-engineered NLP front-ends. It's a very local risk. As far as the future, I can't see 70 years into the future, so anything is possible, but is it likely? I, personally, don't believe so.
- BizarroLand 3y ago
  I don't think it's there or even necessarily close to being GAI. Within our own human brains, we have many sections dedicated to different tasks and processes that all must work together with years and decades of real world interaction to produce what we consider to be a generally intelligent human being, and even in the case of a certain percentage of us humans, even a small amount of damage to a single part of the brain can cause us to become functionally subhuman in our intelligence, barely able to move or eat on our own. A human that has been lobotomized can still sometimes speak full sentences after all. The current models seem to be able to imagine an image or video and show that imagination to us, and they can parrot words to us with a large vocabulary and with many references, but I find myself feeling like these are similar to the sections of our brains that can compute words and imagine pictures. Doesn't quite equal a human yet. These systems need "statefulness", short and long-term memory that can be referenced by multiple discreet interconnected AI systems to take a worthwhile step towards GAI. These systems need an overseer AI that manages and shepherds the LLAMAs and CHATGPTs and StableDiffusions to all work together towards some goal, one that can manage statefulness and manage a limited pool of computational resources (because any GAI would automatically assume that the entire world would provide it with every available resource, because why wouldn't you, right?) Until there is an AI system that has multiple AI systems under its subconscious control, to the point where it can surprise itself with the products it produces, has a memory bank that can be referred back to repeatedly by all of those processes, and that can accept that even if its reasoning is perfect it has been born into an imperfect world that is run by imperfect creatures and so it cannot have or do everything that it might could do even under the best of circumstances, we will not have a GAI.