10 ms·

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether ri

> We're so far away from AGI Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I've seen several prominent people say things to the effect of "We just don't know if we're six months or 600 years from AGI,", which to me is a little like saying, "We don't know if we'll have time travel and ray guns in six months or 600 years." No, we don't know, but all signs seem to be pointing toward a longer timeframe rather than a short one. The counterargument to that seems to be, "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?", which is a sentiment I have also seen expressed by some of the same people.

colordrops 3y ago
> If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? I'm confused by this. What's the difference? Is there a measure other than how well it "mimics" a human? If a robot is indistinguishable from a human, what other measure would make it different in practice vs principle?
- appplication 3y ago
  And if that’s the definition of AGI we’re going by, we’re in no danger of it murdering us all. Imagine thinkjng GPT4 was somehow capable of really any harm at all. It’s also just incredible failure of risk assessment. Global warming is the real and pressing threat, and it’s here right now. Not sentient murderous AI. But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it. Whoops.
  - josephg 3y ago
    > Imagine thinkjng GPT4 was somehow capable of really any harm at all. There’s plenty of new ways you could use chatgpt to mess with society. For example, apparently you only need 8% of people to be talking about something for it to seem like a mass movement. It would be pretty easy for a malicious actor to use LLMs to flood the internet with some new - but fake - story or motivated point of view and kickstart a “mass movement” that way. Or use something like that to heavily influence politics. That might already be happening. We have no idea.
    - RandomLensman 3y ago
      It would also be easy for a benevolent actor to counter-flood etc.
      - josephg 3y ago
        Sure; but multiple AI generated "mass movements" don't cancel each other out.
        RandomLensman 3y ago
        Who knows? We never had a world with a huge supply of mass movement attachment points.
    - appplication 3y ago
      And no amount of well intended AI ethics or prompt denial would stop this sort of malevolent activity.
  - pixl97 3y ago
    >But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it Science fiction invents scenarios that once created cannot be stopped quite often. The "don't create a black hole on earth unless you want to be in the black hole" effect.
ary 3y ago
> Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. There is a clear and perverse incentive to talk about "the coming AGI uprising" incessantly because it's free marketing. It gets and keeps people's attention as the majority of humanity is entirely unqualified to have a reasonable conversation regarding anything AI/ML related. Doomsday fearmongering about AI is a form of grift, and I'll change that opinion only when shown clear evidence that there is something to actually worry about. The real danger of AI is as a weapon to be wielded by people against other people. Good luck having that conversation in any meaningful way because it basically amounts to the long overdue ethics discussion that this entire industry has been avoiding for decades.
- moneytide1 3y ago
  > fearmongering about AI is a form of grift If you cry wolf and agitate the masses with fear, then you gain free advertising for a "new" search engine because the spin of the "journalist" requires anxiety to circulate organically.
dr_dshiv 3y ago
It is a relatively mainstream opinion (e.g. Peter Norvig [1]) that AGI is already here but in the early stages. He specifically compares it to the beginning of general purpose computers. In any case, I think it’s a mistake to consider it a 0/1 situation. This means there will be an ecology of competitive intelligences in the market. It won’t be monolithic. [1] https://www.noemamag.com/artificial-general-intelligence-is-already-here https://www.noemamag.com/artificial-general-intelligence-is-...
- a_wild_dandan 3y ago
  Well said. AGI is already here, because intelligence & generality exist on a spectrum, not a checkbox. Evaluating AGI's presence isn't "yes/no?", but "how much?" LLMs have an astonishingly general set of emergent skills, which they can combine to solve novel problems (intelligence). LLMs smoke humans in many broad intellectual domains, are competitive in others, and fail abysmally in some. Folks try skirting this messy reality, squaring the circle, by conflating AGI with ASI or some techno-godhood outcome. We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety. This is what the "statistical parrot" (i.e: 'No True Scotsman') critics miss. Their handwavy statements about LLMs not "really" understanding the world, "only" doing simple token prediction, having no autonomy/drive, etc. miss the point: it doesn't matter. The damn thing communicating with you (on HN or elsewhere!) may display broad intelligence. I think that's why Turing's test requires a text interface -- people can't help themselves from judging agents by silly anthropomorphic heuristics (appearance, similarity to me, spacial behaviors, etc). So you must literally wall off those biases. AGI is here. It just looks inhuman and our GI is "better", for now.
  - dr_dshiv 3y ago
    > We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety. This is key. All humans don’t understand everything—and the way we assess human conceptual understanding is through tests. This implies that we need a lot more tests for AI. I’d like to see a big investment in “Machine Psychology.”
notatoad 3y ago
>If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? for the purposes of safety, no, absolutely not. the concern is that humans build an AI capable of improving itself, leading to a runaway cycle of rapid advancement beyond the realm of human capabilities. "appearing to mimic human responses" is a long ways off that, and not a "robots take over the world" scale of safety risk.
dkjaudyeqooe 3y ago
> "We don't know if we'll have time travel and ray guns in six months or 600 years." That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host. > "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?" That's another problem, people's credulity when it comes to AI performance. ELIZA was pretty convincing in its time. ChatGPT is convincing a lot of people. But mimicking human responses turns out to be relatively easy, especially when models are trained on human output, but that doesn't move us very far along the path to AGI.
- ghostzilla 3y ago
  The best description of AGI I've seen is in Michael Crichton's Prey. He describes in a fairly plausible detail how AGI rises from the building blocks. The emotion he captures is that of dealing with a psychopath -- goal oriented with no empathy -- to the infinite degree. That said, I think ChatGPT and LLMs trick us into seeing intelligence where there is none because we are evolutionarily wired to look for signs of intelligence as a potential threat. Our defense mechanisms are asymmetrical as they have to err on the side of being overly cautious: you may mistake a rock for a bear but you won't mistake a bear for a rock. So it is with ChatGPT: its conversation style triggers a false alarm deep in us that there is intelligence there and we can't shake that warning off, but this software design is just exploiting the flaw (of sorts) in our wiring.
  - candiddevmike 3y ago
    > goal oriented with no empathy We'll know we have discovered AGI when it gives us a prompt instead of us providing one.
- jdmoreira 3y ago
  Sounds like something really simple to define and grasp. What's wrong with, for example: "A software that beats most humans on most cognitive tasks". People make it harder than it needs to he IMO
jjoonathan 3y ago
> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.
- SuperLynxDeluxe 3y ago
  That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement learning, expert systems (another candidate for AGI in the 90s), soft-optimization. Each of those tools are better than GPT for some problems.
  - catgary 3y ago
    We really have replaced “oh no, you put too much knowledge into a LISP program and it’s sentient!” with “oh no, you’ve stacked too many transformers and now it’s sentient!”
    - josephg 3y ago
      Sentient or no, the power of stacking transformers shocked everyone. 1 or 2 more “happy accidents” like that and we’re in for an interesting few decades.
    - pixl97 3y ago
      I put a few neurons together and got a nematode, then I put 100 trillion together and got a human. While there are other structures involved here, expanding some systems does allow the expansion of capability
  - jjoonathan 3y ago
    > Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap I'm referring to the emergent capabilities that show up with scale, which are a gigantic leap. The usual citation suggests that this came into the public consciousness in 2020 [1]. Do you take issue with the timeline or the idea that these were a gigantic leap? [1] https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165
kypro 3y ago
As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me. I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks. Personally I think humanity is probably okay for another 5 - 10 years, but after that I have almost zero certainty. It seems to me that it's quite possible we could have AGI by the end of the decade if we get another breakthrough or two. Even near-term I have no strong conviction. I have a decent amount of experience building and working with neural nets, but my experience is so far from scale and complexity companies like OpenAI are working at my ability to predict even what might come out in the next 1-2 years is very low. I'd basically just have to assume the next GPT is going to be somewhere between 10-50% jump from what we got this year. But what if it's 200% and the trend is moving exponentially?
- fl7305 3y ago
  > Cripple enough infrastructure and humanity is going to be starving to death within weeks. Just the threat of a malicious AGI that people take seriously enough would cause ourselves to pull the plug on the Internet and all computers until they can be disinfected and new filters put in place. It doesn't matter if the AGI is capable of actually inflicting damage or not.
  - rzl1235 3y ago
    Ah yes, just "pull the plug". That's a hilarious idea, are you sure the AGI that was trying to destroy humanity didn't give you that one?
  - reverius42 3y ago
    We couldn’t even do coordinated lockdowns for a few weeks to prevent millions of Covid deaths, and you think we’ll have a coordinated cessation of computer and internet use?
saulrh 3y ago
> which to me is a little like saying, "We don't know if we'll have time travel and ray guns in six months or 600 years." I wouldn't be surprised by AGI tomorrow, which means I have to consider the possibility. We have, at this point, an extremely good idea of how matter and energy behave. We also know roughly how long it takes to build physical objects and to do R&D cycles with physical prototypes in the loop and how much progress you get out of each prototype cycle. We have a number of good ideas for what a ray gun would look like and we know where we need advances. We've probably already made all of the necessary fundamental breakthroughs, things like laser diodes and capacitors and optical sapphire. We even have some deployed, just at artillery size and less-lethal energies rather than handheld blaster power densities. If you asked DARPA how long it would take them to field a ray gun for the US Army they could give you a date. So I would be extremely surprised if deployment was announced today. I would go looking for fundamental breakthroughs in battery tech and laser media that I had missed. We can't even agree what AGI is, even on an "I'll know it when I see it" level. We have no idea how to get there. R&D cycles are days or weeks long and produce unpredictable and sometimes significant advances in observed functionality. Fundamental breakthroughs come unpredictably but probably about once every two to five years and their effects are even more unpredictable than the effects of individual R&D cycles. Even worse, we have no idea what required breakthroughs are left or if there even are any. So, yes, I believe that I would not be surprised by - and therefore must consider a future containing - one of these AI researchers glueing the output into the input in exactly the right way or something and producing a living, thinking being in their computer tomorrow. I cannot concretely refute the possibility of AGI Tomorrow the way I can argue, in specific detail, against Ray Guns Tomorrow. (There is, of course, the possibility that I've missed some breakthrough that would enable Ray Guns Tomorrow, the same way the people who pooh-poohed heavier-than-air flight the week before the Wright Brothers' fight missed gasoline engines and aerofoils and did their calculations based on a dude flapping wings with his arms. This is why I say that I would be extremely surprised, not why it's impossible.)
bananaflag 3y ago
Time travel is something that is thought to be most probably ruled out by future laws of physics. Intelligence is something that we know exists in nature in a manageable form (namely humans), so it does not seem implausible that we might create it.
adventured 3y ago
> the issue seems to be that there are a lot of people who feel otherwise They're self-serving. It's virtue signaling and doom-pandering for their own attention and benefit. It's the same reason the online ad industry loves the infamous quote about how their generation's best and brightest are wasting their time building ad systems and trying to figure out how to drive higher click rates. It's not true (the best and brightest are not working in advertising), it just makes them feel better so they are ok with the quote and propagate it. A variation of Von Neumann's premise about confessing a sin so as to take credit for it is applicable here. The AGI-is-near camp is intentionally being dramatic, they want attention, and it's wholly self-serving. The louder they are the more attention they get, that has long since been figured out.
- RandomLensman 3y ago
  Anyone remember grey goo from the nanotechnology apocalypse?
joe_the_user 3y ago
Well, one should consider that the deep learning system being developed today are developed through a process that's said to be "closer to alchemy than chemistry". The best deep learning developers aren't thinking about "what is intelligence, really?" but rather figuring ways to optimize systems on benchmarks. And at the same time, current system definitely have made advances in seeming intelligence. Moreover, the Eliza Effect [1] means it's easy for human being interpret a machine that interacts appropriately as intelligent even given very simple programming. If the engineers were engineering these systems, one would expect some immunity to this but as mentioned, they are doing something subtly different. You can see this in action with the Google engineer who tried to get a lawyer for an unreleased LLM and Geoffrey Hinton's turn to worrying after an AI explained a joke to him might arguably be included. And oppositely, because these are developed like alchemy, maybe further qualities will just pop up the way transformers changed language models and maybe this will yield all relevant human capacities. I'm doubtful but not as doubtful as I was five years ago. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect
- pixl97 3y ago
  One thing I keep seeing is 'human capabilities'... Since humans are(?) the smartest things around we keep using ourselves as the benchmark. But what happens if we accidentally create 'alien intelligence'? That is something capable of solving problems but in a way we don't recognize therefore we discount and do not measure it. Humans get quite surprised by other humans when they do things they didn't think of. I don't think it is beyond reason that machine intelligence could optimize in metrics we don't easily comprehend.
skohan 3y ago
But you don't need AGI to be dangerous right? > all signs seem to be pointing toward a longer timeframe rather than a short one. Not saying we are closer, but isn't the risk being described that we could pass an inflection point without knowing it, and things could start accelerating beyond our control? Human beings tend to think about things linearly, and have trouble conceptualizing exponential growth.
skepticATX 3y ago
> the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I think the problem here is this view isn't widely held in academia and more traditional AI companies, but the relatively few AGI labs are commanding all of the attention right now. And we have a whole new crop of "AI engineers" who haven't actually studied AI (this isn't a dig, it's just the truth), and so mostly just parrot what the AGI labs say. It's like going to a flat earth conference and becoming convinced because the majority of the attendees believe that the earth is flat. Of course, this doesn't mean that the AGI labs are incorrect, but thus far the evidence mostly seems to be "trust us, scale will work". I wish that more technologists realized that the prevailing opinion amongst AI experts is not necessarily aligned with the prevailing opinion amongst AGI folks.
hackinthebochs 3y ago
>but all signs seem to be pointing toward a longer timeframe rather than a short one. What signs are those? If anything, the "signs" point to timelines being much shorter than anyone would have thought two years ago. I keep asking those who are confident AGI is a long way away to argue their case. For some reason they never do.
deleted 3y ago
[deleted]
mountainriver 3y ago
How would all signs possibly be pointing to a longer timeline? 3-4 years ago everyone thought it was 50-100 years away. Now with GPT-4 that timeline has radically shifted. The rate of improvement in this space right now is like nothing we’ve ever seen. All signs are quite literally pointing to us getting there by the end of the decade if not much sooner
- junofan 3y ago
  We’re actually maybe a little behind schedule if you look at the “with funding” case from the AGI-09 conference survey: https://sethbaum.com/ac/2011_AI-Experts.pdf https://sethbaum.com/ac/2011_AI-Experts.pdf Matches my recollection. The field is prone to hype cycles.
fudged71 3y ago
While I think we're probably at least years away, I do think there's a certain level of inevitability, and that certain individuals/companies/governments can and will put a lot of funding and effort towards making this happen.
jalapenos 3y ago
I think the people worried about AGI have an oddly positive view of the world - like it's so wonderful as-is that we need to guard it from any disturbances. Perhaps if they paid closer attention to how profoundly bad so much of human existence is, people living in squalor, dying from awful diseases, killing each other, they'd understand how truly acceptable that dice roll is. Pedal to the metal all the way.
- FormerBandmate 3y ago
  I’m curious what AGI discourse looks like in Israel right now where they’re working on the same shit but actually facing an existential threat. Somehow doubt it’s that high
  - itointegral 3y ago
    Obviously, https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs.
- ethanbond 3y ago
  This doesn’t describe any AI pessimist/critic I’ve ever met.
- voitvoder 3y ago
  I think there are many highly intelligent people that would prefer less intelligent people don't gain additional intelligence. They are perfectly happy with the relative value of their own intelligence and know that relative value goes way down in a world that everyone has access to additional intelligence.
  - jalapenos 3y ago
    Interesting take & good point

colordrops 3y ago

> If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? I'm confused by this. What's the difference? Is there a measure other than how well it "mimics" a human? If a robot is indistinguishable from a human, what other measure would make it different in practice vs principle?

appplication 3y ago
And if that’s the definition of AGI we’re going by, we’re in no danger of it murdering us all. Imagine thinkjng GPT4 was somehow capable of really any harm at all. It’s also just incredible failure of risk assessment. Global warming is the real and pressing threat, and it’s here right now. Not sentient murderous AI. But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it. Whoops.
- josephg 3y ago
  > Imagine thinkjng GPT4 was somehow capable of really any harm at all. There’s plenty of new ways you could use chatgpt to mess with society. For example, apparently you only need 8% of people to be talking about something for it to seem like a mass movement. It would be pretty easy for a malicious actor to use LLMs to flood the internet with some new - but fake - story or motivated point of view and kickstart a “mass movement” that way. Or use something like that to heavily influence politics. That might already be happening. We have no idea.
  - RandomLensman 3y ago
    It would also be easy for a benevolent actor to counter-flood etc.
    - josephg 3y ago
      Sure; but multiple AI generated "mass movements" don't cancel each other out.
      - RandomLensman 3y ago
        Who knows? We never had a world with a huge supply of mass movement attachment points.
  - appplication 3y ago
    And no amount of well intended AI ethics or prompt denial would stop this sort of malevolent activity.
- pixl97 3y ago
  >But it’s easier to dream about a threat that doesn’t exist than one that does because you can still imagine a perfect scenario in which you prevent it Science fiction invents scenarios that once created cannot be stopped quite often. The "don't create a black hole on earth unless you want to be in the black hole" effect.

ary 3y ago

> Although I personally suspect this is correct, the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. There is a clear and perverse incentive to talk about "the coming AGI uprising" incessantly because it's free marketing. It gets and keeps people's attention as the majority of humanity is entirely unqualified to have a reasonable conversation regarding anything AI/ML related. Doomsday fearmongering about AI is a form of grift, and I'll change that opinion only when shown clear evidence that there is something to actually worry about. The real danger of AI is as a weapon to be wielded by people against other people. Good luck having that conversation in any meaningful way because it basically amounts to the long overdue ethics discussion that this entire industry has been avoiding for decades.

moneytide1 3y ago
> fearmongering about AI is a form of grift If you cry wolf and agitate the masses with fear, then you gain free advertising for a "new" search engine because the spin of the "journalist" requires anxiety to circulate organically.

dr_dshiv 3y ago

It is a relatively mainstream opinion (e.g. Peter Norvig [1]) that AGI is already here but in the early stages. He specifically compares it to the beginning of general purpose computers. In any case, I think it’s a mistake to consider it a 0/1 situation. This means there will be an ecology of competitive intelligences in the market. It won’t be monolithic. [1] https://www.noemamag.com/artificial-general-intelligence-is-already-here https://www.noemamag.com/artificial-general-intelligence-is-...

a_wild_dandan 3y ago
Well said. AGI is already here, because intelligence & generality exist on a spectrum, not a checkbox. Evaluating AGI's presence isn't "yes/no?", but "how much?" LLMs have an astonishingly general set of emergent skills, which they can combine to solve novel problems (intelligence). LLMs smoke humans in many broad intellectual domains, are competitive in others, and fail abysmally in some. Folks try skirting this messy reality, squaring the circle, by conflating AGI with ASI or some techno-godhood outcome. We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety. This is what the "statistical parrot" (i.e: 'No True Scotsman') critics miss. Their handwavy statements about LLMs not "really" understanding the world, "only" doing simple token prediction, having no autonomy/drive, etc. miss the point: it doesn't matter. The damn thing communicating with you (on HN or elsewhere!) may display broad intelligence. I think that's why Turing's test requires a text interface -- people can't help themselves from judging agents by silly anthropomorphic heuristics (appearance, similarity to me, spacial behaviors, etc). So you must literally wall off those biases. AGI is here. It just looks inhuman and our GI is "better", for now.
- dr_dshiv 3y ago
  > We should evaluate AGI the same way we evaluate human general intelligence, which also manifests with dizzying variety. This is key. All humans don’t understand everything—and the way we assess human conceptual understanding is through tests. This implies that we need a lot more tests for AI. I’d like to see a big investment in “Machine Psychology.”

notatoad 3y ago

>If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle? for the purposes of safety, no, absolutely not. the concern is that humans build an AI capable of improving itself, leading to a runaway cycle of rapid advancement beyond the realm of human capabilities. "appearing to mimic human responses" is a long ways off that, and not a "robots take over the world" scale of safety risk.

dkjaudyeqooe 3y ago

> "We don't know if we'll have time travel and ray guns in six months or 600 years." That's not the (only) problem, it's that no one can describe what AGI will really look like outside of vague and circular definitions. People have an idea of what they would like it to be, but there's no evidence any of it can be built, particularly human like intelligence, which may be only possible in the human host. > "If we can make something that appears to be able to mimic human responses, isn't that AGI in practice even if it's not in principle?" That's another problem, people's credulity when it comes to AI performance. ELIZA was pretty convincing in its time. ChatGPT is convincing a lot of people. But mimicking human responses turns out to be relatively easy, especially when models are trained on human output, but that doesn't move us very far along the path to AGI.

ghostzilla 3y ago
The best description of AGI I've seen is in Michael Crichton's Prey. He describes in a fairly plausible detail how AGI rises from the building blocks. The emotion he captures is that of dealing with a psychopath -- goal oriented with no empathy -- to the infinite degree. That said, I think ChatGPT and LLMs trick us into seeing intelligence where there is none because we are evolutionarily wired to look for signs of intelligence as a potential threat. Our defense mechanisms are asymmetrical as they have to err on the side of being overly cautious: you may mistake a rock for a bear but you won't mistake a bear for a rock. So it is with ChatGPT: its conversation style triggers a false alarm deep in us that there is intelligence there and we can't shake that warning off, but this software design is just exploiting the flaw (of sorts) in our wiring.
- candiddevmike 3y ago
  > goal oriented with no empathy We'll know we have discovered AGI when it gives us a prompt instead of us providing one.
jdmoreira 3y ago
Sounds like something really simple to define and grasp. What's wrong with, for example: "A software that beats most humans on most cognitive tasks". People make it harder than it needs to he IMO

jjoonathan 3y ago

> all signs seem to be pointing toward a longer timeframe rather than a short one If we time-traveled back to 2019 and showed past-you a demo of GPT4's capabilities and asked you to guess the timeline, would you have gotten it correct to within a decade? I wouldn't have. It's not crazy to prepare for another jump.

SuperLynxDeluxe 3y ago
That really depends on who you asked. Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap. Deep learning is to neural networks what GPT4 is to transformers. It's a lot bigger, enabled by better hardware and more data, but fundamentally not a new technology. There needs to be a lot more leaps for AGI, language models are not the holy grail, but another tool like reinforcement learning, expert systems (another candidate for AGI in the 90s), soft-optimization. Each of those tools are better than GPT for some problems.
- catgary 3y ago
  We really have replaced “oh no, you put too much knowledge into a LISP program and it’s sentient!” with “oh no, you’ve stacked too many transformers and now it’s sentient!”
  - josephg 3y ago
    Sentient or no, the power of stacking transformers shocked everyone. 1 or 2 more “happy accidents” like that and we’re in for an interesting few decades.
  - pixl97 3y ago
    I put a few neurons together and got a nematode, then I put 100 trillion together and got a human. While there are other structures involved here, expanding some systems does allow the expansion of capability
- jjoonathan 3y ago
  > Transformers existed in 2019, and while GPT4 is genuinely advanced, it's not a gigantic leap I'm referring to the emergent capabilities that show up with scale, which are a gigantic leap. The usual citation suggests that this came into the public consciousness in 2020 [1]. Do you take issue with the timeline or the idea that these were a gigantic leap? [1] https://arxiv.org/abs/2005.14165 https://arxiv.org/abs/2005.14165

kypro 3y ago

As a principal I tend to lower my conviction of any opinion I hold the more that people I intellectually respect disagree with me. I think to say with confidence that we're doomed or that there is no risk to the AI system currently in development would be intellectually arrogant. I'm not even convinced you necessarily need AGI before AI becomes a risk to humanity. An advanced enough auto-GPT bot given a bad goal might be all you need. Cripple enough infrastructure and humanity is going to be starving to death within weeks. Personally I think humanity is probably okay for another 5 - 10 years, but after that I have almost zero certainty. It seems to me that it's quite possible we could have AGI by the end of the decade if we get another breakthrough or two. Even near-term I have no strong conviction. I have a decent amount of experience building and working with neural nets, but my experience is so far from scale and complexity companies like OpenAI are working at my ability to predict even what might come out in the next 1-2 years is very low. I'd basically just have to assume the next GPT is going to be somewhere between 10-50% jump from what we got this year. But what if it's 200% and the trend is moving exponentially?

fl7305 3y ago
> Cripple enough infrastructure and humanity is going to be starving to death within weeks. Just the threat of a malicious AGI that people take seriously enough would cause ourselves to pull the plug on the Internet and all computers until they can be disinfected and new filters put in place. It doesn't matter if the AGI is capable of actually inflicting damage or not.
- rzl1235 3y ago
  Ah yes, just "pull the plug". That's a hilarious idea, are you sure the AGI that was trying to destroy humanity didn't give you that one?
- reverius42 3y ago
  We couldn’t even do coordinated lockdowns for a few weeks to prevent millions of Covid deaths, and you think we’ll have a coordinated cessation of computer and internet use?

saulrh 3y ago

> which to me is a little like saying, "We don't know if we'll have time travel and ray guns in six months or 600 years." I wouldn't be surprised by AGI tomorrow, which means I have to consider the possibility. We have, at this point, an extremely good idea of how matter and energy behave. We also know roughly how long it takes to build physical objects and to do R&D cycles with physical prototypes in the loop and how much progress you get out of each prototype cycle. We have a number of good ideas for what a ray gun would look like and we know where we need advances. We've probably already made all of the necessary fundamental breakthroughs, things like laser diodes and capacitors and optical sapphire. We even have some deployed, just at artillery size and less-lethal energies rather than handheld blaster power densities. If you asked DARPA how long it would take them to field a ray gun for the US Army they could give you a date. So I would be extremely surprised if deployment was announced today. I would go looking for fundamental breakthroughs in battery tech and laser media that I had missed. We can't even agree what AGI is, even on an "I'll know it when I see it" level. We have no idea how to get there. R&D cycles are days or weeks long and produce unpredictable and sometimes significant advances in observed functionality. Fundamental breakthroughs come unpredictably but probably about once every two to five years and their effects are even more unpredictable than the effects of individual R&D cycles. Even worse, we have no idea what required breakthroughs are left or if there even are any. So, yes, I believe that I would not be surprised by - and therefore must consider a future containing - one of these AI researchers glueing the output into the input in exactly the right way or something and producing a living, thinking being in their computer tomorrow. I cannot concretely refute the possibility of AGI Tomorrow the way I can argue, in specific detail, against Ray Guns Tomorrow. (There is, of course, the possibility that I've missed some breakthrough that would enable Ray Guns Tomorrow, the same way the people who pooh-poohed heavier-than-air flight the week before the Wright Brothers' fight missed gasoline engines and aerofoils and did their calculations based on a dude flapping wings with his arms. This is why I say that I would be extremely surprised, not why it's impossible.)

bananaflag 3y ago

Time travel is something that is thought to be most probably ruled out by future laws of physics. Intelligence is something that we know exists in nature in a manageable form (namely humans), so it does not seem implausible that we might create it.

adventured 3y ago

> the issue seems to be that there are a lot of people who feel otherwise They're self-serving. It's virtue signaling and doom-pandering for their own attention and benefit. It's the same reason the online ad industry loves the infamous quote about how their generation's best and brightest are wasting their time building ad systems and trying to figure out how to drive higher click rates. It's not true (the best and brightest are not working in advertising), it just makes them feel better so they are ok with the quote and propagate it. A variation of Von Neumann's premise about confessing a sin so as to take credit for it is applicable here. The AGI-is-near camp is intentionally being dramatic, they want attention, and it's wholly self-serving. The louder they are the more attention they get, that has long since been figured out.

RandomLensman 3y ago
Anyone remember grey goo from the nanotechnology apocalypse?

joe_the_user 3y ago

Well, one should consider that the deep learning system being developed today are developed through a process that's said to be "closer to alchemy than chemistry". The best deep learning developers aren't thinking about "what is intelligence, really?" but rather figuring ways to optimize systems on benchmarks. And at the same time, current system definitely have made advances in seeming intelligence. Moreover, the Eliza Effect [1] means it's easy for human being interpret a machine that interacts appropriately as intelligent even given very simple programming. If the engineers were engineering these systems, one would expect some immunity to this but as mentioned, they are doing something subtly different. You can see this in action with the Google engineer who tried to get a lawyer for an unreleased LLM and Geoffrey Hinton's turn to worrying after an AI explained a joke to him might arguably be included. And oppositely, because these are developed like alchemy, maybe further qualities will just pop up the way transformers changed language models and maybe this will yield all relevant human capacities. I'm doubtful but not as doubtful as I was five years ago. [1] https://en.wikipedia.org/wiki/ELIZA_effect https://en.wikipedia.org/wiki/ELIZA_effect

pixl97 3y ago
One thing I keep seeing is 'human capabilities'... Since humans are(?) the smartest things around we keep using ourselves as the benchmark. But what happens if we accidentally create 'alien intelligence'? That is something capable of solving problems but in a way we don't recognize therefore we discount and do not measure it. Humans get quite surprised by other humans when they do things they didn't think of. I don't think it is beyond reason that machine intelligence could optimize in metrics we don't easily comprehend.

skohan 3y ago

But you don't need AGI to be dangerous right? > all signs seem to be pointing toward a longer timeframe rather than a short one. Not saying we are closer, but isn't the risk being described that we could pass an inflection point without knowing it, and things could start accelerating beyond our control? Human beings tend to think about things linearly, and have trouble conceptualizing exponential growth.

skepticATX 3y ago

> the issue seems to be that there are a lot of people who feel otherwise, whether right or wrong, including some of the people involved in the research and/or running these companies. I think the problem here is this view isn't widely held in academia and more traditional AI companies, but the relatively few AGI labs are commanding all of the attention right now. And we have a whole new crop of "AI engineers" who haven't actually studied AI (this isn't a dig, it's just the truth), and so mostly just parrot what the AGI labs say. It's like going to a flat earth conference and becoming convinced because the majority of the attendees believe that the earth is flat. Of course, this doesn't mean that the AGI labs are incorrect, but thus far the evidence mostly seems to be "trust us, scale will work". I wish that more technologists realized that the prevailing opinion amongst AI experts is not necessarily aligned with the prevailing opinion amongst AGI folks.

hackinthebochs 3y ago

>but all signs seem to be pointing toward a longer timeframe rather than a short one. What signs are those? If anything, the "signs" point to timelines being much shorter than anyone would have thought two years ago. I keep asking those who are confident AGI is a long way away to argue their case. For some reason they never do.

deleted 3y ago

[deleted]

mountainriver 3y ago

How would all signs possibly be pointing to a longer timeline? 3-4 years ago everyone thought it was 50-100 years away. Now with GPT-4 that timeline has radically shifted. The rate of improvement in this space right now is like nothing we’ve ever seen. All signs are quite literally pointing to us getting there by the end of the decade if not much sooner

junofan 3y ago
We’re actually maybe a little behind schedule if you look at the “with funding” case from the AGI-09 conference survey: https://sethbaum.com/ac/2011_AI-Experts.pdf https://sethbaum.com/ac/2011_AI-Experts.pdf Matches my recollection. The field is prone to hype cycles.

fudged71 3y ago

While I think we're probably at least years away, I do think there's a certain level of inevitability, and that certain individuals/companies/governments can and will put a lot of funding and effort towards making this happen.

jalapenos 3y ago

I think the people worried about AGI have an oddly positive view of the world - like it's so wonderful as-is that we need to guard it from any disturbances. Perhaps if they paid closer attention to how profoundly bad so much of human existence is, people living in squalor, dying from awful diseases, killing each other, they'd understand how truly acceptable that dice roll is. Pedal to the metal all the way.

FormerBandmate 3y ago
I’m curious what AGI discourse looks like in Israel right now where they’re working on the same shit but actually facing an existential threat. Somehow doubt it’s that high
- itointegral 3y ago
  Obviously, https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs https://en.wikipedia.org/wiki/Maslow%27s_hierarchy_of_needs.
ethanbond 3y ago
This doesn’t describe any AI pessimist/critic I’ve ever met.
voitvoder 3y ago
I think there are many highly intelligent people that would prefer less intelligent people don't gain additional intelligence. They are perfectly happy with the relative value of their own intelligence and know that relative value goes way down in a world that everyone has access to additional intelligence.
- jalapenos 3y ago
  Interesting take & good point