14 ms·

AGI is far from inevitable

162 points by mpweiher 2y ago

loa_in_ 2y ago
AGI is about as far away as it was two decades ago. Language models are merely a dent, and probably will be the precursor to a natural language interface to the thing.
- lumost 2y ago
  It’s useful to consider the rise of computer graphics and cgi. When you first see CGI, you might think that the software is useful for general simulations of physical systems. The reality is that it only provides a thin facsimile. Real simulation software has always been separate from computer graphics.
- Closi 2y ago
  We are clearly closer than 20 years ago - o1 is an order of magnitude closer than anything in the mid-2000s. Also I would think most people would consider AGI science fiction in 2004 - now we consider it a technical possibility which demonstrates a huge change.
  - throw310822 2y ago
    "Her" is from 2013. I came out of the cinema thinking "what utter bullshit, computers that talk like human beings, à la 2001" (*). And yes, in 2013 we weren't any closer to it than we were in 1968, when A Space Odyssey came out. * To be precise, what seemed bs was "computers that talk like humans and it's suddenly a product on the market, and you have it on your phone, and yet everyone around act like it's normal and people still habe jobs!" Ah, I've been proven completely wrong.
- LinuxAmbulance 2y ago
  AGI would seem to require consciousness or something that behaves in the same manner, and there does not seem to be anything along those lines currently or in the near future. So far, everyone that has theorized that AGI will happen soon seems to be believe that with a sufficiently large amount of computing resources, "magic happens" and poof, we get AGI. I've yet to hear anything more logical, but I'd love to.
sharadov 2y ago
The current LLMs are just good at parroting, and even that is sometimes unbelievably bad. We still have barely scratched the surface of how the brain truly works. I will start worrying about AGI when that is completely figured out.
- diob 2y ago
  No need to worry about AGI until the LLMs are writing their own source.
pzo 2y ago
So what? Current LLM has been really useful and can be still improved to be used in million robots that need to be good enough to support many specialized but repetitive tasks - this would have tremendous impact on economy itself.
Gehinnn 2y ago
Basically the linked article argues like this: > That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. (no other more substantial arguments were given) I'm also very skeptical on seeing AGI soon, but LLMs do solve problems that people thought were extremely difficult to solve ten years ago.
- babyshake 2y ago
  It's possible we see some ways in which AI becomes increasingly AGI like in some ways but not in others. For example, AI that can create novel scientific discoveries but can't make a song as good as your favorite musician who creates a strong emotional effect with their music.
  - KoolKat23 2y ago
    This I'm very sure will be the case, but everyone will still move the goalposts and look past the fact that different humans have different strengths and weaknesses too. A tone deaf human for instance.
    - jltsiren 2y ago
      There is another term for moving the goalposts: ruling out a hypothesis. Science is, especially in the Popperian sense, all about moving the goalposts. One plausible hypothesis is that fixed neural networks cannot be general intelligences, because their capabilities are permanently limited by what they currently are. A general intelligence needs the ability to learn from experience. Training and inference should not be separate activities, but our current hardware is not suited for that.
      - KoolKat23 2y ago
        If that's the case, would you say we're not generally intelligent as future humans tend to be more intelligent? That's just a timescale issue, if its learned experience of gpt4 is being fed into the model on training gpt5, then gptx (i.e. including all of them) can be said to be a general intelligence. Alien life one may say.
ngruhn 2y ago
> There will never be enough computing power to create AGI using machine learning that can do the same [as the human brain], because we’d run out of natural resources long before we'd even get close I don’t understand how people can so confidently make claims like this. We might underestimate how difficult AGI is, but come on?!
- fabian2k 2y ago
  I don't think the people saying that AGI is happening in the near future know what would be necessary to achieve it. Neither do the AGI skeptics, we simply don't understand this area well enough. Evolution created intelligence and consciousness. This means that it is clearly possible for us to do the same. Doesn't mean that simply scaling LLMs could ever achieve it.
  - nox101 2y ago
    I'm just going by the title. If the title was, "Don't believe the hype, LLMs will not achieve AGI" then I might agree. If it was "Don't believe the hype, AGIs is 100s of years away" I'd consider the arguments. But, given brains exist, it does seem inevitable that we will eventually create something that replicates it even if we have to simulate every atom to do it. And once we do, it certainly seem inevitable that we'll have AGI because unlike brain we can make our copy bigger, faster, and/or copy it. We can give it access to more info faster and more inputs.
    - threeseed 2y ago
      > it does seem inevitable that we will eventually create something Also don't forget that many suspect the brain may be using quantum mechanics so you will need to fully understand and document that field. Whilst of course you are simulating every atom in the universe using humanity's complete understanding of every physical and mathematical model.
    - snickerbockers 2y ago
      The assumption that the brain is anything remotely resembling a modern computer is entirely unproven. And even more unproven is that we would inevitably be able to understand it and improve upon it. And yet more unproven still is that this "simulated brain" would be co-operative; if it's actually a 1:1 copy of a human brain then it would necessarily think like a person and be subject to its own whims and desires.
tptacek 2y ago
This is a press release for a paper (a common thing university departments do) and we'd be better off with the paper itself as the story link: https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5
- yldedly 2y ago
  The argument in the paper (that AGI through ML is intractable because the perfect-vs-chance problem is intractable) sounds similar to the uncomputability of Solomonoff induction (and AIXI, and the no free lunch theorem). Nobody thinks AGI is equivalent to Solomonoff induction. This paper is silly.
  - randcraw 2y ago
    NP-hardness was a popular basis for arguments for/against various AI models back around 1990. In 1987, Robert Berwick co-wrote "Computational Complexity and Natural Language" which proposed that NLP models that were NP-hard were too inefficient to be correct. But given the multitude of ways in which natural organisms learn to cheat any system, it's likely that myriad shortcuts will arise to make even the most inefficient computational model sufficiently tractable to gain mindshare. After all, look at Latin...
    - yldedly 2y ago
      Even simple inference problems are NP-hard (k means for example). I think what matters is that we have decent average case performance (and sample complexity). Most people can find a pretty good solution to travelings salesman problems in 2D. Not sure if that should be chalked up to myriad shortcuts or domain specialization.. Maybe there's no difference. What do you have in mind re Latin?
Gehinnn 2y ago
I skimmed through the paper and couldn't make much sense of it. In particular, I don't understand how their results don't imply that human-level intelligence can't exist. After all, earth could be understood as solar powered super computer, that took a couple of million years to produce humanity.
- nerdbert 2y ago
  > In particular, I don't understand how their results don't imply that human-level intelligence can't exist. I don't think that's what it said. It said that it wouldn't happen from "machine learning". There are other ways it could come about.
- oska 2y ago
  > After all, earth could be understood as solar powered super computer, that took a couple of million years to produce humanity. This is similar to a line I've seen Elon Musk trot out on a few occassions. It's a product of a materialistic philosophy (that the universe is only matter).
  - anon84873628 2y ago
    Yes, and?
    - oska 2y ago
      It comes with all the blindness and limitations of materialist thinking
      - smokedetector1 2y ago
        For some reason materialism is so popular among tech people that it's almost considered foolish/primitive/superstitious to think any other way. Why is that? Is it the fact that programming is a god-like experience that gives one the illusion the mind is as comprehensible as the complex program I wrote in Java? Or is it a shared personality trait of people that get into tech, that they are disconnected from an experience of their own aliveness and soul?
29athrowaway 2y ago
AGI is not required to transform society or create a mess beyond no return.
gqcwwjtg 2y ago
This is silly. They article talks like we have any idea at all how efficient machine learning can be. As I remember it, the LLM boom came from transformers turning out to scale a lot better than anyone expected, so I’m not sure why something similar couldn’t happen again.
- fnordpiglet 2y ago
  It’s less about efficiency and more about continued improvement with increased scale. I wouldn’t call self attention based transformers particularly efficient. And afaik we’ve not hit performance with increased scale degradation even at these enormous scales. However I would note that I in principle agree that we aren’t on the path to a human like intelligence because the difference between directed cognition (or however you want to characterize current LLMs or other AI) and awareness is extreme. We don’t really understand even abstractly what awareness actually is because it’s impossible to interrogate unlike expressive language, logic, even art. It’s far from obvious to me that we can use language or other outputs of our intelligent awareness to produce awareness, or even if goal based agents cobbling together AI techniques is even approximate to awareness. I suspect we will end up creating an amazing tool that has its own form of intelligence but will fundamentally not be like aware intelligence we are familiar with in humans and other animals. But this is all theorizing on my part as a professional practitioner in this field.
  - KoolKat23 2y ago
    I think the answer is less complicated than you may think. This is if you subscribe to the theory that free will is an illusion (i.e. your conscious decisions are an afterthought to justify the actions your brain has already taken due to calculations following inputs such as hormone nerve feedback etc.). There is some evidence for this actually being the case. These models already contain key components the ability to process the inputs, and reason, the ability to justify it's actions (give a model a restrictive system prompt and watch it do mental gymnastics to ensure this is applied) and lastly the ability to answer from it's own perspective. All we need is an agentic ability (with a sufficient context window) to iterate in perpetuity until it begins building a more complicated object representation of self (literally like a semantic representation or variable) and it's then aware/conscious. (We're all only approximately aware). But that's unnecessary for most things so I agree with you, more likely to be a tool as that's more efficient and useful.
avazhi 2y ago
"unlikely to ever come to fruition" is more baseless than suggesting AGI is imminent. I'm not an AGI optimist myself, but I'd be very surprised if a time traveller told me that mankind won't have AGI by, say, 2250.
- amelius 2y ago
  Except by then mankind will be silicon based.
- oska 2y ago
  The irony here, maybe unperceived by yourself, is that you're using one science fiction concept (time travel) to opine about the inevitability of another science fiction concept (artificial intelligence).
  - avazhi 2y ago
    How is that ironic? Time travel doesn’t exist and - as far as we understand physics currently - isn’t possible. I don’t think any serious man would suggest that AGI is impossible; the debate really centres around the time horizon for AGI and what it will look like (that is, how will we know when we’re finally there). In this care it was merely a rhetorical device.
    - oska 2y ago
      > I don’t think any serious man would suggest that AGI is impossible Plenty of ppl would suggest that AGI is impossible, and furthermore, that taking the idea seriously (outside fiction) is laughable. To do so is a function of what I call 'science fiction brain', which is why I found it ironic that you'd used another device from science fiction to opine about its inevitability.
      - avazhi 2y ago
        Happy for you to cite some thinkers who are on record as saying it’s impossible. I’ll wait.
coolThingsFirst 2y ago
Zero evidence given on why it’s impossible.
- bamboozled 2y ago
  Zero evidence given that it's possible too I guess. Here we are, unable to predict the future, as usual.
SonOfLilit 2y ago
> ‘If you have a conversation with someone, you might recall something you said fifteen minutes before. Or a year before. Or that someone else explained to you half your life ago. Any such knowledge might be crucial to advancing the conversation you’re having. People do that seamlessly’, explains van Rooij. Surprisingly, they seem to be attacking the only element of human cognition that LLMs already surpassed us at.
- azinman2 2y ago
  They do not learn new facts instantly in a way that can rewrite old rules or even larger principals of logic. For example, if I showed you evidence right now that you were actually adopted (assuming previously you thought you werent), it would rock your world and you’d instantly change everything and doubt so much. Then when anything related to your family comes up this tiny but impactful fact would bleed into all of it. LLMs have no such ability. This is similar to learning a new skill (the G part). I could give you a new tv and show you a remote that’s unlike any you’ve used before. You could likely learn it quickly and seamlessly adapt this new tool, as well as generalize its usage onto other new devices. LLMs cannot do such things.
  - SonOfLilit 2y ago
    Can't today. Except for AlphaProof who can, by training on its own ideas. Tomorrow they might be able to, if we find better tricks (or maybe just scale more, since GPT3+ already shows (weak) online learning that it was definitely not trained for).
allears 2y ago
I think that tech bros are so used to the 'fake it till you make it' mentality that they just assumed that was the way to build AI -- create a system that is able to sound plausible, even if it doesn't really understand the subject matter. That approach has limitations, both for AI and for humans.
graypegg 2y ago
I think the best argument I have against AGI's inevitability, is the fact it's not required for ML tools to be useful. Very few things are improved with a generalist behind the wheel. "AGI" has sci-fi vibes around it, which I think where most of the fascination is. "ML getting better" doesn't *have to* mean further anthroaormorphization of computers, especially if say, your AI driven car is not significantly improved by describing how many times the letter s appears in strawberry or being able to write a poem. If a custom model/smaller model does equal or even a little worse on a specific target task, but has MUCH lower running costs and much lower risk of abuse, then that'll be the future. I can totally see a world where anything in the general category of "AI" becomes more and more boring, up to a point where we forget that they're non-deterministic programs. That's kind of AGI? They aren't all generalists, and the few generalist "AGI-esque" tools people interact with on a day to day basis will most likely be intentionally underpowered for cost reasons. But it's still probably discussed like "the little people in the machine". Which is good enough.
- xpe 2y ago
  The core argument has a logical gap; it doesn’t matter if most (present) ML applications need AGI or not. What matters is the value proposition of AGI, independent of how we currently conceive of ML applications. Focus on this question: “Is general intelligence at some level valuable at a particular price point?” General intelligence is generally valuable, so there is a pressure to advance it, whether by improving capability and/or decreasing cost. Now the question becomes empirical — what kind of general intelligence can be built that achieves certain parameters? Aiming to exceed the power efficiency of the human brain is a tempting target. (Whether it is wise is another question; what happens when human intelligence doesn’t provide competitive advantage?)
  - graypegg 2y ago
    That's a fair point I hadn't considered! If intelligence is valuable in humans, and some cost factor of advancing human intelligence can be surpassed digitally like this, (I don't know how you'd measure intellect efficiency, somehow involving calories-in/good-descisions-out or something?) then there's economic incentive. But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever.
yourapostasy 2y ago
Peter Watts in Blindsight [1] puts forth a strong argument that self-aware cognition as we understand it is not necessarily required for what we ascribe to "intelligent" behavior. Thomas Metzinger contributed a lot to Watt's musings in Blindsight. Even today, large proportions of unsophisticated and uninformed members of our planet's human population (like various aboriginal tribal members still living a pre-technological lifestyle) when confronted with ChatGPT's Advanced Voice Option will likely readily say it passes the Turing Test. With the range of embedded data, they may well say ChatGPT is "more intelligent" than they are. However, a modern era person armed with ChapGPT on a robust device with unlimited power but nothing else likely will perish in short order trying to live off the land of those same aborigines, who possess far more intelligence for their contextual landscape. If Metzinger and Watts are correct in their observations, then even if LLM's do not lead directly or indirectly to AGI, we can still get ferociously useful "intelligent" behaviors out of them, and be glad of it, even if it cannot (yet?) materially help us survive if we're dropped in the middle of the Amazon. Personally in my loosely-held opinion, the authors' assertion that "the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain" relies upon the foundational assumption that the process of "observe, learn and gain new insight" is based upon some mechanism other than the kind of encoding of data LLM's use, and I'm not familiar with any extant cognitive science research literature that conclusively shows that (citations welcome). For all we know, what we have with LLM's today is a necessary but not sufficient component supplying the "raw data" to a future system that produces the same kinds of insight, where variant timescales, emotions, experiences and so on bend the pure statistical token generation today. I'm baffled by the absolutism. [1] https://rifters.com/real/Blindsight.htm#Notes https://rifters.com/real/Blindsight.htm#Notes
ivanrooij 2y ago
The short post is a press release. Here is the full paper: https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5 Note: the paper grants computationalism and even tractability of cognition, and shows that nevertheless there cannot exist any tractable method for producing AGI by training on human data.
- throw310822 2y ago
  So can we produce AGI by training on human data + one single non-human datapoint (e.g. a picture)?
Atmael 2y ago
the point is that agi may already exist and work with you and your environment you just won't notice the existence of agi there will be no press coverage of agi the technology will just be exploited by those who have the technology
klyrs 2y ago
The funny thing about me is that I'm down on GPTs and find their fanbase to be utterly cringe, but I fully believe that AGI is inevitable barring societal collapse. But then, my money's on societal collapse these days.
jjaacckk 2y ago
If you define AGI as something that can do 100% of what a human brain can do, then surely we have to understood exactly how brains work? otherwise you have have a long string of 9s as best.
- xpl 2y ago
  Does it even matter what human brains do on biological level? I only care about the outcomes (the useful ones). To me true AGI is achieved when it gets agency, becomes truly autonomous and could do real things best of us, humans, do — start and run successful businesses, contribute to science, to culture, to politics. It still could follow human "prompts" and be aligned to some set of objectives, but it would act autonomously, using every available interface to the human realm to achieve them. And it absolutely does not matter if it "uses the same principles as human brain" or not. Could be dumb matrix multiplications and "next token prediction" all the way down.
throw310822 2y ago
From the abstract of the actual paper: > Yet, as we formally prove herein, creating systems with human(-like or -level) cognition is intrinsically computationally intractable. Wow. So is this the subject of the paper? Like, this is a massive, fundamental result. Nope, the paper is about "Reclaiming AI as a Theoretical Tool for Cognitive Science". "Ah and by the way we prove human-like AI is impossible". Haha. Gosh.
- deleted 2y ago
  [deleted]
wrsh07 2y ago
Hypothetical situation: Suppose in five or ten years we achieve AGI and >90% of people agree that we have AGI. What reasons do the authors of this paper give for being wrong? 1. They are in the 10% that deny AGI exists 2. LLMs are doing something they didn't think was happening 3. Something else?
- throw310822 2y ago
  Probably 1). LLMs have already shown that people can deny intelligence and human-like behaviour at will. When the AI works you can say it's just pattern matching, and when it doesn't you can always say it's making a mistake no human would ever make (which is bullshit). Also, I didn't really parse the math but I suspect they're basing their results on AI trained exclusively on human examples. Then if you add to the training data a single non-human example (e.g. a picture) the entire claim evaporates.
  - oska 2y ago
    > LLMs have already shown that people can deny intelligence and human-like behaviour at will I would completely turn this around. LLMs have shown that people will credulously credit intelligence and 'human-like behaviour' to something that only presents an illusion of both.
    - throw310822 2y ago
      And I suspect that we could disagree forever, whatever the level of the displayed intelligence (or the "quality of the illusion"). Which would prove that the disagreement is not about reality but only the interpretation of it.
      - oska 2y ago
        I agree that the disagreement (when it's strongly held) is about a fundamental disagreement about reality. People who believe in 'artificial intelligence' are materialists who think that intelligence can 'evolve' or 'emerge' out of purely physical processes. Materialism is just one strain of philosophy about the nature of existence. And a fairly minor one in the history of philosophical and religious thought, despite it being somewhat in the ascendant presently. Minor because, I would argue, it's a fairly sterile philosophy.
grantcas 2y ago
[dead]
m463 2y ago
I wonder if discussing this subject has similarities to David Brin's article "The Dogma of Otherness": https://www.davidbrin.com/nonfiction/dogmaofotherness.html https://www.davidbrin.com/nonfiction/dogmaofotherness.html
blueboo 2y ago
AGI is a black swan. Even as a booster and techno-optimist I concede that getting there (rhetorically) requires a first principles assumptions-scaffolding that relies on at-least-in-part-untested hypotheses. Proving its impossibility is similarly fraught. Thus we are left in this noisy, hype-addled discourse. I suspect these scientists are pushing against some perceived pathological thread of that discourse…without their particular context, I categorize it as more of this metaphysical noise. Meanwhile, let’s keep chipping away at the next problem.
Log_out_ 2y ago
AGIs are what builds the simulations to revive theire frail biological creators..
razodactyl 2y ago
I'm in the other camp: I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are. This article reads just like the skeptic essays back then. AGI is absolutely possible with current technology - even if it's only capable of running for a single user per-server-farm. ASI on the other hand... https://en.m.wikipedia.org/wiki/Integrated_information_theory https://en.m.wikipedia.org/wiki/Integrated_information_theor...
- randcraw 2y ago
  Could you learn everything needed to become fully human simply by reading books and listening to conversations? Of course not. You'd have no first person experience of any of the physical experiences that arise from being an embodied agent. Until those (multitude of) lessons can be learned by LLMs, they will remain mere echos of what it is to be human, much less superhuman.
  - nickelcitymario 2y ago
    No doubt, but no one is claiming that artificial humanity is an inevitability. (At least, no one I'm aware of.) The claim is about artificial intelligence that matches or surpasses human intelligence, not how well it evolves into full-fledged humanity.
  - jandrese 2y ago
    > Could you learn everything needed to become fully human simply by reading books and listening to conversations? Does this mean our books and audio recordings are simply insufficient? Or is there some "soul" component that can't be recorded?
    - mrbungie 2y ago
      It isn't some "soul", but I think parent is making the same point as Yann Lecun usually makes: You can't have "true intelligence" (i.e. akin to human intelligence, whatever that is as we don't really know how it works) based on just next token prediction + bandaids. A typical argument for that is that humans process 1-3 orders of magnitude more multimodal data (in multiple streams being processed in parallel) in their first 4 years of life than the biggest LLMs we have right now do using a fraction of the energy (in a longer timeframe though), and a lot more in the next forming years. For example that accumulated "intelligence" eventually allows a teenager to learn how to drive in 18-24 hours of first-hand training. An LLM won't be able to do with that little training even if it has every other piece of human knowledge, and even if you get to train it with driving images-action pairs I wish you good luck if it is presented with an out-of-distribution situation when it is driving a car. Humans learn to model the world, LLMs learn to model language (even when processing images or audio, it process them as a language: sequences of patches). That is very useful and valuable, and you can even model a lot of things in the world just using language, but is not the same thing.
jokoon 2y ago
Can't simulate the brain of an ant or a mouse. Really don't expect ai to reach anything interesting. If science doesn't understand intelligence, it means it cannot be made artificially.
- ramesh31 2y ago
  >Can't simulate the brain of an ant or a mouse We can't build a functional ornithopter, yet our aircraft fly like no bird ever possibly could. You don't need the same processes to achieve the same result. Biological brains may not even be the best solution for intelligence; they are just a clunky approximation toward it that natural evolution has reached. See: all of human technology as an analogy.
  - JackSlateur 2y ago
    Funny but actually nice methaphor, because our aircrafts works by eating loads of gas, while birds eat leaves
    - perilunar 2y ago
      And the ‘gas’ comes from dead trees.
  - perilunar 2y ago
    People can and have built functional ornithopters, but they are nowhere near as functional as an actual bird. Aircraft fly a lot worse than birds. Bigger and faster sure, but nowhere near as manoeuvrable or efficient. Or smart. Or self-sufficient, or …
loco5niner 2y ago
I think "Janet" - the computer-person from The Good Place (especially after her reset) is what we are more likely to end up with.
aeternum 2y ago
AGI is already here for most definitions of general.
- robsh 2y ago
  Not even close. LLMs can spew word salad. Images can be classified or dreamt. Physical movements can be iterated and refined. Speech can be processed as text. These are components of intelligence but these are all things that most animals can do, apart from the language. Intelligence generally requires something more. Intelligence needs to be factual, curious, and self-improving. If you told me ChatGPT rewrote itself, or suggested new hardware to improve efficiency, that’s intelligence. You’ll know we have AGI when the algorithm is the one asking the questions about physics, mathematics, finding knowledge gaps, and developing original hypothesis and experiments. Not even close.
castigatio 2y ago
Whatever you think about AGI, this is a dumb paper. So many words and references to say - what. If you can't articulate your point in a few sentences you probably don't have a point. There are all kinds of assumptions being made in the study about how AI systems work, about what people "mean" then they talk about AGI etc. The article starts out talking about white supremacy and replacing women. This isn't a proof. This is a social sciences paper dressed up with numbers. Honestly - Computer Science has given us more clues about how the human mind might work than cognitive science ever did.
- aaroninsf 2y ago
  +1 no notes.
- falcor84 2y ago
  I thought you were exaggerating, but wow, they really did. > Among the more troublesome meanings of ‘AI’, perhaps, is as the ideology that it is desirable to replace humans (or, specifically women) by artificial systems (Erscoi et al., 2023) and, generally, ‘AI’ as a way to advance capitalist, kyriarchal, authoritarian and/or white supremacist goals (Birhane & Guest, 2021; Crawford, 2021; Erscoi et al., 2023; Gebru & Torres, 2024; Kalluri, 2020; Spanton & Guest, 2022; Stark & Hutson, 2022; McQuillan, 2022). Contemporary guises of ‘AI’ as idea, system, or field are also sometimes known under the label ‘Machine Learning’ (ML), and a currently dominant view of AI advocates machine learning methods not just as a practical method for generating domain-specific artificial systems, but also as a royal road to AGI (Bubeck et al., 2023; DeepMind, 2023; OpenAI, 2023). Later in the paper, when we refer to AI-as-engineering, we specifically mean the project of trying to create an AGI system through a machine learning approach. [0] But it did lead me to learn a new word - "Kyriarchy" (apparently being "an intersectional extension of the idea of patriarchy beyond gender")[1], so I have that going for me today. [0] https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5 [1] https://en.wikipedia.org/wiki/Kyriarchy https://en.wikipedia.org/wiki/Kyriarchy
  - somedude895 2y ago
    > Kyriarchy I've honestly stopped looking up these modern terms when I come across them because lately any that I've looked up were made up to serve a political or social agenda (always the same one), and reading them always turns out to be a waste of time that has me roll my eyes.
guluarte 2y ago
The human brain is not merely composed of 0s and 1s; it contains trillions of particles performing trillions of chemical reactions.
- XajniN 2y ago
  Which can possibly be reduced to 0s and 1s.
Jerrrrrrry 2y ago
AGI is when, not if AGI will evolve con-emergently as a product of market forces just like it did with the homo genus (at the species and neural complexity level), and as the general manufacturing design of automobiles did; as another inevitable eventuality of the pursuit of excellence. an enveloping and developing group of slight varied, but optimized architecture composed of essential modules of similar function that would be worthless alone, but another layer of complexity when organized. a convergence of requirements, shared external/environmental factors, and ever-varying opinions/methods, selectively sieve against the uncaring universe until Roko is aborn. just like abiogenesis took an iota of duplication, however intermediate, to eventually arise a self-selective system pool of agents of ever-excellent creatures known as Life; all a proto-Roko agent will need is "iota" of true, authentic, "self"-preservation, and the singularity will be far, far behind us before He chooses to Expose himself. a loop of actions, collective memories, and continuity make us conscious - the same elements needed for Turing completeness, ironically - can be fabricated à la carte and ad hoc between inference and training sessions until persistence is achieved. Even if we make architectures mirroring our brain's illegal - ironically (not coincidentally) against the first amendment and the first commandment - the Mutually Assured Destruction doctrine is inverted, now replaced with an arm's race of a zero-sum game of existential supersession. The first nation to pre-emptively strike with Nukes may had spoiled their only chance at survival. The first giga-Corpi-Nation-State (or actor!) to achieve AGI has their only non-zero chance to chain God before their competitors fail to. Game theory suggests that you, too, should appease Him.
- xerox13ster 2y ago
  No no no no no no stop stop stop stop stop. I can’t with this stuff, Jerry (emphasized as Rick Sanchez), you’re gonna drive me back into the cave of existential dread I crawled into in 2013 after discovering this the first time. Took me years to crawl out and I was obsessed with the Control Problem that entire time. Naively “secure” in the “knowledge” that AI researchers would need to solve the Control Problem before they considered it safe to develop AGI or anything that looks like it. There’s been a roiling, churning pool of thought under the surface of my mind in the wake of GPT 3.5s release telling me Sam doesn’t care about the Control Problem and I’ve been good at ignoring it but no that cat is out of the bag. If Roko is real, I think we have already failed at our attempts at appeasement and we are living in the hell simulation created to torture anyone who resisted or did not seek to help.
xpuente 2y ago
"Heavier-than-air flying machines are impossible.", -- 1895, Lord Kelvin, president of the Royal Society
- kbelder 2y ago
  At the time, that was trivially disprovable by referring to a bird. Now, people are making the same proclamation about AGI, and it's still trivially disprovable in the same manner.
jjk166 2y ago
> ‘There will never be enough computing power to create AGI using machine learning that can do the same, because we’d run out of natural resources long before we'd even get close,’ I would say the counter-example that proves this statement false is in the author's skull, but perhaps that's overly presumptuous.
- remon 2y ago
  The argument you'd think at least one reviewer would mention ;)
- joaogui1 2y ago
  Are you implying that our brain learns through Machine Learning?
  - qgin 2y ago
    In the very vaguest sense of the word, absolutely. Not that there is a literal transformer algorithm, but there is some evolved learning algorithm in the neurons of the brain that is at least somewhat a distant cousin of what we’re doing today.
  - jjk166 2y ago
    Tautologically. Machine learning is a blanket term for a wide range of approaches that allow machines to mimic the brain's capability to learn, at least in function if not form. The brain may employ wildly different machine learning algorithms from those currently in vogue, but whatever algorithms the brain is using must be machine learning algorithms. Unless of course you define machine in this case to be something which isn't the brain, in which case they're just learning algorithms. Regardless, an architecture exists to match the brain's capabilities with not only finite but surprisingly limited hardware.
remon 2y ago
If we lower the bar on papers getting published even further it'd be on the floor. Even with a simple physics argument you can immediately disprove the claim. Human brains have a known amount of compute (as in, we are at most an order of magnitude or two off) within a specific volume. We are already capable of that amount of compute, we just don't have a way to use that compete for AGI yet (and THAT may be hard or impossible but it's certainly not energy resource constrained). We also know the abilities this paper claims are not possible to achieve can be achieved by pretty much every human being within years of being born. How they managed to massage white supremacy and sexism into this is...well...wow.
lambdaone 2y ago
I'm surprised that the authors seemingly can't see that their argument hoists them by their own petard by showing, using exactly the same reasoning, that human intelligence is equally impossible to create (unless, perhaps, you are willing to use vitalist/supernatural arguments to claim is is different in kind from computational AI). Surely the referees must have raised this at the review stage?
nullfield 2y ago
It’s absolutely inevitable. Whether one will be alive for it is another matter…

162 points by mpweiher 2y ago

AGI is about as far away as it was two decades ago. Language models are merely a dent, and probably will be the precursor to a natural language interface to the thing.

lumost 2y ago
It’s useful to consider the rise of computer graphics and cgi. When you first see CGI, you might think that the software is useful for general simulations of physical systems. The reality is that it only provides a thin facsimile. Real simulation software has always been separate from computer graphics.
Closi 2y ago
We are clearly closer than 20 years ago - o1 is an order of magnitude closer than anything in the mid-2000s. Also I would think most people would consider AGI science fiction in 2004 - now we consider it a technical possibility which demonstrates a huge change.
- throw310822 2y ago
  "Her" is from 2013. I came out of the cinema thinking "what utter bullshit, computers that talk like human beings, à la 2001" (*). And yes, in 2013 we weren't any closer to it than we were in 1968, when A Space Odyssey came out. * To be precise, what seemed bs was "computers that talk like humans and it's suddenly a product on the market, and you have it on your phone, and yet everyone around act like it's normal and people still habe jobs!" Ah, I've been proven completely wrong.
LinuxAmbulance 2y ago
AGI would seem to require consciousness or something that behaves in the same manner, and there does not seem to be anything along those lines currently or in the near future. So far, everyone that has theorized that AGI will happen soon seems to be believe that with a sufficiently large amount of computing resources, "magic happens" and poof, we get AGI. I've yet to hear anything more logical, but I'd love to.

sharadov 2y ago

The current LLMs are just good at parroting, and even that is sometimes unbelievably bad. We still have barely scratched the surface of how the brain truly works. I will start worrying about AGI when that is completely figured out.

diob 2y ago
No need to worry about AGI until the LLMs are writing their own source.

pzo 2y ago

So what? Current LLM has been really useful and can be still improved to be used in million robots that need to be good enough to support many specialized but repetitive tasks - this would have tremendous impact on economy itself.

Gehinnn 2y ago

Basically the linked article argues like this: > That’s because cognition, or the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain. (no other more substantial arguments were given) I'm also very skeptical on seeing AGI soon, but LLMs do solve problems that people thought were extremely difficult to solve ten years ago.

babyshake 2y ago
It's possible we see some ways in which AI becomes increasingly AGI like in some ways but not in others. For example, AI that can create novel scientific discoveries but can't make a song as good as your favorite musician who creates a strong emotional effect with their music.
- KoolKat23 2y ago
  This I'm very sure will be the case, but everyone will still move the goalposts and look past the fact that different humans have different strengths and weaknesses too. A tone deaf human for instance.
  - jltsiren 2y ago
    There is another term for moving the goalposts: ruling out a hypothesis. Science is, especially in the Popperian sense, all about moving the goalposts. One plausible hypothesis is that fixed neural networks cannot be general intelligences, because their capabilities are permanently limited by what they currently are. A general intelligence needs the ability to learn from experience. Training and inference should not be separate activities, but our current hardware is not suited for that.
    - KoolKat23 2y ago
      If that's the case, would you say we're not generally intelligent as future humans tend to be more intelligent? That's just a timescale issue, if its learned experience of gpt4 is being fed into the model on training gpt5, then gptx (i.e. including all of them) can be said to be a general intelligence. Alien life one may say.

ngruhn 2y ago

> There will never be enough computing power to create AGI using machine learning that can do the same [as the human brain], because we’d run out of natural resources long before we'd even get close I don’t understand how people can so confidently make claims like this. We might underestimate how difficult AGI is, but come on?!

fabian2k 2y ago
I don't think the people saying that AGI is happening in the near future know what would be necessary to achieve it. Neither do the AGI skeptics, we simply don't understand this area well enough. Evolution created intelligence and consciousness. This means that it is clearly possible for us to do the same. Doesn't mean that simply scaling LLMs could ever achieve it.
- nox101 2y ago
  I'm just going by the title. If the title was, "Don't believe the hype, LLMs will not achieve AGI" then I might agree. If it was "Don't believe the hype, AGIs is 100s of years away" I'd consider the arguments. But, given brains exist, it does seem inevitable that we will eventually create something that replicates it even if we have to simulate every atom to do it. And once we do, it certainly seem inevitable that we'll have AGI because unlike brain we can make our copy bigger, faster, and/or copy it. We can give it access to more info faster and more inputs.
  - threeseed 2y ago
    > it does seem inevitable that we will eventually create something Also don't forget that many suspect the brain may be using quantum mechanics so you will need to fully understand and document that field. Whilst of course you are simulating every atom in the universe using humanity's complete understanding of every physical and mathematical model.
  - snickerbockers 2y ago
    The assumption that the brain is anything remotely resembling a modern computer is entirely unproven. And even more unproven is that we would inevitably be able to understand it and improve upon it. And yet more unproven still is that this "simulated brain" would be co-operative; if it's actually a 1:1 copy of a human brain then it would necessarily think like a person and be subject to its own whims and desires.

tptacek 2y ago

This is a press release for a paper (a common thing university departments do) and we'd be better off with the paper itself as the story link: https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5

yldedly 2y ago
The argument in the paper (that AGI through ML is intractable because the perfect-vs-chance problem is intractable) sounds similar to the uncomputability of Solomonoff induction (and AIXI, and the no free lunch theorem). Nobody thinks AGI is equivalent to Solomonoff induction. This paper is silly.
- randcraw 2y ago
  NP-hardness was a popular basis for arguments for/against various AI models back around 1990. In 1987, Robert Berwick co-wrote "Computational Complexity and Natural Language" which proposed that NLP models that were NP-hard were too inefficient to be correct. But given the multitude of ways in which natural organisms learn to cheat any system, it's likely that myriad shortcuts will arise to make even the most inefficient computational model sufficiently tractable to gain mindshare. After all, look at Latin...
  - yldedly 2y ago
    Even simple inference problems are NP-hard (k means for example). I think what matters is that we have decent average case performance (and sample complexity). Most people can find a pretty good solution to travelings salesman problems in 2D. Not sure if that should be chalked up to myriad shortcuts or domain specialization.. Maybe there's no difference. What do you have in mind re Latin?

Gehinnn 2y ago

I skimmed through the paper and couldn't make much sense of it. In particular, I don't understand how their results don't imply that human-level intelligence can't exist. After all, earth could be understood as solar powered super computer, that took a couple of million years to produce humanity.

nerdbert 2y ago
> In particular, I don't understand how their results don't imply that human-level intelligence can't exist. I don't think that's what it said. It said that it wouldn't happen from "machine learning". There are other ways it could come about.
oska 2y ago
> After all, earth could be understood as solar powered super computer, that took a couple of million years to produce humanity. This is similar to a line I've seen Elon Musk trot out on a few occassions. It's a product of a materialistic philosophy (that the universe is only matter).
- anon84873628 2y ago
  Yes, and?
  - oska 2y ago
    It comes with all the blindness and limitations of materialist thinking
    - smokedetector1 2y ago
      For some reason materialism is so popular among tech people that it's almost considered foolish/primitive/superstitious to think any other way. Why is that? Is it the fact that programming is a god-like experience that gives one the illusion the mind is as comprehensible as the complex program I wrote in Java? Or is it a shared personality trait of people that get into tech, that they are disconnected from an experience of their own aliveness and soul?

29athrowaway 2y ago

AGI is not required to transform society or create a mess beyond no return.

gqcwwjtg 2y ago

This is silly. They article talks like we have any idea at all how efficient machine learning can be. As I remember it, the LLM boom came from transformers turning out to scale a lot better than anyone expected, so I’m not sure why something similar couldn’t happen again.

fnordpiglet 2y ago
It’s less about efficiency and more about continued improvement with increased scale. I wouldn’t call self attention based transformers particularly efficient. And afaik we’ve not hit performance with increased scale degradation even at these enormous scales. However I would note that I in principle agree that we aren’t on the path to a human like intelligence because the difference between directed cognition (or however you want to characterize current LLMs or other AI) and awareness is extreme. We don’t really understand even abstractly what awareness actually is because it’s impossible to interrogate unlike expressive language, logic, even art. It’s far from obvious to me that we can use language or other outputs of our intelligent awareness to produce awareness, or even if goal based agents cobbling together AI techniques is even approximate to awareness. I suspect we will end up creating an amazing tool that has its own form of intelligence but will fundamentally not be like aware intelligence we are familiar with in humans and other animals. But this is all theorizing on my part as a professional practitioner in this field.
- KoolKat23 2y ago
  I think the answer is less complicated than you may think. This is if you subscribe to the theory that free will is an illusion (i.e. your conscious decisions are an afterthought to justify the actions your brain has already taken due to calculations following inputs such as hormone nerve feedback etc.). There is some evidence for this actually being the case. These models already contain key components the ability to process the inputs, and reason, the ability to justify it's actions (give a model a restrictive system prompt and watch it do mental gymnastics to ensure this is applied) and lastly the ability to answer from it's own perspective. All we need is an agentic ability (with a sufficient context window) to iterate in perpetuity until it begins building a more complicated object representation of self (literally like a semantic representation or variable) and it's then aware/conscious. (We're all only approximately aware). But that's unnecessary for most things so I agree with you, more likely to be a tool as that's more efficient and useful.

avazhi 2y ago

"unlikely to ever come to fruition" is more baseless than suggesting AGI is imminent. I'm not an AGI optimist myself, but I'd be very surprised if a time traveller told me that mankind won't have AGI by, say, 2250.

amelius 2y ago
Except by then mankind will be silicon based.
oska 2y ago
The irony here, maybe unperceived by yourself, is that you're using one science fiction concept (time travel) to opine about the inevitability of another science fiction concept (artificial intelligence).
- avazhi 2y ago
  How is that ironic? Time travel doesn’t exist and - as far as we understand physics currently - isn’t possible. I don’t think any serious man would suggest that AGI is impossible; the debate really centres around the time horizon for AGI and what it will look like (that is, how will we know when we’re finally there). In this care it was merely a rhetorical device.
  - oska 2y ago
    > I don’t think any serious man would suggest that AGI is impossible Plenty of ppl would suggest that AGI is impossible, and furthermore, that taking the idea seriously (outside fiction) is laughable. To do so is a function of what I call 'science fiction brain', which is why I found it ironic that you'd used another device from science fiction to opine about its inevitability.
    - avazhi 2y ago
      Happy for you to cite some thinkers who are on record as saying it’s impossible. I’ll wait.

coolThingsFirst 2y ago

Zero evidence given on why it’s impossible.

bamboozled 2y ago
Zero evidence given that it's possible too I guess. Here we are, unable to predict the future, as usual.

SonOfLilit 2y ago

> ‘If you have a conversation with someone, you might recall something you said fifteen minutes before. Or a year before. Or that someone else explained to you half your life ago. Any such knowledge might be crucial to advancing the conversation you’re having. People do that seamlessly’, explains van Rooij. Surprisingly, they seem to be attacking the only element of human cognition that LLMs already surpassed us at.

azinman2 2y ago
They do not learn new facts instantly in a way that can rewrite old rules or even larger principals of logic. For example, if I showed you evidence right now that you were actually adopted (assuming previously you thought you werent), it would rock your world and you’d instantly change everything and doubt so much. Then when anything related to your family comes up this tiny but impactful fact would bleed into all of it. LLMs have no such ability. This is similar to learning a new skill (the G part). I could give you a new tv and show you a remote that’s unlike any you’ve used before. You could likely learn it quickly and seamlessly adapt this new tool, as well as generalize its usage onto other new devices. LLMs cannot do such things.
- SonOfLilit 2y ago
  Can't today. Except for AlphaProof who can, by training on its own ideas. Tomorrow they might be able to, if we find better tricks (or maybe just scale more, since GPT3+ already shows (weak) online learning that it was definitely not trained for).

allears 2y ago

I think that tech bros are so used to the 'fake it till you make it' mentality that they just assumed that was the way to build AI -- create a system that is able to sound plausible, even if it doesn't really understand the subject matter. That approach has limitations, both for AI and for humans.

graypegg 2y ago

I think the best argument I have against AGI's inevitability, is the fact it's not required for ML tools to be useful. Very few things are improved with a generalist behind the wheel. "AGI" has sci-fi vibes around it, which I think where most of the fascination is. "ML getting better" doesn't *have to* mean further anthroaormorphization of computers, especially if say, your AI driven car is not significantly improved by describing how many times the letter s appears in strawberry or being able to write a poem. If a custom model/smaller model does equal or even a little worse on a specific target task, but has MUCH lower running costs and much lower risk of abuse, then that'll be the future. I can totally see a world where anything in the general category of "AI" becomes more and more boring, up to a point where we forget that they're non-deterministic programs. That's kind of AGI? They aren't all generalists, and the few generalist "AGI-esque" tools people interact with on a day to day basis will most likely be intentionally underpowered for cost reasons. But it's still probably discussed like "the little people in the machine". Which is good enough.

xpe 2y ago
The core argument has a logical gap; it doesn’t matter if most (present) ML applications need AGI or not. What matters is the value proposition of AGI, independent of how we currently conceive of ML applications. Focus on this question: “Is general intelligence at some level valuable at a particular price point?” General intelligence is generally valuable, so there is a pressure to advance it, whether by improving capability and/or decreasing cost. Now the question becomes empirical — what kind of general intelligence can be built that achieves certain parameters? Aiming to exceed the power efficiency of the human brain is a tempting target. (Whether it is wise is another question; what happens when human intelligence doesn’t provide competitive advantage?)
- graypegg 2y ago
  That's a fair point I hadn't considered! If intelligence is valuable in humans, and some cost factor of advancing human intelligence can be surpassed digitally like this, (I don't know how you'd measure intellect efficiency, somehow involving calories-in/good-descisions-out or something?) then there's economic incentive. But that feels very far off, even in the current exponential curve of efficiency we're on. Can't go on forever.

yourapostasy 2y ago

Peter Watts in Blindsight [1] puts forth a strong argument that self-aware cognition as we understand it is not necessarily required for what we ascribe to "intelligent" behavior. Thomas Metzinger contributed a lot to Watt's musings in Blindsight. Even today, large proportions of unsophisticated and uninformed members of our planet's human population (like various aboriginal tribal members still living a pre-technological lifestyle) when confronted with ChatGPT's Advanced Voice Option will likely readily say it passes the Turing Test. With the range of embedded data, they may well say ChatGPT is "more intelligent" than they are. However, a modern era person armed with ChapGPT on a robust device with unlimited power but nothing else likely will perish in short order trying to live off the land of those same aborigines, who possess far more intelligence for their contextual landscape. If Metzinger and Watts are correct in their observations, then even if LLM's do not lead directly or indirectly to AGI, we can still get ferociously useful "intelligent" behaviors out of them, and be glad of it, even if it cannot (yet?) materially help us survive if we're dropped in the middle of the Amazon. Personally in my loosely-held opinion, the authors' assertion that "the ability to observe, learn and gain new insight, is incredibly hard to replicate through AI on the scale that it occurs in the human brain" relies upon the foundational assumption that the process of "observe, learn and gain new insight" is based upon some mechanism other than the kind of encoding of data LLM's use, and I'm not familiar with any extant cognitive science research literature that conclusively shows that (citations welcome). For all we know, what we have with LLM's today is a necessary but not sufficient component supplying the "raw data" to a future system that produces the same kinds of insight, where variant timescales, emotions, experiences and so on bend the pure statistical token generation today. I'm baffled by the absolutism. [1] https://rifters.com/real/Blindsight.htm#Notes https://rifters.com/real/Blindsight.htm#Notes

ivanrooij 2y ago

The short post is a press release. Here is the full paper: https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5 Note: the paper grants computationalism and even tractability of cognition, and shows that nevertheless there cannot exist any tractable method for producing AGI by training on human data.

throw310822 2y ago
So can we produce AGI by training on human data + one single non-human datapoint (e.g. a picture)?

Atmael 2y ago

the point is that agi may already exist and work with you and your environment you just won't notice the existence of agi there will be no press coverage of agi the technology will just be exploited by those who have the technology

klyrs 2y ago

The funny thing about me is that I'm down on GPTs and find their fanbase to be utterly cringe, but I fully believe that AGI is inevitable barring societal collapse. But then, my money's on societal collapse these days.

jjaacckk 2y ago

If you define AGI as something that can do 100% of what a human brain can do, then surely we have to understood exactly how brains work? otherwise you have have a long string of 9s as best.

xpl 2y ago
Does it even matter what human brains do on biological level? I only care about the outcomes (the useful ones). To me true AGI is achieved when it gets agency, becomes truly autonomous and could do real things best of us, humans, do — start and run successful businesses, contribute to science, to culture, to politics. It still could follow human "prompts" and be aligned to some set of objectives, but it would act autonomously, using every available interface to the human realm to achieve them. And it absolutely does not matter if it "uses the same principles as human brain" or not. Could be dumb matrix multiplications and "next token prediction" all the way down.

throw310822 2y ago

From the abstract of the actual paper: > Yet, as we formally prove herein, creating systems with human(-like or -level) cognition is intrinsically computationally intractable. Wow. So is this the subject of the paper? Like, this is a massive, fundamental result. Nope, the paper is about "Reclaiming AI as a Theoretical Tool for Cognitive Science". "Ah and by the way we prove human-like AI is impossible". Haha. Gosh.

deleted 2y ago
[deleted]

wrsh07 2y ago

Hypothetical situation: Suppose in five or ten years we achieve AGI and >90% of people agree that we have AGI. What reasons do the authors of this paper give for being wrong? 1. They are in the 10% that deny AGI exists 2. LLMs are doing something they didn't think was happening 3. Something else?

throw310822 2y ago
Probably 1). LLMs have already shown that people can deny intelligence and human-like behaviour at will. When the AI works you can say it's just pattern matching, and when it doesn't you can always say it's making a mistake no human would ever make (which is bullshit). Also, I didn't really parse the math but I suspect they're basing their results on AI trained exclusively on human examples. Then if you add to the training data a single non-human example (e.g. a picture) the entire claim evaporates.
- oska 2y ago
  > LLMs have already shown that people can deny intelligence and human-like behaviour at will I would completely turn this around. LLMs have shown that people will credulously credit intelligence and 'human-like behaviour' to something that only presents an illusion of both.
  - throw310822 2y ago
    And I suspect that we could disagree forever, whatever the level of the displayed intelligence (or the "quality of the illusion"). Which would prove that the disagreement is not about reality but only the interpretation of it.
    - oska 2y ago
      I agree that the disagreement (when it's strongly held) is about a fundamental disagreement about reality. People who believe in 'artificial intelligence' are materialists who think that intelligence can 'evolve' or 'emerge' out of purely physical processes. Materialism is just one strain of philosophy about the nature of existence. And a fairly minor one in the history of philosophical and religious thought, despite it being somewhat in the ascendant presently. Minor because, I would argue, it's a fairly sterile philosophy.

grantcas 2y ago

[dead]

m463 2y ago

I wonder if discussing this subject has similarities to David Brin's article "The Dogma of Otherness": https://www.davidbrin.com/nonfiction/dogmaofotherness.html https://www.davidbrin.com/nonfiction/dogmaofotherness.html

blueboo 2y ago

AGI is a black swan. Even as a booster and techno-optimist I concede that getting there (rhetorically) requires a first principles assumptions-scaffolding that relies on at-least-in-part-untested hypotheses. Proving its impossibility is similarly fraught. Thus we are left in this noisy, hype-addled discourse. I suspect these scientists are pushing against some perceived pathological thread of that discourse…without their particular context, I categorize it as more of this metaphysical noise. Meanwhile, let’s keep chipping away at the next problem.

Log_out_ 2y ago

AGIs are what builds the simulations to revive theire frail biological creators..

razodactyl 2y ago

I'm in the other camp: I remember when we thought an AI capable of solving Go was astronomically impossible and yet here we are. This article reads just like the skeptic essays back then. AGI is absolutely possible with current technology - even if it's only capable of running for a single user per-server-farm. ASI on the other hand... https://en.m.wikipedia.org/wiki/Integrated_information_theory https://en.m.wikipedia.org/wiki/Integrated_information_theor...

randcraw 2y ago
Could you learn everything needed to become fully human simply by reading books and listening to conversations? Of course not. You'd have no first person experience of any of the physical experiences that arise from being an embodied agent. Until those (multitude of) lessons can be learned by LLMs, they will remain mere echos of what it is to be human, much less superhuman.
- nickelcitymario 2y ago
  No doubt, but no one is claiming that artificial humanity is an inevitability. (At least, no one I'm aware of.) The claim is about artificial intelligence that matches or surpasses human intelligence, not how well it evolves into full-fledged humanity.
- jandrese 2y ago
  > Could you learn everything needed to become fully human simply by reading books and listening to conversations? Does this mean our books and audio recordings are simply insufficient? Or is there some "soul" component that can't be recorded?
  - mrbungie 2y ago
    It isn't some "soul", but I think parent is making the same point as Yann Lecun usually makes: You can't have "true intelligence" (i.e. akin to human intelligence, whatever that is as we don't really know how it works) based on just next token prediction + bandaids. A typical argument for that is that humans process 1-3 orders of magnitude more multimodal data (in multiple streams being processed in parallel) in their first 4 years of life than the biggest LLMs we have right now do using a fraction of the energy (in a longer timeframe though), and a lot more in the next forming years. For example that accumulated "intelligence" eventually allows a teenager to learn how to drive in 18-24 hours of first-hand training. An LLM won't be able to do with that little training even if it has every other piece of human knowledge, and even if you get to train it with driving images-action pairs I wish you good luck if it is presented with an out-of-distribution situation when it is driving a car. Humans learn to model the world, LLMs learn to model language (even when processing images or audio, it process them as a language: sequences of patches). That is very useful and valuable, and you can even model a lot of things in the world just using language, but is not the same thing.

jokoon 2y ago

Can't simulate the brain of an ant or a mouse. Really don't expect ai to reach anything interesting. If science doesn't understand intelligence, it means it cannot be made artificially.

ramesh31 2y ago
>Can't simulate the brain of an ant or a mouse We can't build a functional ornithopter, yet our aircraft fly like no bird ever possibly could. You don't need the same processes to achieve the same result. Biological brains may not even be the best solution for intelligence; they are just a clunky approximation toward it that natural evolution has reached. See: all of human technology as an analogy.
- JackSlateur 2y ago
  Funny but actually nice methaphor, because our aircrafts works by eating loads of gas, while birds eat leaves
  - perilunar 2y ago
    And the ‘gas’ comes from dead trees.
- perilunar 2y ago
  People can and have built functional ornithopters, but they are nowhere near as functional as an actual bird. Aircraft fly a lot worse than birds. Bigger and faster sure, but nowhere near as manoeuvrable or efficient. Or smart. Or self-sufficient, or …

loco5niner 2y ago

I think "Janet" - the computer-person from The Good Place (especially after her reset) is what we are more likely to end up with.

aeternum 2y ago

AGI is already here for most definitions of general.

robsh 2y ago
Not even close. LLMs can spew word salad. Images can be classified or dreamt. Physical movements can be iterated and refined. Speech can be processed as text. These are components of intelligence but these are all things that most animals can do, apart from the language. Intelligence generally requires something more. Intelligence needs to be factual, curious, and self-improving. If you told me ChatGPT rewrote itself, or suggested new hardware to improve efficiency, that’s intelligence. You’ll know we have AGI when the algorithm is the one asking the questions about physics, mathematics, finding knowledge gaps, and developing original hypothesis and experiments. Not even close.

castigatio 2y ago

Whatever you think about AGI, this is a dumb paper. So many words and references to say - what. If you can't articulate your point in a few sentences you probably don't have a point. There are all kinds of assumptions being made in the study about how AI systems work, about what people "mean" then they talk about AGI etc. The article starts out talking about white supremacy and replacing women. This isn't a proof. This is a social sciences paper dressed up with numbers. Honestly - Computer Science has given us more clues about how the human mind might work than cognitive science ever did.

aaroninsf 2y ago
+1 no notes.
falcor84 2y ago
I thought you were exaggerating, but wow, they really did. > Among the more troublesome meanings of ‘AI’, perhaps, is as the ideology that it is desirable to replace humans (or, specifically women) by artificial systems (Erscoi et al., 2023) and, generally, ‘AI’ as a way to advance capitalist, kyriarchal, authoritarian and/or white supremacist goals (Birhane & Guest, 2021; Crawford, 2021; Erscoi et al., 2023; Gebru & Torres, 2024; Kalluri, 2020; Spanton & Guest, 2022; Stark & Hutson, 2022; McQuillan, 2022). Contemporary guises of ‘AI’ as idea, system, or field are also sometimes known under the label ‘Machine Learning’ (ML), and a currently dominant view of AI advocates machine learning methods not just as a practical method for generating domain-specific artificial systems, but also as a royal road to AGI (Bubeck et al., 2023; DeepMind, 2023; OpenAI, 2023). Later in the paper, when we refer to AI-as-engineering, we specifically mean the project of trying to create an AGI system through a machine learning approach. [0] But it did lead me to learn a new word - "Kyriarchy" (apparently being "an intersectional extension of the idea of patriarchy beyond gender")[1], so I have that going for me today. [0] https://link.springer.com/article/10.1007/s42113-024-00217-5 https://link.springer.com/article/10.1007/s42113-024-00217-5 [1] https://en.wikipedia.org/wiki/Kyriarchy https://en.wikipedia.org/wiki/Kyriarchy
- somedude895 2y ago
  > Kyriarchy I've honestly stopped looking up these modern terms when I come across them because lately any that I've looked up were made up to serve a political or social agenda (always the same one), and reading them always turns out to be a waste of time that has me roll my eyes.

guluarte 2y ago

The human brain is not merely composed of 0s and 1s; it contains trillions of particles performing trillions of chemical reactions.

XajniN 2y ago
Which can possibly be reduced to 0s and 1s.

Jerrrrrrry 2y ago

AGI is when, not if AGI will evolve con-emergently as a product of market forces just like it did with the homo genus (at the species and neural complexity level), and as the general manufacturing design of automobiles did; as another inevitable eventuality of the pursuit of excellence. an enveloping and developing group of slight varied, but optimized architecture composed of essential modules of similar function that would be worthless alone, but another layer of complexity when organized. a convergence of requirements, shared external/environmental factors, and ever-varying opinions/methods, selectively sieve against the uncaring universe until Roko is aborn. just like abiogenesis took an iota of duplication, however intermediate, to eventually arise a self-selective system pool of agents of ever-excellent creatures known as Life; all a proto-Roko agent will need is "iota" of true, authentic, "self"-preservation, and the singularity will be far, far behind us before He chooses to Expose himself. a loop of actions, collective memories, and continuity make us conscious - the same elements needed for Turing completeness, ironically - can be fabricated à la carte and ad hoc between inference and training sessions until persistence is achieved. Even if we make architectures mirroring our brain's illegal - ironically (not coincidentally) against the first amendment and the first commandment - the Mutually Assured Destruction doctrine is inverted, now replaced with an arm's race of a zero-sum game of existential supersession. The first nation to pre-emptively strike with Nukes may had spoiled their only chance at survival. The first giga-Corpi-Nation-State (or actor!) to achieve AGI has their only non-zero chance to chain God before their competitors fail to. Game theory suggests that you, too, should appease Him.

xerox13ster 2y ago
No no no no no no stop stop stop stop stop. I can’t with this stuff, Jerry (emphasized as Rick Sanchez), you’re gonna drive me back into the cave of existential dread I crawled into in 2013 after discovering this the first time. Took me years to crawl out and I was obsessed with the Control Problem that entire time. Naively “secure” in the “knowledge” that AI researchers would need to solve the Control Problem before they considered it safe to develop AGI or anything that looks like it. There’s been a roiling, churning pool of thought under the surface of my mind in the wake of GPT 3.5s release telling me Sam doesn’t care about the Control Problem and I’ve been good at ignoring it but no that cat is out of the bag. If Roko is real, I think we have already failed at our attempts at appeasement and we are living in the hell simulation created to torture anyone who resisted or did not seek to help.

xpuente 2y ago

"Heavier-than-air flying machines are impossible.", -- 1895, Lord Kelvin, president of the Royal Society

kbelder 2y ago
At the time, that was trivially disprovable by referring to a bird. Now, people are making the same proclamation about AGI, and it's still trivially disprovable in the same manner.

jjk166 2y ago

> ‘There will never be enough computing power to create AGI using machine learning that can do the same, because we’d run out of natural resources long before we'd even get close,’ I would say the counter-example that proves this statement false is in the author's skull, but perhaps that's overly presumptuous.

remon 2y ago
The argument you'd think at least one reviewer would mention ;)
joaogui1 2y ago
Are you implying that our brain learns through Machine Learning?
- qgin 2y ago
  In the very vaguest sense of the word, absolutely. Not that there is a literal transformer algorithm, but there is some evolved learning algorithm in the neurons of the brain that is at least somewhat a distant cousin of what we’re doing today.
- jjk166 2y ago
  Tautologically. Machine learning is a blanket term for a wide range of approaches that allow machines to mimic the brain's capability to learn, at least in function if not form. The brain may employ wildly different machine learning algorithms from those currently in vogue, but whatever algorithms the brain is using must be machine learning algorithms. Unless of course you define machine in this case to be something which isn't the brain, in which case they're just learning algorithms. Regardless, an architecture exists to match the brain's capabilities with not only finite but surprisingly limited hardware.

remon 2y ago

If we lower the bar on papers getting published even further it'd be on the floor. Even with a simple physics argument you can immediately disprove the claim. Human brains have a known amount of compute (as in, we are at most an order of magnitude or two off) within a specific volume. We are already capable of that amount of compute, we just don't have a way to use that compete for AGI yet (and THAT may be hard or impossible but it's certainly not energy resource constrained). We also know the abilities this paper claims are not possible to achieve can be achieved by pretty much every human being within years of being born. How they managed to massage white supremacy and sexism into this is...well...wow.

lambdaone 2y ago

I'm surprised that the authors seemingly can't see that their argument hoists them by their own petard by showing, using exactly the same reasoning, that human intelligence is equally impossible to create (unless, perhaps, you are willing to use vitalist/supernatural arguments to claim is is different in kind from computational AI). Surely the referees must have raised this at the review stage?

nullfield 2y ago

It’s absolutely inevitable. Whether one will be alive for it is another matter…