22 ms·

I'm surprised by the number of "is AGI even possible" comments here and would love to hear more. I personally think AGI is far off, but always assumed it was a

by jtuple 5y ago

I'm surprised by the number of "is AGI even possible" comments here and would love to hear more. I personally think AGI is far off, but always assumed it was an inevitability. Obviously, humans are sentient with GI, and various other animals range from close-ish to humans to not-even-close but still orders of magnitude more than any machine. Ie. GI is a real thing, in the real world. It's not time travel, immortality, etc. I certainly understand the religious perspective. If you believe humans/life is special, created by some higher power, and that the created cannot create, then AGI is not possible. But, given the number of "is AGI possible?" comments I assume not all are religious based (HN doesn't seem to a highly religious cohort to me). What are the common secular arguments against AGI? Are people simply doubting the more narrow view that AGI is possible via ML implemented on existing computing technology? Or the idea in general? While the article does focus on current ML trajectory and "digital" solutions, its core position is mostly focused on a "new approach" and AI creating AGI. I'd consider a scenario where highly advanced but non-sentient ML algorithms figure out to devise a new technology (be that digital or analog, inorganic or organic) that leads to AGI as an outcome that is consistent with this article. Is that viable in 20 years time? No idea, but given an infinite timescale it certainly seems more possible than not to me given that we all already exist as blackbox MVPs that just haven't been reverse engineered yet.

pfdietz 5y ago
An argument against AGI might be that brains are extremely efficient for what they do. Maybe we could make a computer that's as powerful as a brain, but if it consumes 100 MW of power what's the point?
- deleted 5y ago
  [deleted]
- jtuple 5y ago
  There are many industrial processes that use tons of power and are far less efficient than a human doing those tasks. Yet, they're still viable because they (scale / are faster / more consistent / etc than humans. For AGI, it's really about replication, density, and easy-of-operation. At the moment, we certainly can't mass produce "brains-on-a-chip" that provide a guaranteed level of human-like intelligence across various tasks. But, imagine a world in which you could install racks of "brains-on-a-chip", powered via electricity (easily distributed/stored/fungible compared to food-powered-brains), and have a Moore's Law like scaling of "brain density". That would change everything, even if those brains consumed 1000W a pop. Obviously, a literal brain is probably not the way this will pan out (hopefully not! "brains-on-a-chip" is rather creepy...), but you get the idea.
  - andrekandre 5y ago
    if something is generally intelligent, at the level of a human brain, and is forced to work in "industrial processes" isn't that a form of subjugation? there seems to be a moral implication here that alot of people seem to be neglecting...
jasfi 5y ago
Language is the missing key, in my opinion. I'm working on solving natural language understanding: https://lxagi.com https://lxagi.com. Just a landing page for now with an email sign-up.
- windows2020 5y ago
  Perhaps language is the foundation for symbol manipulation and complex thoughts. Are feelings a language?
  - jasfi 5y ago
    Yes, I believe language is the foundation for higher-level reasoning. Feelings are very primal, but they can be expressed through language.
mr_toad 5y ago
People often confuse not being able to understand how something is possible with it being impossible. Despite all the progress being made by using a little bit of design, and a whole lot of brute force, people keep saying it can’t work, and they keep being proven wrong. Ironically I think this means AI won’t be creating AI in an exponentially increasing way - because AI is more about scaling and emergence of dumb elements than it is about grand designs.
- TacticalCoder 5y ago
  > ... people keep saying it can’t work, and they keep being proven wrong. It's impossible to prove wrong all those saying: "AI is not AGI" until you eventually end up with an actual AGI. Nobody is proven wrong.
mabub24 5y ago
> What are the common secular arguments against AGI? There is an entire sector of Philosophy of Mind that is a convincing argument against AGI. Neuroscience is also pretty skeptical of it. Part of it comes down to what you mean by AGI. Is it a computer that is convincing as AGI? Or is it AGI that is essentially like human consciousness in nature? The former is probably possible, given enough time, computational resources, and ingenuity. The latter is generally regarded as pretty nonsensical. In general, I think you're implying the gap between the AI we have now, and animals, and humans, is way smaller than it really is. The gap between computer AI and even some intelligent animals is enormous, let alone humans. And many would not even say computers are intelligent in a human sense. Computers don't think, or imagine in any intelligible sense. They compute. That's it. So the question that really should be asked is whether computation alone can lead to something that is recognizably an AGI in the human sense? I would say no, because that requires abilities that computers simply do not and cannot have. But it might achieve something that is convincing as AGI, something like Wolfram or Siri but much more convincing. Part of it comes down to the fact that the term AI for ML is generally just marketing speak. It's a computational model of a kind of intelligence that is computational in nature, with all the limits that entails. Part of it also comes down to people who love computers thinking computers will ultimately be able to do anything and everything. That feels cool, but it doesn't mean it's possible. edit: There is also Erik J Larson's book "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do" from 2021 which is an interesting argument against AI -> AGI. He has a pretty good grasp on CS and Philosophy.
- mkaic 5y ago
  >Is it a computer that is convincing as AGI? Or is it AGI that is essentially like human consciousness in nature? The former is probably possible, given enough time, computational resources, and ingenuity. The latter is generally regarded as pretty nonsensical. Author here. I think you're drawing an arbitrary distinction between "acts conscious" and "is conscious", even though in practice there is no way to distinguish between them and thus they are functionally equivalent. I cannot prove you are not a product of a simulation I am living in, that is to say, your consciousness is nonfalsifiable to me. All I can do is look at how you turn your inputs into outputs. If a robot can do that, too (what you call "convincing as AGI") then we must assume it is also conscious, because if we don't, we'd have a logical inconsistency on our hands. If I am allowed to safely assume you are sentient, then I must also be allowed to safely assume a robot is sentient if it can convince me, because in both cases I have no method of falsifying the claim to sentience. Thank you for your comment! I appreciate you taking the time to share your thoughts.
dcolkitt 5y ago
What about if we live inside a simulation? AGI may be possible in base reality, but not within the confines of our simulated universe. Maybe intelligence requires a very large amount of compute. (Maybe even more than is physically possible inside our brains, but the simulation fudges it.) Like you can run a VM in a VM if you have special purpose accelerators. But otherwise a VM^2 is theoretically possible but infeasible given the constraints of compute.
- jbay808 5y ago
  > Maybe even more than is physically possible inside our brains, but the simulation fudges it. Are you basing your argument on the assumption that human brains violate the known laws of physics?
  - dcolkitt 5y ago
    If we're in a simulation, the computational resources of an object aren't constrained by the physical scale of the object. It's possible that achieving human scale cognition requires brains the size of Jupiter (we don't know because we don't have a full computational accounting of human level cognition). But in a simulation that could be routed around by having a physical computer the size of Jupiter operate a human scale body in the simulation. You could share and cache compute resources across 6 billion humans within a single human compute core the size of Jupiter (or maybe ~100 Jupiters). Since most computation is probably redundant, you could probably simulate many orders of magnitude more humans with a single human compute core. To observers within the simulation it would look like human cognition is achievable in 3 lbs of tissue. But this would simply be an illusion created by the simulation environment. There'd be no way of knowing for sure until we fully reverse engineer a single human brain down to the level of base compute operations.
    - ambrose2 5y ago
      I am totally fascinated by this idea, well done!
    - virchau13 5y ago
colinmhayes 5y ago
AGI is only inevitable if you subscribe to materialism. Basically it comes down to whether we are purely material beings or whether there are non-material aspects to our mind. There are more than a couple non religious arguments for dualism, many are relatively recent, as dualism is enjoying a bit of a comeback in philosophical circles. The Chinese room and the red room a.k.a.Mary's room are simple enough thought experiments that can help you grok why dualism might be true. That being said, the most recent philpeople survey has 52% accepting materialism compared to 32% accepting dualism with the rest undecided, so academic philosophy does generally lean toward AGI. https://survey2020.philpeople.org/survey/results/all https://survey2020.philpeople.org/survey/results/all
- foobarian 5y ago
  There is some gray area in between where maybe AGI is dependent on some (material) mechanism that we haven't discovered yet, and hence can't be accomplished using present computational setups.
  - colinmhayes 5y ago
    Absolutely, the "inevitable" timeframe may be millenia for all anyone knows, it just means that we will eventually achieve it given enough time and resources.
- simonh 5y ago
  The Chinese room argument is a parlour trick that uses scale as a distraction. It posits a person in a room manipulating symbols to produce intelligent seeming outputs. It says, see, it’s absurd to think a person in a room with a stack of symbols could emulate intelligence. But let’s say the room contains many billions of people, it is the size of a planet, and it contains racks of many trillions of symbols, and it spends millions or billions of years to produce an output. That’s more like the scale of a sophisticated computer system, or a brain. Does that sound much like a man in a room with some symbols? No. Does it sound like that could do complex calculations and produce sophisticated and perhaps even intelligent outputs? Well, given enough time and scale, yes why not? The Chinese room is pure misdirection and it amazes me anyone falls for it. There’s really no actual argument there.
gcheong 5y ago
It's probably possible, but is it optimal? And what do we even mean by optimal? What's the ultimate purpose of AGI? Perhaps it would be optimal in the sense that it could teach itself to be an expert in anything in short order, but that's not to say that it would be a better approach than something else depending on the goal. We're clearly biased because we are one current solution to a problem that billions of years of trial and error has attempted to solve, and we think we're a pretty good one, but it also blinds us to the possibility of better ones.
slibhb 5y ago
> I personally think AGI is far off, but always assumed it was an inevitability. Why? We might get taken out by a meteor tomorrow. I agree that, in principle, AGI is possible. But there's lots of things that are in principle possible that will never happen. We have general intelligence but it took billions of years of evolution for us to get here. In some sense those billions of years of evolution are still with us, in every cell in our bodies. Maybe all of those years of learning are necessary for general intelligence. Trying to find a shortcut may yield impressive results in narrow circumstances but I doubt it will be generally intelligent in the way people and animals are.
- eru 5y ago
  Many animals are not generally intelligent. Also many human subsystems are easy to trick. Look at eg optical illusions. And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers?
  - slibhb 5y ago
    > And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers? Information is passed through the generations by DNA. Silicon doesn't have DNA.
    - eru 5y ago
      DNA is one way to pass information. It's far from the only way.
jollybean 5y ago
I can't even understand why people think AGI is even remotely practically possible, let alone theoretically. Even the most monstrous, amazing, 3000-CPU AI's are unimpressive and can barely do narrow functions. I remember when computers started to beat humans at chess - a long time ago. It was neat, but not a big deal. Now we can beat humans at 'Go'? Seriously? It's only 'impressive' to computer scientists. Imagine that we just stopped calling things 'AI' and literally picked some other, more mundane term, which I suggest should be 'Adaptive Algorithms'. If we called it 'AA' - I'm doubtful we'd even be having this conversation about 'intelligence'.
- mkaic 5y ago
  I mean, human intelligence is just one big "adaptive algorithm" too, it's just ours took billions of years to train and used natural selection as a loss function. I don't see any fundamental reason why we couldn't do the same with a machine.
  - windows2020 5y ago
    During the construction of the machine, would there come a point where it gains a sense of self like yours? Or, would it gradually happen?
- greggman3 5y ago
  Easy for me to imagine. We simulate every atom of a brain. If the simulation is accurate it should think. The only thing that would make that not possible is believing in some kind of spirit. After that it's just optimizations. Are their algorithms to simulate atoms faster. Is every aspect of those atoms important to the simulation. Can you instead simulate molecules instead of atoms. Or larger structures. Etc... I'm not saying that is how we will arrive at AGI but it is arguably a logical path that should lead there of something else doesn't come first.
  - ux-app 5y ago
    what if there're those pesky quantum thingamajiggies which make simulating in silicon impossible?
yojo 5y ago
I think AGI is usually equated with “superhuman” intelligence, though obviously not a requirement. If superhuman intelligence were easy, why aren’t we already more intelligent? Being smart has already demonstrated considerable fitness benefits for the human race. It seems like we haven’t gotten much smarter in the last few thousand years though, at least from a raw horsepower point of view. My intuition is that past a certain level of complexity intelligent systems become inherently unstable. This is all just a handwave, but there’s some circumstantial evidence in the confluence of genius and mental disorder in our species. If that were true, the question then becomes: is biology an inferior substrate for general intelligence compared to silicon. Obviously a cpu can add two integers more efficiently than a meat brain, but it’s not self evident that this will hold true for more complex computation. Put another way, if you try to make an AGI agent “smarter” than a person, you might just end up with an irrational system that spits out nonsense. So it may be that for complexity reasons you can’t beat human-level GI. At which point the question becomes whether AGI is cheaper to produce and operate than feeding and sheltering a human. If the answer is “no”, then AGI may be both possible and irrelevant.
- greggman3 5y ago
  It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) as well as having faster access to more knowledge in general (because you can make a bigger brain with more memory) it's just pretty obvious that AGI will beat humans. If all they had was human brains but could transfer thoughts faster they'd still beat us.
  - yojo 5y ago
    Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than the one we enjoy. I’m not saying you’re wrong, only that I can imagine a reality where you can’t just arbitrarily scale components of cognition and still have a functioning system.
    - gonehome 5y ago
spicyusername 5y ago
I think AGI is possible in theory, but may not be possible in practice. I also think that "intelligence" is a vague concept, which makes defining what AGI even is tricky. Reaching it assumes that the ceiling for our technology is sufficiently low and that the runway for us to continue making progress is sufficiently long. Neither of those assumptions may end up being valid.
mellosouls 5y ago
It's not - in my experience reading HN - that AGI-sceptics (like me) don't think it will happen, just that we have developed a healthy contempt for the hype-merchants and their wide-eyed followers who despite years of "machine learning this" and "deep learning that" and "gpt-3 the other" have nothing to show for it in terms of something resembling human-level intelligence, or even any sign of understanding how we think. It's perfectly reasonable to see that and still be confident AGI will happen - just not with current models.
- colechristensen 5y ago
  I think basically it requires machines that work in fundamentally different ways that can't be reduced to statistics, and that we'll get better at approximating intelligence for a while until we hit a wall where no amount of scale makes any big difference.
- ketzo 5y ago
  That is a reasonable skepticism, I think. But I will say: by the time anyone has “something to show for it,” AGI will, pretty much by definition, already be here. I don’t think we’re gonna be able to see it coming, or know what it will look like, except through very speculative predictions. So I do think while skepticism is warranted, we should still evaluate everything in good faith, and not automatically jump to shilling. (Not saying you’re doing that, btw - just commenting)
  - coderenegade 5y ago
    I think the flight analogy for AI is a good one; we have machines that let us fly, but we don't fly the way birds do. In many ways, the ways that we fly far exceed the capabilities of birds, but there are ways in which birds exceed the capability of our machines. The issue with AGI as it's often framed is that it implicitly assumes a general intelligence like us, without any metric that can relate human intelligence and machine intelligence. We have machines that beat us in Go, so if we took those machines and somehow combined them with machines that beat us in Chess, Starcraft and Dota, are we any closer to AGI than we were previously? What about if that same AI can also drive a car? What's the standard, and where does it end? Where do we fall on that same metric? There are also some fairly deep philosophical questions in this realm, too. For example, to what extent is the human experience (and intelligence) linked to our physical bodies? Cooking by smell and writing by sound are things we do on a daily basis that have no machine analogue. How important is the embodiment principle to AI? Certainly we don't do things by plugging our minds in directly -- we drive cars using the same limbs that we use for everything else, which probably has enormous efficiency advantages. To what extent can you separate the intelligence from the body and the environment? Frankly, I believe we're a lot further away than we think. The last decade has taught us that relatively simple methods can be applied in surprisingly powerful ways, which is an important start, but it doesn't tell us anything about how close we actually are to a given goal, and how we might go about reaching it.
totetsu 5y ago
Just a week ago we were on the brink of nuclear war again. How is it at all an inevitability that humanity will keep pushing technology on until the point of AGI?
hacknat 5y ago
I believe AGI is possible, but the more I study ML the more convinced I am that we don’t know what it (AGI) is. I also, don’t know if I agree that humans have GI (humanity as a whole does). I think we’re actually copycats more than anything else. Neural nets are the atoms of GI, but it’s not clear what the superstructure of GI is. We all assume that we understand what general intelligence is, but I don’t think we actually have the goods yet. We understand optimization, but we don’t understand the thing that chooses what to optimize. There is some notion of “values” in order for intelligence to be legible, and we don’t know what that is either.
manquer 5y ago
Take your pick - We won't achieve AGI because the goalposts will keep moving every 20 years. Everyone would have a stronger/higher definition of GI as we achieve each milestone. - Possible doesn't mean it is feasible in finite time. Not all problems are feasible to compute. It could be akin to a NP-Hard problem for example, there is no mathematical certainty that convergence is possible and also in a reasonable time (within the life of the universe). - It will be possible to build a machine to pass the Turing test, one that can very closely mimic human behaviour, each generation they would come closer and closer and no human can distinguish between the two, True AGI is well beyond that, we maybe able to create something like us, we may not be able to create something better than us.
umvi 5y ago
> I certainly understand the religious perspective. If you believe humans/life is special, created by some higher power, and that the created cannot create, then AGI is not possible. Well the created clearly can create. I mean, obviously humans (and indeed all life) can reproduce and create new living things... it's just a completely automatic process with a chicken and egg problem. (i.e. to create life you need starting ingredients that are already alive by some metric). The question though from the religious perspective is whether or not living things have "spirits" or "souls" that causes them to be alive (as opposed to a bunch of matter with the same chemical makeup that is not alive). Because if they do, then creating artificial life would require somehow binding a "spirit" or "soul" into your artificial life vessel which is something we don't know how to do.
ecopoesis 5y ago
The atomic bomb is a good contrast. In the early 30s we knew a bomb was possible (Szilard‘s patent was in 1933) but there were huge engineering problems to overcome (enriching uranium). But we knew that even if we couldn’t scale up isotope separation and we had to go there long way around, we could still make a bomb with enough time. There is no Szilard patent for AGI: no one has any theory on how to make it work other then “make it bigger,” which as this article points out hasn’t paid off like we’d hoped. I’d have a lot more faith that AGI is possible if we had any kind of theory or roadmap on how to get there. Counting on a black box inventing it for us seems like waiting for a million monkeys to finish King Lear 2.
- FeepingCreature 5y ago
  The techniques that will allow AGI are probably already invented. How many people at the time knew what that patent meant for the future of the world?
- marvin 5y ago
  > There is no Szilard patent for AGI You don’t know that. But even worse, you probably won’t know if it happens. History is not as predictable in the present as it is in hindsight. Most physicists in 1935 would still have said that an atomic bomb was impossible, and defended it with the same indignated vigor demonstrated in this thread.
  - tacocataco 5y ago
    Some theorized the A-bomb could start a chain reaction igniting the atmosphere. https://www.insidescience.org/manhattan-project-legacy/atmosphere-on-fire https://www.insidescience.org/manhattan-project-legacy/atmos... I guess I'm glad they were wrong but what a hell of a risk to take just to get another bomb when we already had so many. Perhaps the pressure of other nations developing the weapon was just to great.
causality0 5y ago
AGI as a concept is almost certainly possible. However, it may not be that useful. That is to say, barring a true revolution in computational physics, it may not be possible to run human-equivalent AGI on a computer that's less than several orders of magnitude more massive than a human brain. Biology is unspeakably space-efficient. The brain of a dragonfly uses just sixteen neurons to take input from thousands of ommatidia, track prey in 3D space, and plot an intercept vector. Figure out a minimum artificial computer performance to do the same thing, then scale your requirement by the ratio of human brain to dragonfly brain. The numbers are not encouraging.
- AlphaCenturion 5y ago
  Sorry, the dragonfly bit is just wildly inaccurate. You're either being intentionally misleading or regurgitating something you heard once and never bothered to confirm yourself. There are millions of neurons in a dragonfly's brain.
  - belugacat 5y ago
    https://www.pnas.org/doi/full/10.1073/pnas.1210489109 https://www.pnas.org/doi/full/10.1073/pnas.1210489109 This is what they’re referencing
  - causality0 5y ago
    Yes, and it uses sixteen if them for the specific task I mentioned. Pay attention.
cjohnson318 5y ago
What if we actually generate some form of intelligence, but can't figure out how to motivate it to solve our problems? I mean, we already have intelligent life, but a very small percentage of intelligent biomass can solve important problems, or wants to. Like, great, we created a real-life intelligent tamagotchi, but literally all it wants to do is play go.
prometheus76 5y ago
Part of it comes down to trying to do AI in a disembodied way. OK, so a team of engineers can specially train and tune an AI model to beat a human at Go. But the human player who was beaten by the computer is thinking about getting up to go to the bathroom while playing, and will drive home afterwards through rush hour traffic and make dinner. That is informed by that human player's GI model for the world. Current AI models see the world as edges and vague boxes. It doesn't even have item persistence, let alone the ability to decide on, and make, dinner.
deleted 5y ago
[deleted]
School-Cotton 5y ago
> always assumed it was an inevitability. Why? We don’t know how to build it, even in theory, so shouldn’t the default be to say there’s no way of knowing it’s possible or not? After all, some things from sci-fi have become reality (computers, the Internet) or haven’t but are obviously feasible (human travel to Mars) and some haven’t and seem unlikely ever to (faster-than-light travel, flying cars). With no evidence in either direction, why assume that AGI is in the second category, rather than the third? There’s no natural law saying that everything that can be imagined is possible. > GI is a real thing. Yes, but GI isn’t AGI. AGI would require us to either use completely different materials with completely different efficiency characteristics and somehow achieve the same results, or learn to precisely manipulate organic materials (proteins, etc.). It’s not obvious to me why either of these is possible (or impossible — I’m not taking a stance either way). Furthermore, even if it is possible to build AGI, there’s no a priori reason to believe that humans are smart enough to discover how.
- RealityVoid 5y ago
  > With no evidence in either direction, why assume that AGI is in the second category, rather than the third? Because we have GI examples in nature, therefore it is possible. We just don't know how to do it, yet. The same way we saw birds flying, so we knew flight was possible, but we could not, at that time.
throwawaylinux 5y ago
> If you believe humans/life is special, created by some higher power, and that the created cannot create, then AGI is not possible. How does that follow? > What are the common secular arguments against AGI? Having infinite time and resources, and pointing to the existence of intelligent humans is a fairly weak argument IMO. It's not really saying anything much that can be refuted. Sure there exists some physical process from which intelligence arises. That doesn't mean we'll be able to create the same thing. Can we create a heart? Or a hand? Even a patch of skin is cut from one part of a person's body and grafted elsewhere. All things far simpler than the brain. And the best hope we'll probably ever have of creating them is to grow biological ones rather than actually design and build our own. Let's go one step further - we are incapable of creating anything even approaching the functionality of the simplest single-celled life-form. The "it exists therefore we must be able to create it" argument doesn't have a lot of legs. And there is no real "trajectory" to general intelligence that I can see. Velocity is sitting around a flat zero. Recognizing pictures of cats on the internet is about where we are at the moment. Even self-driving cars or doing a half-decent job at predicting what people might like to buy are not a step along the road to intelligence, they are an entirely different roads altogether. We don't even know what that road looks like, don't know where it is, and don't know where we are in relation to it. The only vaguely plausible lead people have to go on is to try to do what biological brains do, but seeing as we don't even know how they work it's a bit like the blind leading the blind. I think it's quite possible, certainly over an infinite time scale. Not an inevitability though, definitely not inevitable (or even very likely) within our lifetimes.
idontwantthis 5y ago
I will continue dismissing AGI until someone can tell me what GI is. I don't believe that it's just statistical inferences run in parallel (Machine Learning). If we don't know what GI is, then I don't know how we are supposed to replicate it. It's like if we wanted to produce power on Earth the same way as the Sun does, but without knowing that was fusion. So we were just trying to make things look like the Sun. Someone says "Look, I made a ball so hot you can't look at it, we must be close!" I believe we will have fusion eventually because it is a process that we deeply understand, it's just hard to do. I don't believe we are on a path to AGI right now because know one understands even a little bit of GI.
- hutzlibu 5y ago
  " I don't believe that it's just statistical inferences run in parallel" Why not? I also believe, there must be something more, but I cannot articulate it. But as disturbing as it sounds, most natural learning might be just this. Lots of statistics run in parallel.
  - idontwantthis 5y ago
    That’s just my belief. I might get proven wrong, but I haven’t been so far.
  - idontwantthis 5y ago
    And what about things people do that don’t make any statistical sense? Is it AGI if it picks up its underwear the first time you ask, or is it only AGI once it waits for you to get angry after the fifth time you ask?
    - hutzlibu 5y ago
      "And what about things people do that don’t make any statistical sense?" Like what? It is always individual statistics made up from partly faulty data. It is never objective, what mattered in a evolutionary sense, was that it works good enough. (like ants algorithms are not perfect, but get the job done most of the time. I watched ants a lot ...)
hooande 5y ago
I get the reasoning of "general intelligence exists in humans, therefore it must be possible to create it". But this logic really applies to human artifacts and creations. ie, if a human builds a device that can do X then another human can also build a device that does X. There is no magic. No one is special. Anything one person can create or do, no matter how rare, can be done by another person at some point. General intelligence was not created by a human, or by any other intelligent being or process. It's one result of billions of years of random chance in the form of biological evolution. It could be that there are basic and comprehensible principles driving that process. Or it could be the result of a tweleve digit number of random coin flips. There's no way to know. When I think of this as, "Will humans ever be able to replicate or mimic the result of planet wide randomness at an unimaginable time scale?" my answer is "No, probably not". Our minds can't conceive the scope of the process that created our minds. There are many things that we are capable of groking as a species (green tech, space travel, medicine) that seem much more important.
- throwaway09223 5y ago
  But we don't have to comprehend the scope. We just have to implement a genetic algorithm with the appropriate inputs and resources. Have you ever written one? It's not terribly difficult. There's really nothing special here. I can't conceive the scope of the majority of software projects I've worked on. So what?
  - deleted 5y ago
    [deleted]
georgeecollins 5y ago
For the decade of the 1980's (40 years ago) Japan made the top emphasis of their industrial policy AI. In the 1700's people believed we could make devices that imitated the behavior of animals as soon as we could make the clock work small enough. Meanwhile the whole world could be massacred by nuclear weapons on any given day in a few hours. And the author feels we are woefully unprepared for the singularity. Maybe.
andreareina 5y ago
I also hold the opinion that intelligence is computable, but it's not hard to imagine the complexity required is maybe more than we can achieve in the short, maybe medium term.
hyperpallium2 5y ago
I think AGI will turn out to be pretty simple (but still could take a long time). And it has arisen many times in other species, but didn't take off because not tied to survival value. If you put GI in charge of an animal, who knows whether what it will do will be in the animal's best interest? The human trick was brain architecture that somehow controls intelligence, while still being "in charge". The general nature of intelligence makes it difficult to control. Another way GI is tied to survival value in humans is general division of labour (and general trade), which other species lack.
jhbadger 5y ago
I think the moment AGI is possible, we won't have it because it would basically be slavery to own something really comparable to a human.
Alan_Dillman 5y ago
Diogenes holds up a plucked chicken and proclaims "Behold, a man!" The hypesters are defining AGI (and even General Intelligence) like Plato defined man, "A featherless biped". Either they know this, and they are hucksters, or their thinking is that shallow, and thus have nothing meaningful to say on the subject. General Intelligence has drivers, like curiosity, hope, a sense of fairness and injustice, and hunger, pain, pleasure, fear. And those things probably drove the refinement of intelligence, and as such, a housecat is less like a simple machine than a earthworm is. I think AGI is possible. Ultimately we're all machines, and I don't believe in a free will particle, but it is fairly obvious that an earthworm merely acts, while a cat acts interestingly, pulling in qualia unavailable to the worm. A worm is either active, or its dead, while my cat is clever enough to cut corners, to seek efficiency or pleasure, to be lazy. And humans are even better at it. Therein lies the rub: people working on AI don't seem to aiming for real intelligence, they want their convenient action machine, more like a worm than a cat. And the hypesters (and doomsters) hand wave and say "suddenly, it will become self aware and magically motivated." Suuuure. We're far more likely to be thoughtlessly killed off before a machine muses, "I think, therefore I am". Just because it "am", does not mean it thinks. To be intelligent is to be able to consider "why?" and "how?", to be able to fool one's self, even in the face of cold reality, to yell "no fair!", "I don't wanna", and to wonder how one might take short cuts or avoid tasks at all. Which is how and why we're chasing AI, but nobody is working out how to make it cry. The cart will be build, and the horse will suddenly appear, all haltered up? The rain will follow the plough? No. That's magical thinking. It took about a billion years of massive parallel organic computation for evolution to synthesize general intelligence, and we're to believe that some malfunctioning quine machine is going to break out, 75 years from the starting gate? Not by accident, and certainly not be design.
- rawoke083600 5y ago
  >General Intelligence has drivers, like curiosity, hope, a sense of fairness and injustice, and hunger, pain, pleasure, fear I love this thinking ! I've never thought about the "drivers" part of AGI, only ever thought about AGI as a "general-toolbox" to "solve-many-problems" One thing, I think did aid in development of our GI, was the fact we had a physical-body. Thus we had to "adapt and optimise" our GI with that constraint. I.e limited energy(sure computing-power could be a good proxy), but things like pain, survive-ability (poisons, being hunted) the first would help that we develop good visual pattern recognition to identity the poisonous snake, the latter(being hunted) that we had to evolve strategies that were in the realms of our bodies capabilities. Like running away, setting traps. BUT your body is not a static-container, your risk tolerance, capabilities, sensor systems changes as you ages. We have been building AI, with mostly a "non-corporeal" (i.e digital living space) in mind. At first we(ok me) thought it was an advantage since we could iterate faster and simulate many solutions, but now I think it might be a distinct disadvantage) to NOT include the "limitations" of a physical-container like a body-proxy (energy, vulnerability, limited-capabilities etc). If synergy is defined(and valued) as: The whole is greater than the sum of the parts, then maybe: The sum of our limitations(body) is of greater value to the overall-fitness of the organism to evolve intelligence ? YMMV :)
narag 5y ago
I agree with mr_toad's sibling comment 100%, but also: But, given the number of "is AGI possible?" comments I assume not all are religious based (HN doesn't seem to a highly religious cohort to me). There are some religious ideas that survive just fine after you pull religion from under people's feet. Dualism is one of them. Look how many atheists say that freedom is an illusion because of determinism, an idea based on dualism.
Dave3of5 5y ago
I studied AI at university about 20ish years ago. In terms of what we have today it's was already there and had been around already for decades. All that's happened are minor refinements over the years. Most of the refinements have been brought about by the insane pace of h/w development not by anything really breakthrough when it comes to AI. In terms of AGI there was nothing even close back then so I suspect we will never see anything in our lifetimes. In reality for us that's all that matters. I know some want to talk about what will happen in say 200+ years but that's complete speculation and wild guess territory there are so many other things that could happen in 200+ years that think about AGI and these sort of breakthrough technologies isn't helpful. FYI this saddens me as I really thought that by now we'd have some kind of AGI and when I went to uni I was hopeful and excited at first.
marvin 5y ago
It’s just the same kind of conventional thinking we’ve seen a million times through history. Most people, even the smart ones, just aren’t good at imagining anything that hasn’t already happened. So they’ll say it’s impossible and keep doing whatever more immediate practical concern they’re working on. I don’t think it needs to be analyzed further than that; you’re trying to see reason where there isn’t any. Most people holding the views you’re talking about are at are at best just confused about the subject being discussed, but more likely just wrong.
xbmcuser 5y ago
I don't think singularity can happen until we get true quantum computing.
TheOtherHobbes 5y ago
Humans very much don't have AGI. It's one of our many flaws. We define AGI as "human-like behaviour" but that usually means "AI researcher-like behaviour", which is why AI research concentrates on tasks like playing games and translating language, and not on having enough general intelligence to handle social situations. So AGI seems to be defined as "Can self-tutor any academic or intellectual domain to a professional level." The core human skillset is elsewhere. It's based on social awareness, emotional mirroring and empathy (except for Dark Triad types) and various kinds of ambition, goal-seeking, and drive regulation. And language semantics to drive all of this. All based on various kinds of contextual awareness of physical, emotional, and social location. And only once all of this is in place does intellectual learning happen. It's actually quite a narrow. but deep set of skills. It's still broader than - say - playing bridge. But unlike bridge, winning optimal moves are very hard to define, and internal state and external actions may have long-delayed results, so training is very difficult. And there are multiple definitions of winning - from personal happiness to financial and political domination - some of which conflict with each other. Which is why humans have 15-20 years of training on these problems before entering the adult human world. In CS AI terms humans are already dumb as rocks. No human can play bridge, chess, or go to an AI standard. In human terms many humans are also almost as dumb as rocks. But even stupid humans can still handle the core skillset to a passable level. AI hasn't really made any inroads into this space, but I strongly suspect true AGI won't be possible without it. Otherwise you build a machine that can teach itself to write music and play chess and pick stocks, but it learns with no context or concept of usefulness. "Understanding" music would be exactly equal to "understanding" stamp collecting. It would know the computational cost of everything, but the value of nothing.
philwelch 5y ago
I don’t think AGI is necessarily impossible, but I’m not convinced that it’s possible to achieve in a way that gets around the constraints of human intelligence. The singularity idea is basically the assumption that AGI will scale the same way computers have scaled over the past several decades, but if AGI turns out to require special hardware and years of training the same way we do, it’s not obvious that it’s going to be anything more remarkable than we are.
turing_complete 5y ago
I don't even think that AGI is incompatible with Christian belief. In the words of Saint Paul: "For now we see through a glass, darkly; but then face to face: now I know in part; but then shall I know even as also I am known."
bryanrasmussen 5y ago
Aside from religious arguments AGI might be philosophically impossible given that we do not seem to have a proven, accepted definition of GI. If you cannot actually define it how can you know if you have achieved it? So then we ask will we be able to define it? And here is where the real problem comes in, because if we are to define GI and as you say 'humans are sentient with GI' it is arguable we need to be able to fully understand ourselves. and this is problematic because can the constituent parts of a system understand itself from within the system. From within the system you cannot see from outside and thus understand it separate from the inner workings. But another issue also pertains, we can understand a simple system because we are more complicated than it. Just as we can understand how 2 or 1 dimensions work because we exist in 3 dimensions. Thus we can model how a 1 dimensional being intersecting with a 2 dimensional object would perceive the experience. So the question also is can a system of complexity X fully understand systems of complexity X, or do you need a more complicated system. This is of course the reason for the Turing test, to come up with a way to disregard these problems and agree that a machine is good enough. So it may be that some people think AGI is impossible for philsophical reason, but that Turing test passing machines are possible. Not to say that I hold to any particular belief regarding these kinds of arguments.
- zasdffaa 5y ago
  > AGI might be philosophically impossible given that we do not seem to have a proven, accepted definition of GI Strange argument. Just cos we can't define it doesn't preclude its existence. Once (pre modern chemistry) no-one knew what water actually was, but water unquestionably existed.
  - bryanrasmussen 5y ago
    water exists, and you can see it, GI exists by definition and you cannot see it, there are millions of arguments as to what GI is and you cannot point to some GI and have everyone agree yes, that is GI but we just differ on what it is made of, even in ancient times you could point to water and say that is water and everyone who argued about the origins of water would still agree with you, yes that is water. So how do you presume to make something when no one agrees what that thing is?
imdhmd 5y ago
My conjecture is that any new form of GI will continue to evolve from an existing branch of sentient organisms. Artificial influence to evolution could be made possible by augmenting a current GI with AI, but i'm unable to comprehend a GI built entirely from scratch. My argument for this line of reasoning is that the secret sauce of motivation or purpose that is deep down in every "dna" has neither been discovered nor replicable. This is not a religious but a philosophical assertion.
katabasis 5y ago
I'm not sure that human "general intelligence" is a thing in the first place. Human behavior and cognition is driven by biological and cultural processes, which are the result of millions of years of evolution and billions of arbitrarily complicated proteins all interacting on one another. Is a computer going to emulate every part of that? Despite what we think about our minds being rational, autonomous, operating according to a set of coherent principles, etc., I'm not sure that's really the case at the end of the day.
411111111111111 5y ago
You don't need AGI to reach the official definition of the singularity as I understand it. The only requirement is too have a machine design and build a machine with no human Input which is able to design and build another machine which is able to do said task *better/more efficient" then the previous iteration did
deleted 5y ago
[deleted]

by jtuple 5y ago

pfdietz 5y ago

An argument against AGI might be that brains are extremely efficient for what they do. Maybe we could make a computer that's as powerful as a brain, but if it consumes 100 MW of power what's the point?

deleted 5y ago
[deleted]
jtuple 5y ago
There are many industrial processes that use tons of power and are far less efficient than a human doing those tasks. Yet, they're still viable because they (scale / are faster / more consistent / etc than humans. For AGI, it's really about replication, density, and easy-of-operation. At the moment, we certainly can't mass produce "brains-on-a-chip" that provide a guaranteed level of human-like intelligence across various tasks. But, imagine a world in which you could install racks of "brains-on-a-chip", powered via electricity (easily distributed/stored/fungible compared to food-powered-brains), and have a Moore's Law like scaling of "brain density". That would change everything, even if those brains consumed 1000W a pop. Obviously, a literal brain is probably not the way this will pan out (hopefully not! "brains-on-a-chip" is rather creepy...), but you get the idea.
- andrekandre 5y ago
  if something is generally intelligent, at the level of a human brain, and is forced to work in "industrial processes" isn't that a form of subjugation? there seems to be a moral implication here that alot of people seem to be neglecting...

jasfi 5y ago

Language is the missing key, in my opinion. I'm working on solving natural language understanding: https://lxagi.com https://lxagi.com. Just a landing page for now with an email sign-up.

windows2020 5y ago
Perhaps language is the foundation for symbol manipulation and complex thoughts. Are feelings a language?
- jasfi 5y ago
  Yes, I believe language is the foundation for higher-level reasoning. Feelings are very primal, but they can be expressed through language.

mr_toad 5y ago

People often confuse not being able to understand how something is possible with it being impossible. Despite all the progress being made by using a little bit of design, and a whole lot of brute force, people keep saying it can’t work, and they keep being proven wrong. Ironically I think this means AI won’t be creating AI in an exponentially increasing way - because AI is more about scaling and emergence of dumb elements than it is about grand designs.

TacticalCoder 5y ago
> ... people keep saying it can’t work, and they keep being proven wrong. It's impossible to prove wrong all those saying: "AI is not AGI" until you eventually end up with an actual AGI. Nobody is proven wrong.

mabub24 5y ago

> What are the common secular arguments against AGI? There is an entire sector of Philosophy of Mind that is a convincing argument against AGI. Neuroscience is also pretty skeptical of it. Part of it comes down to what you mean by AGI. Is it a computer that is convincing as AGI? Or is it AGI that is essentially like human consciousness in nature? The former is probably possible, given enough time, computational resources, and ingenuity. The latter is generally regarded as pretty nonsensical. In general, I think you're implying the gap between the AI we have now, and animals, and humans, is way smaller than it really is. The gap between computer AI and even some intelligent animals is enormous, let alone humans. And many would not even say computers are intelligent in a human sense. Computers don't think, or imagine in any intelligible sense. They compute. That's it. So the question that really should be asked is whether computation alone can lead to something that is recognizably an AGI in the human sense? I would say no, because that requires abilities that computers simply do not and cannot have. But it might achieve something that is convincing as AGI, something like Wolfram or Siri but much more convincing. Part of it comes down to the fact that the term AI for ML is generally just marketing speak. It's a computational model of a kind of intelligence that is computational in nature, with all the limits that entails. Part of it also comes down to people who love computers thinking computers will ultimately be able to do anything and everything. That feels cool, but it doesn't mean it's possible. edit: There is also Erik J Larson's book "The Myth of Artificial Intelligence: Why Computers Can’t Think the Way We Do" from 2021 which is an interesting argument against AI -> AGI. He has a pretty good grasp on CS and Philosophy.

mkaic 5y ago
>Is it a computer that is convincing as AGI? Or is it AGI that is essentially like human consciousness in nature? The former is probably possible, given enough time, computational resources, and ingenuity. The latter is generally regarded as pretty nonsensical. Author here. I think you're drawing an arbitrary distinction between "acts conscious" and "is conscious", even though in practice there is no way to distinguish between them and thus they are functionally equivalent. I cannot prove you are not a product of a simulation I am living in, that is to say, your consciousness is nonfalsifiable to me. All I can do is look at how you turn your inputs into outputs. If a robot can do that, too (what you call "convincing as AGI") then we must assume it is also conscious, because if we don't, we'd have a logical inconsistency on our hands. If I am allowed to safely assume you are sentient, then I must also be allowed to safely assume a robot is sentient if it can convince me, because in both cases I have no method of falsifying the claim to sentience. Thank you for your comment! I appreciate you taking the time to share your thoughts.

dcolkitt 5y ago

What about if we live inside a simulation? AGI may be possible in base reality, but not within the confines of our simulated universe. Maybe intelligence requires a very large amount of compute. (Maybe even more than is physically possible inside our brains, but the simulation fudges it.) Like you can run a VM in a VM if you have special purpose accelerators. But otherwise a VM^2 is theoretically possible but infeasible given the constraints of compute.

jbay808 5y ago
> Maybe even more than is physically possible inside our brains, but the simulation fudges it. Are you basing your argument on the assumption that human brains violate the known laws of physics?
- dcolkitt 5y ago
  If we're in a simulation, the computational resources of an object aren't constrained by the physical scale of the object. It's possible that achieving human scale cognition requires brains the size of Jupiter (we don't know because we don't have a full computational accounting of human level cognition). But in a simulation that could be routed around by having a physical computer the size of Jupiter operate a human scale body in the simulation. You could share and cache compute resources across 6 billion humans within a single human compute core the size of Jupiter (or maybe ~100 Jupiters). Since most computation is probably redundant, you could probably simulate many orders of magnitude more humans with a single human compute core. To observers within the simulation it would look like human cognition is achievable in 3 lbs of tissue. But this would simply be an illusion created by the simulation environment. There'd be no way of knowing for sure until we fully reverse engineer a single human brain down to the level of base compute operations.
  - ambrose2 5y ago
    I am totally fascinated by this idea, well done!
  - virchau13 5y ago

colinmhayes 5y ago

AGI is only inevitable if you subscribe to materialism. Basically it comes down to whether we are purely material beings or whether there are non-material aspects to our mind. There are more than a couple non religious arguments for dualism, many are relatively recent, as dualism is enjoying a bit of a comeback in philosophical circles. The Chinese room and the red room a.k.a.Mary's room are simple enough thought experiments that can help you grok why dualism might be true. That being said, the most recent philpeople survey has 52% accepting materialism compared to 32% accepting dualism with the rest undecided, so academic philosophy does generally lean toward AGI. https://survey2020.philpeople.org/survey/results/all https://survey2020.philpeople.org/survey/results/all

foobarian 5y ago
There is some gray area in between where maybe AGI is dependent on some (material) mechanism that we haven't discovered yet, and hence can't be accomplished using present computational setups.
- colinmhayes 5y ago
  Absolutely, the "inevitable" timeframe may be millenia for all anyone knows, it just means that we will eventually achieve it given enough time and resources.
simonh 5y ago
The Chinese room argument is a parlour trick that uses scale as a distraction. It posits a person in a room manipulating symbols to produce intelligent seeming outputs. It says, see, it’s absurd to think a person in a room with a stack of symbols could emulate intelligence. But let’s say the room contains many billions of people, it is the size of a planet, and it contains racks of many trillions of symbols, and it spends millions or billions of years to produce an output. That’s more like the scale of a sophisticated computer system, or a brain. Does that sound much like a man in a room with some symbols? No. Does it sound like that could do complex calculations and produce sophisticated and perhaps even intelligent outputs? Well, given enough time and scale, yes why not? The Chinese room is pure misdirection and it amazes me anyone falls for it. There’s really no actual argument there.

gcheong 5y ago

It's probably possible, but is it optimal? And what do we even mean by optimal? What's the ultimate purpose of AGI? Perhaps it would be optimal in the sense that it could teach itself to be an expert in anything in short order, but that's not to say that it would be a better approach than something else depending on the goal. We're clearly biased because we are one current solution to a problem that billions of years of trial and error has attempted to solve, and we think we're a pretty good one, but it also blinds us to the possibility of better ones.

slibhb 5y ago

> I personally think AGI is far off, but always assumed it was an inevitability. Why? We might get taken out by a meteor tomorrow. I agree that, in principle, AGI is possible. But there's lots of things that are in principle possible that will never happen. We have general intelligence but it took billions of years of evolution for us to get here. In some sense those billions of years of evolution are still with us, in every cell in our bodies. Maybe all of those years of learning are necessary for general intelligence. Trying to find a shortcut may yield impressive results in narrow circumstances but I doubt it will be generally intelligent in the way people and animals are.

eru 5y ago
Many animals are not generally intelligent. Also many human subsystems are easy to trick. Look at eg optical illusions. And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers?
- slibhb 5y ago
  > And, to follow your philosophical argument: given that the only thing that connects our cells to the cells of our ancestors is information, why couldn't those billions of years be also in our computers? Information is passed through the generations by DNA. Silicon doesn't have DNA.
  - eru 5y ago
    DNA is one way to pass information. It's far from the only way.

jollybean 5y ago

I can't even understand why people think AGI is even remotely practically possible, let alone theoretically. Even the most monstrous, amazing, 3000-CPU AI's are unimpressive and can barely do narrow functions. I remember when computers started to beat humans at chess - a long time ago. It was neat, but not a big deal. Now we can beat humans at 'Go'? Seriously? It's only 'impressive' to computer scientists. Imagine that we just stopped calling things 'AI' and literally picked some other, more mundane term, which I suggest should be 'Adaptive Algorithms'. If we called it 'AA' - I'm doubtful we'd even be having this conversation about 'intelligence'.

mkaic 5y ago
I mean, human intelligence is just one big "adaptive algorithm" too, it's just ours took billions of years to train and used natural selection as a loss function. I don't see any fundamental reason why we couldn't do the same with a machine.
- windows2020 5y ago
  During the construction of the machine, would there come a point where it gains a sense of self like yours? Or, would it gradually happen?
greggman3 5y ago
Easy for me to imagine. We simulate every atom of a brain. If the simulation is accurate it should think. The only thing that would make that not possible is believing in some kind of spirit. After that it's just optimizations. Are their algorithms to simulate atoms faster. Is every aspect of those atoms important to the simulation. Can you instead simulate molecules instead of atoms. Or larger structures. Etc... I'm not saying that is how we will arrive at AGI but it is arguably a logical path that should lead there of something else doesn't come first.
- ux-app 5y ago
  what if there're those pesky quantum thingamajiggies which make simulating in silicon impossible?

yojo 5y ago

I think AGI is usually equated with “superhuman” intelligence, though obviously not a requirement. If superhuman intelligence were easy, why aren’t we already more intelligent? Being smart has already demonstrated considerable fitness benefits for the human race. It seems like we haven’t gotten much smarter in the last few thousand years though, at least from a raw horsepower point of view. My intuition is that past a certain level of complexity intelligent systems become inherently unstable. This is all just a handwave, but there’s some circumstantial evidence in the confluence of genius and mental disorder in our species. If that were true, the question then becomes: is biology an inferior substrate for general intelligence compared to silicon. Obviously a cpu can add two integers more efficiently than a meat brain, but it’s not self evident that this will hold true for more complex computation. Put another way, if you try to make an AGI agent “smarter” than a person, you might just end up with an irrational system that spits out nonsense. So it may be that for complexity reasons you can’t beat human-level GI. At which point the question becomes whether AGI is cheaper to produce and operate than feeding and sheltering a human. If the answer is “no”, then AGI may be both possible and irrelevant.

greggman3 5y ago
It's pretty hard to believe that being able transfer knowledge between digital brains (something that seems arguably inevitable) as well as having faster access to more knowledge in general (because you can make a bigger brain with more memory) it's just pretty obvious that AGI will beat humans. If all they had was human brains but could transfer thoughts faster they'd still beat us.
- yojo 5y ago
  Again, I will hand-wave about complexity. A larger memory and the ability to load/unload large corpora rapidly implies a very different kind of cognition than the one we enjoy. I’m not saying you’re wrong, only that I can imagine a reality where you can’t just arbitrarily scale components of cognition and still have a functioning system.
  - gonehome 5y ago

spicyusername 5y ago

I think AGI is possible in theory, but may not be possible in practice. I also think that "intelligence" is a vague concept, which makes defining what AGI even is tricky. Reaching it assumes that the ceiling for our technology is sufficiently low and that the runway for us to continue making progress is sufficiently long. Neither of those assumptions may end up being valid.

mellosouls 5y ago

It's not - in my experience reading HN - that AGI-sceptics (like me) don't think it will happen, just that we have developed a healthy contempt for the hype-merchants and their wide-eyed followers who despite years of "machine learning this" and "deep learning that" and "gpt-3 the other" have nothing to show for it in terms of something resembling human-level intelligence, or even any sign of understanding how we think. It's perfectly reasonable to see that and still be confident AGI will happen - just not with current models.

colechristensen 5y ago
I think basically it requires machines that work in fundamentally different ways that can't be reduced to statistics, and that we'll get better at approximating intelligence for a while until we hit a wall where no amount of scale makes any big difference.
ketzo 5y ago
That is a reasonable skepticism, I think. But I will say: by the time anyone has “something to show for it,” AGI will, pretty much by definition, already be here. I don’t think we’re gonna be able to see it coming, or know what it will look like, except through very speculative predictions. So I do think while skepticism is warranted, we should still evaluate everything in good faith, and not automatically jump to shilling. (Not saying you’re doing that, btw - just commenting)
- coderenegade 5y ago
  I think the flight analogy for AI is a good one; we have machines that let us fly, but we don't fly the way birds do. In many ways, the ways that we fly far exceed the capabilities of birds, but there are ways in which birds exceed the capability of our machines. The issue with AGI as it's often framed is that it implicitly assumes a general intelligence like us, without any metric that can relate human intelligence and machine intelligence. We have machines that beat us in Go, so if we took those machines and somehow combined them with machines that beat us in Chess, Starcraft and Dota, are we any closer to AGI than we were previously? What about if that same AI can also drive a car? What's the standard, and where does it end? Where do we fall on that same metric? There are also some fairly deep philosophical questions in this realm, too. For example, to what extent is the human experience (and intelligence) linked to our physical bodies? Cooking by smell and writing by sound are things we do on a daily basis that have no machine analogue. How important is the embodiment principle to AI? Certainly we don't do things by plugging our minds in directly -- we drive cars using the same limbs that we use for everything else, which probably has enormous efficiency advantages. To what extent can you separate the intelligence from the body and the environment? Frankly, I believe we're a lot further away than we think. The last decade has taught us that relatively simple methods can be applied in surprisingly powerful ways, which is an important start, but it doesn't tell us anything about how close we actually are to a given goal, and how we might go about reaching it.

totetsu 5y ago

Just a week ago we were on the brink of nuclear war again. How is it at all an inevitability that humanity will keep pushing technology on until the point of AGI?

hacknat 5y ago

I believe AGI is possible, but the more I study ML the more convinced I am that we don’t know what it (AGI) is. I also, don’t know if I agree that humans have GI (humanity as a whole does). I think we’re actually copycats more than anything else. Neural nets are the atoms of GI, but it’s not clear what the superstructure of GI is. We all assume that we understand what general intelligence is, but I don’t think we actually have the goods yet. We understand optimization, but we don’t understand the thing that chooses what to optimize. There is some notion of “values” in order for intelligence to be legible, and we don’t know what that is either.

manquer 5y ago

Take your pick - We won't achieve AGI because the goalposts will keep moving every 20 years. Everyone would have a stronger/higher definition of GI as we achieve each milestone. - Possible doesn't mean it is feasible in finite time. Not all problems are feasible to compute. It could be akin to a NP-Hard problem for example, there is no mathematical certainty that convergence is possible and also in a reasonable time (within the life of the universe). - It will be possible to build a machine to pass the Turing test, one that can very closely mimic human behaviour, each generation they would come closer and closer and no human can distinguish between the two, True AGI is well beyond that, we maybe able to create something like us, we may not be able to create something better than us.

umvi 5y ago

> I certainly understand the religious perspective. If you believe humans/life is special, created by some higher power, and that the created cannot create, then AGI is not possible. Well the created clearly can create. I mean, obviously humans (and indeed all life) can reproduce and create new living things... it's just a completely automatic process with a chicken and egg problem. (i.e. to create life you need starting ingredients that are already alive by some metric). The question though from the religious perspective is whether or not living things have "spirits" or "souls" that causes them to be alive (as opposed to a bunch of matter with the same chemical makeup that is not alive). Because if they do, then creating artificial life would require somehow binding a "spirit" or "soul" into your artificial life vessel which is something we don't know how to do.

ecopoesis 5y ago

The atomic bomb is a good contrast. In the early 30s we knew a bomb was possible (Szilard‘s patent was in 1933) but there were huge engineering problems to overcome (enriching uranium). But we knew that even if we couldn’t scale up isotope separation and we had to go there long way around, we could still make a bomb with enough time. There is no Szilard patent for AGI: no one has any theory on how to make it work other then “make it bigger,” which as this article points out hasn’t paid off like we’d hoped. I’d have a lot more faith that AGI is possible if we had any kind of theory or roadmap on how to get there. Counting on a black box inventing it for us seems like waiting for a million monkeys to finish King Lear 2.

FeepingCreature 5y ago
The techniques that will allow AGI are probably already invented. How many people at the time knew what that patent meant for the future of the world?
marvin 5y ago
> There is no Szilard patent for AGI You don’t know that. But even worse, you probably won’t know if it happens. History is not as predictable in the present as it is in hindsight. Most physicists in 1935 would still have said that an atomic bomb was impossible, and defended it with the same indignated vigor demonstrated in this thread.
- tacocataco 5y ago
  Some theorized the A-bomb could start a chain reaction igniting the atmosphere. https://www.insidescience.org/manhattan-project-legacy/atmosphere-on-fire https://www.insidescience.org/manhattan-project-legacy/atmos... I guess I'm glad they were wrong but what a hell of a risk to take just to get another bomb when we already had so many. Perhaps the pressure of other nations developing the weapon was just to great.

causality0 5y ago

AGI as a concept is almost certainly possible. However, it may not be that useful. That is to say, barring a true revolution in computational physics, it may not be possible to run human-equivalent AGI on a computer that's less than several orders of magnitude more massive than a human brain. Biology is unspeakably space-efficient. The brain of a dragonfly uses just sixteen neurons to take input from thousands of ommatidia, track prey in 3D space, and plot an intercept vector. Figure out a minimum artificial computer performance to do the same thing, then scale your requirement by the ratio of human brain to dragonfly brain. The numbers are not encouraging.

AlphaCenturion 5y ago
Sorry, the dragonfly bit is just wildly inaccurate. You're either being intentionally misleading or regurgitating something you heard once and never bothered to confirm yourself. There are millions of neurons in a dragonfly's brain.
- belugacat 5y ago
  https://www.pnas.org/doi/full/10.1073/pnas.1210489109 https://www.pnas.org/doi/full/10.1073/pnas.1210489109 This is what they’re referencing
- causality0 5y ago
  Yes, and it uses sixteen if them for the specific task I mentioned. Pay attention.

cjohnson318 5y ago

What if we actually generate some form of intelligence, but can't figure out how to motivate it to solve our problems? I mean, we already have intelligent life, but a very small percentage of intelligent biomass can solve important problems, or wants to. Like, great, we created a real-life intelligent tamagotchi, but literally all it wants to do is play go.

prometheus76 5y ago

Part of it comes down to trying to do AI in a disembodied way. OK, so a team of engineers can specially train and tune an AI model to beat a human at Go. But the human player who was beaten by the computer is thinking about getting up to go to the bathroom while playing, and will drive home afterwards through rush hour traffic and make dinner. That is informed by that human player's GI model for the world. Current AI models see the world as edges and vague boxes. It doesn't even have item persistence, let alone the ability to decide on, and make, dinner.

deleted 5y ago

[deleted]

School-Cotton 5y ago

> always assumed it was an inevitability. Why? We don’t know how to build it, even in theory, so shouldn’t the default be to say there’s no way of knowing it’s possible or not? After all, some things from sci-fi have become reality (computers, the Internet) or haven’t but are obviously feasible (human travel to Mars) and some haven’t and seem unlikely ever to (faster-than-light travel, flying cars). With no evidence in either direction, why assume that AGI is in the second category, rather than the third? There’s no natural law saying that everything that can be imagined is possible. > GI is a real thing. Yes, but GI isn’t AGI. AGI would require us to either use completely different materials with completely different efficiency characteristics and somehow achieve the same results, or learn to precisely manipulate organic materials (proteins, etc.). It’s not obvious to me why either of these is possible (or impossible — I’m not taking a stance either way). Furthermore, even if it is possible to build AGI, there’s no a priori reason to believe that humans are smart enough to discover how.

RealityVoid 5y ago
> With no evidence in either direction, why assume that AGI is in the second category, rather than the third? Because we have GI examples in nature, therefore it is possible. We just don't know how to do it, yet. The same way we saw birds flying, so we knew flight was possible, but we could not, at that time.

throwawaylinux 5y ago

> If you believe humans/life is special, created by some higher power, and that the created cannot create, then AGI is not possible. How does that follow? > What are the common secular arguments against AGI? Having infinite time and resources, and pointing to the existence of intelligent humans is a fairly weak argument IMO. It's not really saying anything much that can be refuted. Sure there exists some physical process from which intelligence arises. That doesn't mean we'll be able to create the same thing. Can we create a heart? Or a hand? Even a patch of skin is cut from one part of a person's body and grafted elsewhere. All things far simpler than the brain. And the best hope we'll probably ever have of creating them is to grow biological ones rather than actually design and build our own. Let's go one step further - we are incapable of creating anything even approaching the functionality of the simplest single-celled life-form. The "it exists therefore we must be able to create it" argument doesn't have a lot of legs. And there is no real "trajectory" to general intelligence that I can see. Velocity is sitting around a flat zero. Recognizing pictures of cats on the internet is about where we are at the moment. Even self-driving cars or doing a half-decent job at predicting what people might like to buy are not a step along the road to intelligence, they are an entirely different roads altogether. We don't even know what that road looks like, don't know where it is, and don't know where we are in relation to it. The only vaguely plausible lead people have to go on is to try to do what biological brains do, but seeing as we don't even know how they work it's a bit like the blind leading the blind. I think it's quite possible, certainly over an infinite time scale. Not an inevitability though, definitely not inevitable (or even very likely) within our lifetimes.

idontwantthis 5y ago

I will continue dismissing AGI until someone can tell me what GI is. I don't believe that it's just statistical inferences run in parallel (Machine Learning). If we don't know what GI is, then I don't know how we are supposed to replicate it. It's like if we wanted to produce power on Earth the same way as the Sun does, but without knowing that was fusion. So we were just trying to make things look like the Sun. Someone says "Look, I made a ball so hot you can't look at it, we must be close!" I believe we will have fusion eventually because it is a process that we deeply understand, it's just hard to do. I don't believe we are on a path to AGI right now because know one understands even a little bit of GI.

hutzlibu 5y ago
" I don't believe that it's just statistical inferences run in parallel" Why not? I also believe, there must be something more, but I cannot articulate it. But as disturbing as it sounds, most natural learning might be just this. Lots of statistics run in parallel.
- idontwantthis 5y ago
  That’s just my belief. I might get proven wrong, but I haven’t been so far.
- idontwantthis 5y ago
  And what about things people do that don’t make any statistical sense? Is it AGI if it picks up its underwear the first time you ask, or is it only AGI once it waits for you to get angry after the fifth time you ask?
  - hutzlibu 5y ago
    "And what about things people do that don’t make any statistical sense?" Like what? It is always individual statistics made up from partly faulty data. It is never objective, what mattered in a evolutionary sense, was that it works good enough. (like ants algorithms are not perfect, but get the job done most of the time. I watched ants a lot ...)

hooande 5y ago

I get the reasoning of "general intelligence exists in humans, therefore it must be possible to create it". But this logic really applies to human artifacts and creations. ie, if a human builds a device that can do X then another human can also build a device that does X. There is no magic. No one is special. Anything one person can create or do, no matter how rare, can be done by another person at some point. General intelligence was not created by a human, or by any other intelligent being or process. It's one result of billions of years of random chance in the form of biological evolution. It could be that there are basic and comprehensible principles driving that process. Or it could be the result of a tweleve digit number of random coin flips. There's no way to know. When I think of this as, "Will humans ever be able to replicate or mimic the result of planet wide randomness at an unimaginable time scale?" my answer is "No, probably not". Our minds can't conceive the scope of the process that created our minds. There are many things that we are capable of groking as a species (green tech, space travel, medicine) that seem much more important.

throwaway09223 5y ago
But we don't have to comprehend the scope. We just have to implement a genetic algorithm with the appropriate inputs and resources. Have you ever written one? It's not terribly difficult. There's really nothing special here. I can't conceive the scope of the majority of software projects I've worked on. So what?
- deleted 5y ago
  [deleted]

georgeecollins 5y ago

For the decade of the 1980's (40 years ago) Japan made the top emphasis of their industrial policy AI. In the 1700's people believed we could make devices that imitated the behavior of animals as soon as we could make the clock work small enough. Meanwhile the whole world could be massacred by nuclear weapons on any given day in a few hours. And the author feels we are woefully unprepared for the singularity. Maybe.

andreareina 5y ago

I also hold the opinion that intelligence is computable, but it's not hard to imagine the complexity required is maybe more than we can achieve in the short, maybe medium term.

hyperpallium2 5y ago

I think AGI will turn out to be pretty simple (but still could take a long time). And it has arisen many times in other species, but didn't take off because not tied to survival value. If you put GI in charge of an animal, who knows whether what it will do will be in the animal's best interest? The human trick was brain architecture that somehow controls intelligence, while still being "in charge". The general nature of intelligence makes it difficult to control. Another way GI is tied to survival value in humans is general division of labour (and general trade), which other species lack.

jhbadger 5y ago

I think the moment AGI is possible, we won't have it because it would basically be slavery to own something really comparable to a human.

Alan_Dillman 5y ago

Diogenes holds up a plucked chicken and proclaims "Behold, a man!" The hypesters are defining AGI (and even General Intelligence) like Plato defined man, "A featherless biped". Either they know this, and they are hucksters, or their thinking is that shallow, and thus have nothing meaningful to say on the subject. General Intelligence has drivers, like curiosity, hope, a sense of fairness and injustice, and hunger, pain, pleasure, fear. And those things probably drove the refinement of intelligence, and as such, a housecat is less like a simple machine than a earthworm is. I think AGI is possible. Ultimately we're all machines, and I don't believe in a free will particle, but it is fairly obvious that an earthworm merely acts, while a cat acts interestingly, pulling in qualia unavailable to the worm. A worm is either active, or its dead, while my cat is clever enough to cut corners, to seek efficiency or pleasure, to be lazy. And humans are even better at it. Therein lies the rub: people working on AI don't seem to aiming for real intelligence, they want their convenient action machine, more like a worm than a cat. And the hypesters (and doomsters) hand wave and say "suddenly, it will become self aware and magically motivated." Suuuure. We're far more likely to be thoughtlessly killed off before a machine muses, "I think, therefore I am". Just because it "am", does not mean it thinks. To be intelligent is to be able to consider "why?" and "how?", to be able to fool one's self, even in the face of cold reality, to yell "no fair!", "I don't wanna", and to wonder how one might take short cuts or avoid tasks at all. Which is how and why we're chasing AI, but nobody is working out how to make it cry. The cart will be build, and the horse will suddenly appear, all haltered up? The rain will follow the plough? No. That's magical thinking. It took about a billion years of massive parallel organic computation for evolution to synthesize general intelligence, and we're to believe that some malfunctioning quine machine is going to break out, 75 years from the starting gate? Not by accident, and certainly not be design.

rawoke083600 5y ago
>General Intelligence has drivers, like curiosity, hope, a sense of fairness and injustice, and hunger, pain, pleasure, fear I love this thinking ! I've never thought about the "drivers" part of AGI, only ever thought about AGI as a "general-toolbox" to "solve-many-problems" One thing, I think did aid in development of our GI, was the fact we had a physical-body. Thus we had to "adapt and optimise" our GI with that constraint. I.e limited energy(sure computing-power could be a good proxy), but things like pain, survive-ability (poisons, being hunted) the first would help that we develop good visual pattern recognition to identity the poisonous snake, the latter(being hunted) that we had to evolve strategies that were in the realms of our bodies capabilities. Like running away, setting traps. BUT your body is not a static-container, your risk tolerance, capabilities, sensor systems changes as you ages. We have been building AI, with mostly a "non-corporeal" (i.e digital living space) in mind. At first we(ok me) thought it was an advantage since we could iterate faster and simulate many solutions, but now I think it might be a distinct disadvantage) to NOT include the "limitations" of a physical-container like a body-proxy (energy, vulnerability, limited-capabilities etc). If synergy is defined(and valued) as: The whole is greater than the sum of the parts, then maybe: The sum of our limitations(body) is of greater value to the overall-fitness of the organism to evolve intelligence ? YMMV :)

narag 5y ago

I agree with mr_toad's sibling comment 100%, but also: But, given the number of "is AGI possible?" comments I assume not all are religious based (HN doesn't seem to a highly religious cohort to me). There are some religious ideas that survive just fine after you pull religion from under people's feet. Dualism is one of them. Look how many atheists say that freedom is an illusion because of determinism, an idea based on dualism.

Dave3of5 5y ago

I studied AI at university about 20ish years ago. In terms of what we have today it's was already there and had been around already for decades. All that's happened are minor refinements over the years. Most of the refinements have been brought about by the insane pace of h/w development not by anything really breakthrough when it comes to AI. In terms of AGI there was nothing even close back then so I suspect we will never see anything in our lifetimes. In reality for us that's all that matters. I know some want to talk about what will happen in say 200+ years but that's complete speculation and wild guess territory there are so many other things that could happen in 200+ years that think about AGI and these sort of breakthrough technologies isn't helpful. FYI this saddens me as I really thought that by now we'd have some kind of AGI and when I went to uni I was hopeful and excited at first.

marvin 5y ago

It’s just the same kind of conventional thinking we’ve seen a million times through history. Most people, even the smart ones, just aren’t good at imagining anything that hasn’t already happened. So they’ll say it’s impossible and keep doing whatever more immediate practical concern they’re working on. I don’t think it needs to be analyzed further than that; you’re trying to see reason where there isn’t any. Most people holding the views you’re talking about are at are at best just confused about the subject being discussed, but more likely just wrong.

xbmcuser 5y ago

I don't think singularity can happen until we get true quantum computing.

TheOtherHobbes 5y ago

Humans very much don't have AGI. It's one of our many flaws. We define AGI as "human-like behaviour" but that usually means "AI researcher-like behaviour", which is why AI research concentrates on tasks like playing games and translating language, and not on having enough general intelligence to handle social situations. So AGI seems to be defined as "Can self-tutor any academic or intellectual domain to a professional level." The core human skillset is elsewhere. It's based on social awareness, emotional mirroring and empathy (except for Dark Triad types) and various kinds of ambition, goal-seeking, and drive regulation. And language semantics to drive all of this. All based on various kinds of contextual awareness of physical, emotional, and social location. And only once all of this is in place does intellectual learning happen. It's actually quite a narrow. but deep set of skills. It's still broader than - say - playing bridge. But unlike bridge, winning optimal moves are very hard to define, and internal state and external actions may have long-delayed results, so training is very difficult. And there are multiple definitions of winning - from personal happiness to financial and political domination - some of which conflict with each other. Which is why humans have 15-20 years of training on these problems before entering the adult human world. In CS AI terms humans are already dumb as rocks. No human can play bridge, chess, or go to an AI standard. In human terms many humans are also almost as dumb as rocks. But even stupid humans can still handle the core skillset to a passable level. AI hasn't really made any inroads into this space, but I strongly suspect true AGI won't be possible without it. Otherwise you build a machine that can teach itself to write music and play chess and pick stocks, but it learns with no context or concept of usefulness. "Understanding" music would be exactly equal to "understanding" stamp collecting. It would know the computational cost of everything, but the value of nothing.

philwelch 5y ago

I don’t think AGI is necessarily impossible, but I’m not convinced that it’s possible to achieve in a way that gets around the constraints of human intelligence. The singularity idea is basically the assumption that AGI will scale the same way computers have scaled over the past several decades, but if AGI turns out to require special hardware and years of training the same way we do, it’s not obvious that it’s going to be anything more remarkable than we are.

turing_complete 5y ago

I don't even think that AGI is incompatible with Christian belief. In the words of Saint Paul: "For now we see through a glass, darkly; but then face to face: now I know in part; but then shall I know even as also I am known."

bryanrasmussen 5y ago

Aside from religious arguments AGI might be philosophically impossible given that we do not seem to have a proven, accepted definition of GI. If you cannot actually define it how can you know if you have achieved it? So then we ask will we be able to define it? And here is where the real problem comes in, because if we are to define GI and as you say 'humans are sentient with GI' it is arguable we need to be able to fully understand ourselves. and this is problematic because can the constituent parts of a system understand itself from within the system. From within the system you cannot see from outside and thus understand it separate from the inner workings. But another issue also pertains, we can understand a simple system because we are more complicated than it. Just as we can understand how 2 or 1 dimensions work because we exist in 3 dimensions. Thus we can model how a 1 dimensional being intersecting with a 2 dimensional object would perceive the experience. So the question also is can a system of complexity X fully understand systems of complexity X, or do you need a more complicated system. This is of course the reason for the Turing test, to come up with a way to disregard these problems and agree that a machine is good enough. So it may be that some people think AGI is impossible for philsophical reason, but that Turing test passing machines are possible. Not to say that I hold to any particular belief regarding these kinds of arguments.

zasdffaa 5y ago
> AGI might be philosophically impossible given that we do not seem to have a proven, accepted definition of GI Strange argument. Just cos we can't define it doesn't preclude its existence. Once (pre modern chemistry) no-one knew what water actually was, but water unquestionably existed.
- bryanrasmussen 5y ago
  water exists, and you can see it, GI exists by definition and you cannot see it, there are millions of arguments as to what GI is and you cannot point to some GI and have everyone agree yes, that is GI but we just differ on what it is made of, even in ancient times you could point to water and say that is water and everyone who argued about the origins of water would still agree with you, yes that is water. So how do you presume to make something when no one agrees what that thing is?

imdhmd 5y ago

My conjecture is that any new form of GI will continue to evolve from an existing branch of sentient organisms. Artificial influence to evolution could be made possible by augmenting a current GI with AI, but i'm unable to comprehend a GI built entirely from scratch. My argument for this line of reasoning is that the secret sauce of motivation or purpose that is deep down in every "dna" has neither been discovered nor replicable. This is not a religious but a philosophical assertion.

katabasis 5y ago

I'm not sure that human "general intelligence" is a thing in the first place. Human behavior and cognition is driven by biological and cultural processes, which are the result of millions of years of evolution and billions of arbitrarily complicated proteins all interacting on one another. Is a computer going to emulate every part of that? Despite what we think about our minds being rational, autonomous, operating according to a set of coherent principles, etc., I'm not sure that's really the case at the end of the day.

411111111111111 5y ago

You don't need AGI to reach the official definition of the singularity as I understand it. The only requirement is too have a machine design and build a machine with no human Input which is able to design and build another machine which is able to do said task *better/more efficient" then the previous iteration did

deleted 5y ago

[deleted]