16 ms·

What Is AGI-Hard

131 points by freediver 4y ago

throwaway_5753 4y ago
I like the concept of AGI-hard and the characterization of the common traps of AI productization feels accurate. One shortcoming of the analogy is that we have methods to prove when a problem is NP-hard. Are there ways to prove a problem is AGI-hard? Can it even be rigorously characterized? Relying on someone asserting it on Twitter feels unsatisfying (e.g. how accurate would experts have been at predicting the current capabilities of AI if you asked them 10 years ago? I think not very).
- wilg 4y ago
  Exactly – "perfect visual reasoning is AGI-hard", the tweet that seems to have inspired this, suffers from several problems: • "perfect visual reasoning" is not a thing, because "visual reasoning" isn't clearly defined. Nor "reasoning". Most importantly, neither is "perfect". • It's not clear whether it's even accurate to say you need "perfect visual reasoning" for the application at hand (driving) Determining whether something is AGI-hard is AGI-hard.
- trashtester 4y ago
  > how accurate would experts have been at predicting the current capabilities of AI if you asked them 10 years ago? Maybe less inaccurate than you may think. Especially if you include "experts" that focus specifically at estimating future developments, and ignore "experts" that have focused all their effort into some very specific algorithmic detail. It seems to me that a key factor in longer-term estimations is simply compute performance. If development of AI seems to be lagging somewhat compared to some predictions, it seems to me that this lag is similar to the slowdown in Moore's law. Also, for robotics, wearables and vehicles, power consumption of electronics combined with the fact that batteries as still heavy and expensive is an important limiting factor. Now the METHODS we use to reach predictions of people like Kurzweil may be different from what we imagined. Specifically, it seems that many futurists were thinking that AGI would be reached through Turing Machines and rationalist algorithms. Instead it turns out that most progress is made through building generic learning architectures (like Transformers) and apply them at huge scale with vast amounts of data. Similarily, a generation ago, we may have imagined we could have a single-core processor performing 1petaflop of computation in the 2020's. Instead, we have the 4090 now, that can do that many computations, but they are limited to "tensor" operations. Now, assuming we're not making some drastic breakthrough that enables a "rationalist" type of algorithm to form AGI, we should still reach human brain-level of raw compute power sometime between 2030-2050 (which fits reasonably well with many predictions), and I would be surprised if we don't have full AGI within such a timeframe. I would give about 1:1 odds that it happens before 2040. Still, ideas based on future computers being generic Turing Machines, just faster mean some predictions become fundamentally hard. For instance, Uploading our brains to a computer becomes a lot harder of the computer is not designed simply as a faster Turing Machine, but rather has hardware that is very specialized for some running some simple low level computation massively parallel. It may be close to impossible to run a human brain inside a GPU (at least efficiently), regardless of how many CUDA cores it has. That means that "uploading" may require us to make a near-exact replica of an actual human brain, with it's spiking style neurons, in silicon. Also, related predictions by people like Kurzweil may seem far fetched, such as his preditions about nanotech. On the other hand, we HAVE actually started large scale treatment of a large percentage of the world's population using nanotech medicine (mRNA vaccines). Maybe the revolution he predicted is actually about to happen. But my expectation is that it's going to be relatively slow for a few more decades. It may become radically better if techniques like AlphaFold develop at Moore's law pace in terms of price an capability, but I suspect generic nanotech is a harder problem than AGI simply from a computational perspective.
jasfi 4y ago
As always, what can't be done is shrinking.
palad1n 4y ago
Wikipedia: https://en.wikipedia.org/wiki/AI-complete https://en.wikipedia.org/wiki/AI-complete (AI hard redirected to this)
janalsncm 4y ago
This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Physics? Calculus? Programming other AI agents? Why not add chess to the list? The fact that we already have an algorithm that enables computers to teach themselves to play chess is not an a priori reason to exclude it.
- Animats 4y ago
  > This article is a bit too hand-wavy for my taste. From the article: "I think this is the sort of debate that makes good podcast fodder". So the article author admits that. It's hand-wavy on vision. Parts of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") What's inherently hard is always a good question. Aristotle thought that the human ability to do arithmetic was a key indicator of intelligence. Now we know just how few gates it takes to do arithmetic. A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI. As I've pointed out before, we're still nowhere on "common sense", defined as not screwing up in the next 30 seconds. This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.
  - klipt 4y ago
    > huge volumes of plausible blithering that's factually wrong That describes large parts of the internet, and we trained these models on that, so what did we really expect? ;-)
hakuseki 4y ago
I'm not sold on the concept of AI-hard or AI-complete problems. For example, the Wikipedia article on AI-completeness mentions Bongard problems and Autonomous driving as examples of problems that might be AI-complete. OK, so if I have an AI drives autonomously, is there some known querying strategy that I can use to make it solve Bongard problems? Can a Bongard problem-solving AI be made, by some known procedure to drive a car? Without such reductions, at least the analogy to NP-hardness is incomplete. I believe these reductions are precisely what makes NP-hardness such a useful concept; even though we still haven't proven that any of these problems are objectively "hard," we are still able to show that if one of them is hard, then the others are as well!
sanxiyn 4y ago
> Can AI invent calculus from first principles? The thing is, most people can't. In fact, if you reinvented calculus yourselves, you should be very proud of your mathematical ability.
- xlii 4y ago
  I believe AI is plural here and relates to general existing population. It’s not like asking if Stable Diffusion or ChatGPT can do it. Maybe there is specialized commercial AI that aren’t so widely known. There exist right now people with low levels of physical and mental capabilities that current state of AI knowledge surpasses it by miles. We don’t go around saying “AI is better than humans”, even though - technically - it can be in some selected cases.
- Closi 4y ago
  Ha, yes. AGI-Hard also needs to be contrasted against Human-Hard, which is done in most of the article, but equally we probably get to a place of saying "Hard things are Hard". Another example is that the article says "It would be hard for an AI to compose this image as it has lots of different elements/emotions" - but see the Chat-GPT example below: > Me: Can you generate a prompt for an AI to generate an interesting image? > GPT3: Sure! Here is a prompt for an AI to generate an interesting image: "Create a surreal scene featuring a giant, humanoid robot standing on a desolate, alien planet, with a small group of humans huddled fearfully at its feet. The robot should have a sleek, futuristic design, and its glowing red eyes should convey a sense of intelligence and power. In the background, there should be a faint hint of a cityscape, suggesting that the robot has traveled far from its home world. Use bold colors and contrasting lighting to create a sense of drama and tension in the scene. I can then pipe this into Dall-E 2. Sure, it's not GPT3 drawing the image, and Dall-E 2 might struggle to draw that, but most humans would struggle to draw that too as drawing is hard. Hard things are sometimes hard.
- miguelrochefort 4y ago
  I would be surprised if we did not already have AIs that could invent calculus from first principles. LLMs solve natural langage and common sense, which are not necessary for discovering calculus.
suprjami 4y ago
King's Quest 2 is AGI hard. I have no idea how anyone was supposed to find required items hidden away in any of hundreds of trees in the game. It was definitely a different time for video games back then.
- Tepix 4y ago
  But that sounds like it could just be brute forced.
- dmitriid 4y ago
  Try Noita. No idea how people discovered all the ways to complete all the quests in the game. See e.g. https://noita.wiki.gg/wiki/Sun_Quest_Guide https://noita.wiki.gg/wiki/Sun_Quest_Guide Same goes for Spelunky, or Caves of Qud, or nearly any modern roguelike/rogue-lite game.
logicallee 4y ago
I hereby endorse ChatGPT as a limited form of AGI. It can complete a variety of novel cognitive tasks (example: judging texts and proposals) often enough that it meets my personal requirements for this. My opinion is that the definitions which others believe disqualify ChatGPT from being considered AGI are not relevant, since they focus on things it fails at rather than what it can do. For example, when it tests as well as average human students, people focus on its mistakes rather than its achievements applying general forms of intelligence to succeed at that level. I've also personally interacted with it as an intelligent conversational partner with severe limitations. It's not that it can do everything, but that it does things that require what meets my personal definition of general intelligence often enough to qualify as limited AGI. You can have your own definition. Maybe that means never dropping the ball. Maybe that means getting 100% on tests. It's not my definition. In my opinion, ChatGPT is a general form of intelligence, remarkably, able to generate novel opinions and judgments about all sorts of situations, as well as sometimes briefly appear to learn things. Sure it has limitations, sure it isn't human in its abilities, but it passes my tests often enough that I personally consider AGI a milestone humanity successfully passed with the release of ChatGPT on November 30, 2022. As someone who personally uses ChatGPT's general intelligence on a daily basis and has not encountered issues with its cognitive function that are serious enough to stop me from continuing to do so, I can confidently say that ChatGPT meets my definition of limited AGI.
- robgibbons 4y ago
  Ask ChatGPT if it's an AGI, and it will tell you plainly that it is not, and why. Here's GPT's response: No, ChatGPT is not an AGI (Artificial General Intelligence). It is a large language model that has been trained to predict the next word in a sequence of text given the previous words. While ChatGPT has been trained on a diverse range of text data and is able to generate human-like text, it is not capable of understanding or learning about the world in the way that a human does. It does not have the ability to reason, plan, or solve problems in a general way. Instead, it is specifically designed to generate text based on patterns it has learned from the data it was trained on.
  - vanviegen 4y ago
swyx 4y ago
[author here] oh wow, surprised to see this here! it flopped the first time I posted it. questions/corrections/additions welcome!
- kordlessagain 4y ago
  There's not one mention of AI having a body in your article. IMO, this is the primary missing factor in discussing the technology. And memories of that body's experience, of course.
imtringued 4y ago
If AGI is a class of problems that require AGI then AGI hard only refers to the hardest problems that any AGI problem can be translated to. If we assume that humans are in AGI then AGI hard would be at least as intelligent as humans but possibly even more intelligent than humans. I don't believe most people have "superior over humans" in mind when they say AGI-hard. They just mean a hard problem that is also in AGI.
beefield 4y ago
I occasionally wonder what would be examples of types of problems/questions that would be easy for post-singular superhuman AGI but very hard/impossible for humans? Not in sense how fast the problem is solved, but in the sense that the question/answer is even understood? So far I have come up with high-dimensional spatial awareness (not sure how well we could get what happens in 1000 dimensions).
- zone411 4y ago
  It's an interesting question. Solving extremely difficult math problems, e.g. the Riemann hypothesis or P vs NP, would be very impressive. Or coming up with algorithms given many examples of desired results. There should be many benefits to having a much better short-term memory and instant access to the accumulated human knowledge. We've already seen AI do surprising things through just pattern recognition, such as predicting a patient's race from medical images when human experts cannot do it. AI might be able to understand an extremely complex system, such as the world economy, and tell us what to change to make it work better. It might not be what you asked for but in practice "any sufficiently advanced technology is indistinguishable from magic" rule will come into play even with just quantitative advances. And we might get practical quantum computers at some point...
- sinenomine 4y ago
  It is also interesting to consider problems that will likely be still hard even for advanced AIs: Control of (complex) nonlinear dynamical systems over long timescales will likely pose a problem and will require high-level unsavory shortcuts to work out. This may or may not include large-scale genetic engineering of mammals (including us) to express new characteristics later in life. The moderate-scale genetic engineering, for example to uplift our cognitive capability or extend our lifespans could be easier for AIs, while for our own scientists it turned out to be barely tractable. Some will disagree here, noting the political red tape around the fields in question.
vlovich123 4y ago
Here are simpler definitions I think: An AI system is General and “adult” level if it can improve its own performance on some task without human support to be taught how to do that. In other words, can it correctly decide when it needs practice (skill acquisition) vs knowledge acquisition (filling information gaps) on its own and figure out how to acquire that (or make clear, explicit, detailed, requests for what it needs from a human and maybe negotiate alternatives if the first ask isn’t available). Right now at best we have “baby” AI where we half to spoon feed it everything and the result is coherent but nonsensical speech. Even if we made it sensical though, that would fail the generality piece unless the AI could guide the human on what could be done to make it better (and level above that would be the AI exploring doing that on its own). Can it correctly distinguish knowledge learned vs verified and figure out when learned knowledge should be ideally be verified to double-check the quality of the knowledge. In other words, can it acquire a skill it wasn’t programmed for from scratch without losing its ability to perform similarly on other tasks? An example of a concrete problem would be can it go and analyze a bunch of academic papers looking for obvious fraud but also find when an entire field is based on unreproduced and shaky/conflicting results? Of course, “AGI-Hard” problems can be solved in one of two ways. Humans building more and more capable AI systems that can chip away at problems or AGI systems building more general, smarter, faster AI systems. The whole dream of the singularity is that we build the latter because that basically builds a second intelligent life form we can converse with on some level (although of course we likely won’t be able to understand anything it tries to explain to us that’s sufficiently complex for the same reason humans can’t understand the chess moves that AI engines are making anymore).
- tlb 4y ago
  Any reinforcement learning system meets your first definition. RL systems don’t typically use separate practice sessions, though you could create one that did. But the need for deliberate practice maybe be just a biological brain thing. Instead, RL systems usually have a parameter for how much to explore new options vs. taking what seems to be the best. This lets them learn continuously as they go.
  - vlovich123 4y ago
MichaelRazum 4y ago
Isn't AGI-Hard everything that seems rather simple for animals? I would think just of robotics and navigation in unknown environment. Everything else like Chess, Text, Art, Games seems to be solvable by current AI systems. Robotics seems much much more difficult.
- tuyiown 4y ago
  Yes, the problem is to replicate hundred of millions years of acquisition of heuristics that allows cognition on «natural» environment. Those heuristics defines all bricks our cognition is based upon, think of basics things like objects permanence, we just cannot reason without such ability. To me, as long as no AI manage to replicate those bricks, at least some of them, it won't be able to interact with us in ways that helps us efficiently with the outer, real, world.
  - zone411 4y ago
    I predict that this will a research area that advances hugely after we solve text-to-video and text-to-3D (so maybe in 2 years?): learning these types of heuristics from billions of hours of videos. Implementing this in robots is another matter, but there has been some good progress in robotic grasping and manipulation, for example.
- tgv 4y ago
  Chess and other games are solved, because they use well-defined rules, and have a combinatorial explosion that we cannot comprehend. Text and art are not solved. What would solved even mean for those? There are NNs which produce decent output given a simple prompt, but e.g. art is characterized to a large extent by development over time. Stable Diffusion isn't going to develop by itself. Robotics OTOH is much simpler, but it might depend on your expectations. Look at Boston Dynamics.
  - MichaelRazum 4y ago
    Agree, solved is not well defined. Let's just say it becomes hard to distinguish it from a human. For robotics, I just think of a "simple" tasks like picking up an object and moving it to some place. Haven't seen yet something like this in robotics, that looks human or animal like.
albertzeyer 4y ago
As LeCun always says: The term "AGI" is a bit ill-formed. What does "general" really mean? Human intelligence is not really general. It's actually very very specialized towards the world we live in. So he prefers the term "human-level intelligence", or maybe super-human-level at some point. Another implication: It's not really a yes/no property, whether you have AGI or not. It's a level of intelligence. And this level is continuous. And it's also not really one-dimensional. What does human-level intelligence mean? It's still not very well-defined. You would need to define a number of tests, which measure the performance, and the model need to perform at least as good as a human on average. So, I assume this is what the author means by "AGI-hard": A problem which requires some model to be at least as good as a human in a wide number of tests. But I don't think this is necessary for driving. You want it actually to be much better in some tests (computer vision, reaction time, etc) and many other tests don't really matter (eg. logical reasoning, speech recognition, or so). So the autonomous driving problem is not AGI-hard.
- fnfontana 4y ago
  Indeed. A person can be bad at math, but really good on play musical instruments or to guide himself through a unknown city. That requires a certain level of intelligence. But I think there's no human that is intelligent on every aspect, maybe some exceptions like Da Vinci, but most people are good at some tasks while other people doesn't. The human intelligence seems to be distributed throughout the population. This creates an illusion that human intelligence is general.
  - deleted 4y ago
    [deleted]
  - notahacker 4y ago
    But there's no such thing as.a person who is flawless at maths but has no concept of a musical instrument or navigating a city, or moving their limbs, or communicating, or eating, or setting itself any goals, or achieving any goals other than answer the math question. General intelligence isn't a claim that all humans are polymaths. It's a claim that AIs are so overfitted to succeeding in one narrow human domain like rearranging ASCII strings they fail to achieve infant-level (or even worm-level) competence in virtually all others, and that it responds to new situations by assuming they're an instance of what it has been trained to do rather than considering different approaches.
substation13 4y ago
I dislike using hard-CS-like-language for vague concepts in order to make it sound more scientific.
chriswarbo 4y ago
https://arxiv.org/abs/cs/0605024 https://arxiv.org/abs/cs/0605024 gives a formal model of intelligence (achieving high reward in an agent/environment setup, across all computable environments weighted by their Kolmogorov complexity) https://arxiv.org/abs/1109.5951 https://arxiv.org/abs/1109.5951 gives a computable approximation (dubbed AIQ, Algorithmic Intelligence Quotient), and a reference implementation using Brainfuck programs (personally I would prefer Binary Combinatory Logic or Binary Lambda Calculus ;) )
- sinenomine 4y ago
  To add to it: a recent attempt at designing a psychometric test encoding human cognitive priors: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 - the ARC dataset. https://github.com/google/BIG-bench https://github.com/google/BIG-bench - the most recent kitchen sink of a cognitive test, which includes the previous test as a subtask. Apparently there are large language models (Chinchilla) that perform comparably to a median human level on this second test.
drdrek 4y ago
The goal posts aren't moving, or not more than in any other technology. The graphics of the N64 where super realistic at the time and people will complain that COD games from 3 years ago looks dated. When will graphics be done? when they are higher fidelity than what we can perceive and we can no longer distinguish under no limitations of view time and input. When will AGI be done? when a human can no longer distinguish between a computer and human without limitations on time and input. But I guess there is no article writing left to do than is there?
YeGoblynQueenne 4y ago
>> Software engineers and computer scientists already have a tool for understanding the limits of infinite spaces - the P vs NP problem - as well as an accepted wisdom that there is a known area of algorithm research that is too unrealistic to be productive, because working on it would be functionally equivalent to proving P = NP. Not to be contrary, but P ≠ NP is only conjectured and some computer scientists do indeed expect that a proof of P = NP will eventually be demonstrated. Not me, actually, but for example a computer science author called Donald Knuth [1] thinks that P = NP and that a proof will eventually become known but even so it's not going to be terribly useful to anyone: https://www.youtube.com/watch?v=XDTOs8MgQfg https://www.youtube.com/watch?v=XDTOs8MgQfg ______ [1] Yes I know. But just in case.
xeyownt 4y ago
Weird article. After reading most of it, I still don't know what AGI or AGI-hard is. Going to feed it into ChatGPT and have it explain to me...
- xeyownt 4y ago
  The query: Hello ! I'm going to paste below a copy of a blog post on "AGI", which I don't get at all. Could you help me understand it, and/or possibly rewrite it in a way that would make it clearer or easier to understand. Here the copy-pasted article: <article pasted here> The answer: The author of this blog post is discussing the concept of artificial general intelligence (AGI), which is often described as the hypothetical future ability of artificial intelligence (AI) to understand or learn any intellectual task that a human being can. The author notes that it is difficult to have a meaningful conversation about the potential for AI to perform various tasks because the definition of AI is so broad and vague, and because it is difficult to distinguish between realistic and unrealistic predictions about what AI might be able to achieve. The author suggests that the concept of "NP-hard" problems, which refers to a class of problems in computer science that are difficult to solve and may require a very long time to complete, could be a useful way to frame discussions about the limits of AI and what it might be able to achieve. The author also discusses the importance of considering the social and ethical implications of AI, and the need to be cautious about hype and over-exaggeration in discussions about the potential of AI.
- xeyownt 4y ago
  This would be a nice addition to HackerNews. Feed all linked article into ChatGPT, and have her generate automatically an abstract. (I guess we should say "have her generate AI-tically an abstract)
ben7799 4y ago
I find it interesting that everyone is arguing about AGI-hard or AI-complete or whatever and trying to find where those boundaries are. Those are interesting problems for sure. But that's not what he's talking about, it's not the point. The scary thing and the important point in the article is that most of the press and non-technical people are being sold the idea that nothing is AGI-hard or requires Human Intelligence, whatever that is. Large swaths of press & so called "tech" people in business think all these hard problems are already solved. And they use this to seek investment or pump up companies in weird ways, only to later be mystified when the AI success doesn't appear to work like they thought. It's like arguing about whether something is NP-complete or not when a bunch of businesses are taking lots of money from investors with a pitch that makes it sound like they have figured out a way to efficiently solve all problems that were previously thought to be NP-complete. But they've shown no evidence they can do so.
- swyx 4y ago
  > But that's not what he's talking about, it's not the point. The scary thing and the important point in the article is that most of the press and non-technical people are being sold the idea that nothing is AGI-hard or requires Human Intelligence, whatever that is. wow, you phrased it better than i did. thank you!
ohwellhere 4y ago
What mechanism behind intelligence could only be found in the natural world but not made into a machine? That's the thought experiment I like in this arena. It's not useful for identifying a time to AGI, but it makes me believe AGI is possible eventually. Trillions of connections in the brain? Okay, we can make networks with trillions of parameters. Body with sensory information? We can add that. Emotions via an endocrine system? Separate interconnected subsystems? All workable in some form. A pre-encoded prior of a billion years of evolution? That one is harder, but if we're working in GHz rather than 4 Hz, can we get close enough to bootstrap? In a fully materialist interpretation of the world, I don't see anything stopping AGI from being possible.

131 points by freediver 4y ago

I like the concept of AGI-hard and the characterization of the common traps of AI productization feels accurate. One shortcoming of the analogy is that we have methods to prove when a problem is NP-hard. Are there ways to prove a problem is AGI-hard? Can it even be rigorously characterized? Relying on someone asserting it on Twitter feels unsatisfying (e.g. how accurate would experts have been at predicting the current capabilities of AI if you asked them 10 years ago? I think not very).

wilg 4y ago
Exactly – "perfect visual reasoning is AGI-hard", the tweet that seems to have inspired this, suffers from several problems: • "perfect visual reasoning" is not a thing, because "visual reasoning" isn't clearly defined. Nor "reasoning". Most importantly, neither is "perfect". • It's not clear whether it's even accurate to say you need "perfect visual reasoning" for the application at hand (driving) Determining whether something is AGI-hard is AGI-hard.
trashtester 4y ago
> how accurate would experts have been at predicting the current capabilities of AI if you asked them 10 years ago? Maybe less inaccurate than you may think. Especially if you include "experts" that focus specifically at estimating future developments, and ignore "experts" that have focused all their effort into some very specific algorithmic detail. It seems to me that a key factor in longer-term estimations is simply compute performance. If development of AI seems to be lagging somewhat compared to some predictions, it seems to me that this lag is similar to the slowdown in Moore's law. Also, for robotics, wearables and vehicles, power consumption of electronics combined with the fact that batteries as still heavy and expensive is an important limiting factor. Now the METHODS we use to reach predictions of people like Kurzweil may be different from what we imagined. Specifically, it seems that many futurists were thinking that AGI would be reached through Turing Machines and rationalist algorithms. Instead it turns out that most progress is made through building generic learning architectures (like Transformers) and apply them at huge scale with vast amounts of data. Similarily, a generation ago, we may have imagined we could have a single-core processor performing 1petaflop of computation in the 2020's. Instead, we have the 4090 now, that can do that many computations, but they are limited to "tensor" operations. Now, assuming we're not making some drastic breakthrough that enables a "rationalist" type of algorithm to form AGI, we should still reach human brain-level of raw compute power sometime between 2030-2050 (which fits reasonably well with many predictions), and I would be surprised if we don't have full AGI within such a timeframe. I would give about 1:1 odds that it happens before 2040. Still, ideas based on future computers being generic Turing Machines, just faster mean some predictions become fundamentally hard. For instance, Uploading our brains to a computer becomes a lot harder of the computer is not designed simply as a faster Turing Machine, but rather has hardware that is very specialized for some running some simple low level computation massively parallel. It may be close to impossible to run a human brain inside a GPU (at least efficiently), regardless of how many CUDA cores it has. That means that "uploading" may require us to make a near-exact replica of an actual human brain, with it's spiking style neurons, in silicon. Also, related predictions by people like Kurzweil may seem far fetched, such as his preditions about nanotech. On the other hand, we HAVE actually started large scale treatment of a large percentage of the world's population using nanotech medicine (mRNA vaccines). Maybe the revolution he predicted is actually about to happen. But my expectation is that it's going to be relatively slow for a few more decades. It may become radically better if techniques like AlphaFold develop at Moore's law pace in terms of price an capability, but I suspect generic nanotech is a harder problem than AGI simply from a computational perspective.

jasfi 4y ago

As always, what can't be done is shrinking.

palad1n 4y ago

Wikipedia: https://en.wikipedia.org/wiki/AI-complete https://en.wikipedia.org/wiki/AI-complete (AI hard redirected to this)

janalsncm 4y ago

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Physics? Calculus? Programming other AI agents? Why not add chess to the list? The fact that we already have an algorithm that enables computers to teach themselves to play chess is not an a priori reason to exclude it.

Animats 4y ago
> This article is a bit too hand-wavy for my taste. From the article: "I think this is the sort of debate that makes good podcast fodder". So the article author admits that. It's hand-wavy on vision. Parts of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") What's inherently hard is always a good question. Aristotle thought that the human ability to do arithmetic was a key indicator of intelligence. Now we know just how few gates it takes to do arithmetic. A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI. As I've pointed out before, we're still nowhere on "common sense", defined as not screwing up in the next 30 seconds. This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.
- klipt 4y ago
  > huge volumes of plausible blithering that's factually wrong That describes large parts of the internet, and we trained these models on that, so what did we really expect? ;-)

hakuseki 4y ago

I'm not sold on the concept of AI-hard or AI-complete problems. For example, the Wikipedia article on AI-completeness mentions Bongard problems and Autonomous driving as examples of problems that might be AI-complete. OK, so if I have an AI drives autonomously, is there some known querying strategy that I can use to make it solve Bongard problems? Can a Bongard problem-solving AI be made, by some known procedure to drive a car? Without such reductions, at least the analogy to NP-hardness is incomplete. I believe these reductions are precisely what makes NP-hardness such a useful concept; even though we still haven't proven that any of these problems are objectively "hard," we are still able to show that if one of them is hard, then the others are as well!

sanxiyn 4y ago

> Can AI invent calculus from first principles? The thing is, most people can't. In fact, if you reinvented calculus yourselves, you should be very proud of your mathematical ability.

xlii 4y ago
I believe AI is plural here and relates to general existing population. It’s not like asking if Stable Diffusion or ChatGPT can do it. Maybe there is specialized commercial AI that aren’t so widely known. There exist right now people with low levels of physical and mental capabilities that current state of AI knowledge surpasses it by miles. We don’t go around saying “AI is better than humans”, even though - technically - it can be in some selected cases.
Closi 4y ago
Ha, yes. AGI-Hard also needs to be contrasted against Human-Hard, which is done in most of the article, but equally we probably get to a place of saying "Hard things are Hard". Another example is that the article says "It would be hard for an AI to compose this image as it has lots of different elements/emotions" - but see the Chat-GPT example below: > Me: Can you generate a prompt for an AI to generate an interesting image? > GPT3: Sure! Here is a prompt for an AI to generate an interesting image: "Create a surreal scene featuring a giant, humanoid robot standing on a desolate, alien planet, with a small group of humans huddled fearfully at its feet. The robot should have a sleek, futuristic design, and its glowing red eyes should convey a sense of intelligence and power. In the background, there should be a faint hint of a cityscape, suggesting that the robot has traveled far from its home world. Use bold colors and contrasting lighting to create a sense of drama and tension in the scene. I can then pipe this into Dall-E 2. Sure, it's not GPT3 drawing the image, and Dall-E 2 might struggle to draw that, but most humans would struggle to draw that too as drawing is hard. Hard things are sometimes hard.
miguelrochefort 4y ago
I would be surprised if we did not already have AIs that could invent calculus from first principles. LLMs solve natural langage and common sense, which are not necessary for discovering calculus.

suprjami 4y ago

King's Quest 2 is AGI hard. I have no idea how anyone was supposed to find required items hidden away in any of hundreds of trees in the game. It was definitely a different time for video games back then.

Tepix 4y ago
But that sounds like it could just be brute forced.
dmitriid 4y ago
Try Noita. No idea how people discovered all the ways to complete all the quests in the game. See e.g. https://noita.wiki.gg/wiki/Sun_Quest_Guide https://noita.wiki.gg/wiki/Sun_Quest_Guide Same goes for Spelunky, or Caves of Qud, or nearly any modern roguelike/rogue-lite game.

logicallee 4y ago

I hereby endorse ChatGPT as a limited form of AGI. It can complete a variety of novel cognitive tasks (example: judging texts and proposals) often enough that it meets my personal requirements for this. My opinion is that the definitions which others believe disqualify ChatGPT from being considered AGI are not relevant, since they focus on things it fails at rather than what it can do. For example, when it tests as well as average human students, people focus on its mistakes rather than its achievements applying general forms of intelligence to succeed at that level. I've also personally interacted with it as an intelligent conversational partner with severe limitations. It's not that it can do everything, but that it does things that require what meets my personal definition of general intelligence often enough to qualify as limited AGI. You can have your own definition. Maybe that means never dropping the ball. Maybe that means getting 100% on tests. It's not my definition. In my opinion, ChatGPT is a general form of intelligence, remarkably, able to generate novel opinions and judgments about all sorts of situations, as well as sometimes briefly appear to learn things. Sure it has limitations, sure it isn't human in its abilities, but it passes my tests often enough that I personally consider AGI a milestone humanity successfully passed with the release of ChatGPT on November 30, 2022. As someone who personally uses ChatGPT's general intelligence on a daily basis and has not encountered issues with its cognitive function that are serious enough to stop me from continuing to do so, I can confidently say that ChatGPT meets my definition of limited AGI.

robgibbons 4y ago
Ask ChatGPT if it's an AGI, and it will tell you plainly that it is not, and why. Here's GPT's response: No, ChatGPT is not an AGI (Artificial General Intelligence). It is a large language model that has been trained to predict the next word in a sequence of text given the previous words. While ChatGPT has been trained on a diverse range of text data and is able to generate human-like text, it is not capable of understanding or learning about the world in the way that a human does. It does not have the ability to reason, plan, or solve problems in a general way. Instead, it is specifically designed to generate text based on patterns it has learned from the data it was trained on.
- vanviegen 4y ago

swyx 4y ago

[author here] oh wow, surprised to see this here! it flopped the first time I posted it. questions/corrections/additions welcome!

kordlessagain 4y ago
There's not one mention of AI having a body in your article. IMO, this is the primary missing factor in discussing the technology. And memories of that body's experience, of course.

imtringued 4y ago

If AGI is a class of problems that require AGI then AGI hard only refers to the hardest problems that any AGI problem can be translated to. If we assume that humans are in AGI then AGI hard would be at least as intelligent as humans but possibly even more intelligent than humans. I don't believe most people have "superior over humans" in mind when they say AGI-hard. They just mean a hard problem that is also in AGI.

beefield 4y ago

I occasionally wonder what would be examples of types of problems/questions that would be easy for post-singular superhuman AGI but very hard/impossible for humans? Not in sense how fast the problem is solved, but in the sense that the question/answer is even understood? So far I have come up with high-dimensional spatial awareness (not sure how well we could get what happens in 1000 dimensions).

zone411 4y ago
It's an interesting question. Solving extremely difficult math problems, e.g. the Riemann hypothesis or P vs NP, would be very impressive. Or coming up with algorithms given many examples of desired results. There should be many benefits to having a much better short-term memory and instant access to the accumulated human knowledge. We've already seen AI do surprising things through just pattern recognition, such as predicting a patient's race from medical images when human experts cannot do it. AI might be able to understand an extremely complex system, such as the world economy, and tell us what to change to make it work better. It might not be what you asked for but in practice "any sufficiently advanced technology is indistinguishable from magic" rule will come into play even with just quantitative advances. And we might get practical quantum computers at some point...
sinenomine 4y ago
It is also interesting to consider problems that will likely be still hard even for advanced AIs: Control of (complex) nonlinear dynamical systems over long timescales will likely pose a problem and will require high-level unsavory shortcuts to work out. This may or may not include large-scale genetic engineering of mammals (including us) to express new characteristics later in life. The moderate-scale genetic engineering, for example to uplift our cognitive capability or extend our lifespans could be easier for AIs, while for our own scientists it turned out to be barely tractable. Some will disagree here, noting the political red tape around the fields in question.

vlovich123 4y ago

Here are simpler definitions I think: An AI system is General and “adult” level if it can improve its own performance on some task without human support to be taught how to do that. In other words, can it correctly decide when it needs practice (skill acquisition) vs knowledge acquisition (filling information gaps) on its own and figure out how to acquire that (or make clear, explicit, detailed, requests for what it needs from a human and maybe negotiate alternatives if the first ask isn’t available). Right now at best we have “baby” AI where we half to spoon feed it everything and the result is coherent but nonsensical speech. Even if we made it sensical though, that would fail the generality piece unless the AI could guide the human on what could be done to make it better (and level above that would be the AI exploring doing that on its own). Can it correctly distinguish knowledge learned vs verified and figure out when learned knowledge should be ideally be verified to double-check the quality of the knowledge. In other words, can it acquire a skill it wasn’t programmed for from scratch without losing its ability to perform similarly on other tasks? An example of a concrete problem would be can it go and analyze a bunch of academic papers looking for obvious fraud but also find when an entire field is based on unreproduced and shaky/conflicting results? Of course, “AGI-Hard” problems can be solved in one of two ways. Humans building more and more capable AI systems that can chip away at problems or AGI systems building more general, smarter, faster AI systems. The whole dream of the singularity is that we build the latter because that basically builds a second intelligent life form we can converse with on some level (although of course we likely won’t be able to understand anything it tries to explain to us that’s sufficiently complex for the same reason humans can’t understand the chess moves that AI engines are making anymore).

tlb 4y ago
Any reinforcement learning system meets your first definition. RL systems don’t typically use separate practice sessions, though you could create one that did. But the need for deliberate practice maybe be just a biological brain thing. Instead, RL systems usually have a parameter for how much to explore new options vs. taking what seems to be the best. This lets them learn continuously as they go.
- vlovich123 4y ago

MichaelRazum 4y ago

Isn't AGI-Hard everything that seems rather simple for animals? I would think just of robotics and navigation in unknown environment. Everything else like Chess, Text, Art, Games seems to be solvable by current AI systems. Robotics seems much much more difficult.

tuyiown 4y ago
Yes, the problem is to replicate hundred of millions years of acquisition of heuristics that allows cognition on «natural» environment. Those heuristics defines all bricks our cognition is based upon, think of basics things like objects permanence, we just cannot reason without such ability. To me, as long as no AI manage to replicate those bricks, at least some of them, it won't be able to interact with us in ways that helps us efficiently with the outer, real, world.
- zone411 4y ago
  I predict that this will a research area that advances hugely after we solve text-to-video and text-to-3D (so maybe in 2 years?): learning these types of heuristics from billions of hours of videos. Implementing this in robots is another matter, but there has been some good progress in robotic grasping and manipulation, for example.
tgv 4y ago
Chess and other games are solved, because they use well-defined rules, and have a combinatorial explosion that we cannot comprehend. Text and art are not solved. What would solved even mean for those? There are NNs which produce decent output given a simple prompt, but e.g. art is characterized to a large extent by development over time. Stable Diffusion isn't going to develop by itself. Robotics OTOH is much simpler, but it might depend on your expectations. Look at Boston Dynamics.
- MichaelRazum 4y ago
  Agree, solved is not well defined. Let's just say it becomes hard to distinguish it from a human. For robotics, I just think of a "simple" tasks like picking up an object and moving it to some place. Haven't seen yet something like this in robotics, that looks human or animal like.

albertzeyer 4y ago

As LeCun always says: The term "AGI" is a bit ill-formed. What does "general" really mean? Human intelligence is not really general. It's actually very very specialized towards the world we live in. So he prefers the term "human-level intelligence", or maybe super-human-level at some point. Another implication: It's not really a yes/no property, whether you have AGI or not. It's a level of intelligence. And this level is continuous. And it's also not really one-dimensional. What does human-level intelligence mean? It's still not very well-defined. You would need to define a number of tests, which measure the performance, and the model need to perform at least as good as a human on average. So, I assume this is what the author means by "AGI-hard": A problem which requires some model to be at least as good as a human in a wide number of tests. But I don't think this is necessary for driving. You want it actually to be much better in some tests (computer vision, reaction time, etc) and many other tests don't really matter (eg. logical reasoning, speech recognition, or so). So the autonomous driving problem is not AGI-hard.

fnfontana 4y ago
Indeed. A person can be bad at math, but really good on play musical instruments or to guide himself through a unknown city. That requires a certain level of intelligence. But I think there's no human that is intelligent on every aspect, maybe some exceptions like Da Vinci, but most people are good at some tasks while other people doesn't. The human intelligence seems to be distributed throughout the population. This creates an illusion that human intelligence is general.
- deleted 4y ago
  [deleted]
- notahacker 4y ago
  But there's no such thing as.a person who is flawless at maths but has no concept of a musical instrument or navigating a city, or moving their limbs, or communicating, or eating, or setting itself any goals, or achieving any goals other than answer the math question. General intelligence isn't a claim that all humans are polymaths. It's a claim that AIs are so overfitted to succeeding in one narrow human domain like rearranging ASCII strings they fail to achieve infant-level (or even worm-level) competence in virtually all others, and that it responds to new situations by assuming they're an instance of what it has been trained to do rather than considering different approaches.

substation13 4y ago

I dislike using hard-CS-like-language for vague concepts in order to make it sound more scientific.

chriswarbo 4y ago

https://arxiv.org/abs/cs/0605024 https://arxiv.org/abs/cs/0605024 gives a formal model of intelligence (achieving high reward in an agent/environment setup, across all computable environments weighted by their Kolmogorov complexity) https://arxiv.org/abs/1109.5951 https://arxiv.org/abs/1109.5951 gives a computable approximation (dubbed AIQ, Algorithmic Intelligence Quotient), and a reference implementation using Brainfuck programs (personally I would prefer Binary Combinatory Logic or Binary Lambda Calculus ;) )

sinenomine 4y ago
To add to it: a recent attempt at designing a psychometric test encoding human cognitive priors: https://arxiv.org/abs/1911.01547 https://arxiv.org/abs/1911.01547 - the ARC dataset. https://github.com/google/BIG-bench https://github.com/google/BIG-bench - the most recent kitchen sink of a cognitive test, which includes the previous test as a subtask. Apparently there are large language models (Chinchilla) that perform comparably to a median human level on this second test.

drdrek 4y ago

The goal posts aren't moving, or not more than in any other technology. The graphics of the N64 where super realistic at the time and people will complain that COD games from 3 years ago looks dated. When will graphics be done? when they are higher fidelity than what we can perceive and we can no longer distinguish under no limitations of view time and input. When will AGI be done? when a human can no longer distinguish between a computer and human without limitations on time and input. But I guess there is no article writing left to do than is there?

YeGoblynQueenne 4y ago

>> Software engineers and computer scientists already have a tool for understanding the limits of infinite spaces - the P vs NP problem - as well as an accepted wisdom that there is a known area of algorithm research that is too unrealistic to be productive, because working on it would be functionally equivalent to proving P = NP. Not to be contrary, but P ≠ NP is only conjectured and some computer scientists do indeed expect that a proof of P = NP will eventually be demonstrated. Not me, actually, but for example a computer science author called Donald Knuth [1] thinks that P = NP and that a proof will eventually become known but even so it's not going to be terribly useful to anyone: https://www.youtube.com/watch?v=XDTOs8MgQfg https://www.youtube.com/watch?v=XDTOs8MgQfg ______ [1] Yes I know. But just in case.

xeyownt 4y ago

Weird article. After reading most of it, I still don't know what AGI or AGI-hard is. Going to feed it into ChatGPT and have it explain to me...

xeyownt 4y ago
The query: Hello ! I'm going to paste below a copy of a blog post on "AGI", which I don't get at all. Could you help me understand it, and/or possibly rewrite it in a way that would make it clearer or easier to understand. Here the copy-pasted article: <article pasted here> The answer: The author of this blog post is discussing the concept of artificial general intelligence (AGI), which is often described as the hypothetical future ability of artificial intelligence (AI) to understand or learn any intellectual task that a human being can. The author notes that it is difficult to have a meaningful conversation about the potential for AI to perform various tasks because the definition of AI is so broad and vague, and because it is difficult to distinguish between realistic and unrealistic predictions about what AI might be able to achieve. The author suggests that the concept of "NP-hard" problems, which refers to a class of problems in computer science that are difficult to solve and may require a very long time to complete, could be a useful way to frame discussions about the limits of AI and what it might be able to achieve. The author also discusses the importance of considering the social and ethical implications of AI, and the need to be cautious about hype and over-exaggeration in discussions about the potential of AI.
xeyownt 4y ago
This would be a nice addition to HackerNews. Feed all linked article into ChatGPT, and have her generate automatically an abstract. (I guess we should say "have her generate AI-tically an abstract)

ben7799 4y ago

I find it interesting that everyone is arguing about AGI-hard or AI-complete or whatever and trying to find where those boundaries are. Those are interesting problems for sure. But that's not what he's talking about, it's not the point. The scary thing and the important point in the article is that most of the press and non-technical people are being sold the idea that nothing is AGI-hard or requires Human Intelligence, whatever that is. Large swaths of press & so called "tech" people in business think all these hard problems are already solved. And they use this to seek investment or pump up companies in weird ways, only to later be mystified when the AI success doesn't appear to work like they thought. It's like arguing about whether something is NP-complete or not when a bunch of businesses are taking lots of money from investors with a pitch that makes it sound like they have figured out a way to efficiently solve all problems that were previously thought to be NP-complete. But they've shown no evidence they can do so.

swyx 4y ago
> But that's not what he's talking about, it's not the point. The scary thing and the important point in the article is that most of the press and non-technical people are being sold the idea that nothing is AGI-hard or requires Human Intelligence, whatever that is. wow, you phrased it better than i did. thank you!

ohwellhere 4y ago

What mechanism behind intelligence could only be found in the natural world but not made into a machine? That's the thought experiment I like in this arena. It's not useful for identifying a time to AGI, but it makes me believe AGI is possible eventually. Trillions of connections in the brain? Okay, we can make networks with trillions of parameters. Body with sensory information? We can add that. Emotions via an endocrine system? Separate interconnected subsystems? All workable in some form. A pre-encoded prior of a billion years of evolution? That one is harder, but if we're working in GHz rather than 4 Hz, can we get close enough to bootstrap? In a fully materialist interpretation of the world, I don't see anything stopping AGI from being possible.