5 ms·

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artifi

This article is a bit too hand-wavy for my taste. As a buzzword, AGI Hard sounds cool but until intelligence is more strictly defined (and the modifiers “artificial” and “general” aren’t helping) AGI will always be something we talk about rather than something known. I was also a bit amused at the list of things which the author also claims to be AGI hard. But this is just a list of things we think are difficult. Physics? Calculus? Programming other AI agents? Why not add chess to the list? The fact that we already have an algorithm that enables computers to teach themselves to play chess is not an a priori reason to exclude it.

Animats 4y ago
> This article is a bit too hand-wavy for my taste. From the article: "I think this is the sort of debate that makes good podcast fodder". So the article author admits that. It's hand-wavy on vision. Parts of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") What's inherently hard is always a good question. Aristotle thought that the human ability to do arithmetic was a key indicator of intelligence. Now we know just how few gates it takes to do arithmetic. A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI. As I've pointed out before, we're still nowhere on "common sense", defined as not screwing up in the next 30 seconds. This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.
- klipt 4y ago
  > huge volumes of plausible blithering that's factually wrong That describes large parts of the internet, and we trained these models on that, so what did we really expect? ;-)
- janalsncm 4y ago
  Very well said. Just want to add one thing. > This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong. If because of some law of nature ChatGPT were the last iteration of automatic text generation we could ever create, it would be a big and permanent problem. Right now it’s just a big and temporary problem, with a huge opportunity for automatic fact-checking. (Or more precisely, validity and soundness checking.) Maybe the answer to the bullshit factory is a rebalancing of Brandolini’s law.
  - mannykannot 4y ago
    What you call a huge opportunity is actually (or also, if you prefer) a major shortfall in the current LLMs. Whether it will turn out to be merely temporary, and more pertinently, how it might be solved and whether it can be solved with more of the same, are themselves unanswered questions at this time.
    - janalsncm 4y ago
      Right, it’s a fundamental disconnect between how an autoregressive statistical language model generates text and how people think about talking and writing. We have known this for a long time. It is the essence of the civil war between symbolic NLP folks and statistical language modeling folks. Now that ChatGPT is quasi-mainstream, the strengths and weaknesses of the latter are on display for all to see. My only point is that if we’re approaching a local maxima with purely statistical approaches, it raises the stakes for symbolic approaches to come through.
- notahacker 4y ago
  > of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") Isn't the latter part of vision the part that's "AGI" hard though, and why the article talks/handwaves about "perfect visual reasoning" and not basic classification and object tracking/predicting? LIDAR can determine there's an obstruction in the road, driving AI can determine that the obstruction appears to consist of people and their intent is not possible to predict with any confidence based on observations in its training set, a human applies higher level reasoning to infer - without having seen a visually similar group of people before - that they're protestors showing demonstrable hostility to vehicles that should be avoided, a rabble of drunks pouring out of a neighbouring venue that may part in response to forward movement and judicious use of the horn, or an organized event which probably has a set route and finishing time which can be ascertained by stopping to speak to them. Plus, of course, it can handle the non-vision based bits of that scenario without having done so before. > A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI I strongly agree with this. It's akin to Moravec's paradox. Turns out a machine can give mediocre-undergraduate level responses to complex abstract questions about human psychology purely by pattern matching what people have written (which isn't dissimilar to how mediocre undergraduates approach the question...) whilst possessing less actionable insight into human emotion than the average dog. And we've managed that without having a clue how to make a computer happy or angry.
swyx 4y ago
chess as a closed system is definitely solved by AI, including bootstrapping purely from self-play. the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).
- janalsncm 4y ago
  It’s not clear to me that the bottleneck for simulating the universe is intelligence. Our measurement capabilities are limited, and our computational power is limited. When I say our measurement capabilities are limited, I mean that there are subsystems which are fundamentally chaotic. Small measurement inaccuracies will lead to large prediction inaccuracies, which in my view has little to do with intelligence.
  - trashtester 4y ago
    Clearly, detailed long-term simulations of chaotic systems is impossible with any (non-quantum) turing machine with limited parallelism. I think the argument in the article is rather the other way around, ie that intelligence DEPENDS on a kind of simulation to develop proper intelligence. For instance AlphaZero was able to "solve" chess, go, etc, because it could simulate the game state perfectly. And, the "hard" part comes from the fact that perfect simulation is impossible (with or without AI). For an AI that will never interact with the physical world, this makes some sense. However, as I argued in my other response, while some kind of World Model (ref LeCun) may indeed be necessary for AGI, it doesn't need to be perfect. Predictive about essential aspects (those relevant for decisions and actions) is enough. https://syncedreview.com/2017/02/25/yann-le-cun-predicting-under-uncertainty-the-next-frontier-in-ai/ https://syncedreview.com/2017/02/25/yann-le-cun-predicting-u...
- trashtester 4y ago
  Any AGI that is supposed perform tasks in the physical world, will need some kind of world model. The model doesn't need to be perfect to be immensly useful. It just need to model elements that are necessary to make more accurate predictions in its domain. I would argue that exactly the same goes for the human brain. Consciousness seems to play the part of the World Model for us. Between the quantum wavefunctions of the world we live in and our conscious experience (or even our sub-conscious data processing) there are layers-upon-layars-upon-layers of data loss and data compression/abstraction. Our sensory endpoints probably receive more raw data in a second than our consciousness processes in a year. The reason it still works so well, I would argue, is a mix of darwinian learning that has been going on for hundreds of billions of years before humanity even appeared, where a data pipeline has been created in our hardware. This, combined with our ability to learn by interacting with the real world. Creating an AI with a "perfect" world model is impossible. It will require more compute power than exists in the universe. Rather, AGI will need a simpler world model that captures the parts that are essential for its functioning, and that can be automatically updated using its input data (ideally sensors). Just like us. And, it seems to me, this is already happening. A self driving car will create an approximation for a 6-dimensional state for all objects (position and velocity), possibly with acceleration, speculation about intent or other "mental states. It will then extrapolate this a few seconds (or more) into the future, while keeping track of the confidence of the predictions. This is a similar kind of world model that human brains seem to use, at least for the purpose of driving. Clearly, our brain needs a more complex model, since we make predictions much further into the future (such as the education of our children, or beyond) and we also interact with the world in more ways. When AI systems start to make use of world model of the same complexity (type and level) as our own, and is able to "train" it both "genetically" and by standard "deep learning", I suspect it will be increasingly good at displaying what we consider "common sense". And juding by recent developments, this could happens sooner than many think. If we're able to combine the features of ChatGPT, Tesla Self Driving, Stable Diffusion and perhaps a few other in a single model (that takes both visual data, sound and text as input, and which constructs a wold model that combines physical and abstract/textual/behavioral elements), this could happen within 10 years.
Existenceblinks 4y ago
The article works how it works: here is my provocative thought, comes to engage my stuff to change my mind, or criticize me that works too We should get used to this content creator thingy.
seanhunter 4y ago
Agree. I really dislike this terminology because like many analogies it tempts you into thinking it means more than it really does and thereby reasoning from it to things which are entirely unsupported by anything sound. NP-Hardness and NP-Completeness are rigorously-defined concepts. By proving a problem NP hard you establish some extremely important properties about its nature and its relationship with other problems. AGI-Hard sounds like the same sort of thing but really isn't. It's fundamentally just pure speculation. Of the three letters "AGI" it's arguable that we don't have a hard definition of what constitutes any of them. Then further to that we don't have any kind of idea what being in the set of AGI-Hard problems might mean other than you're in the set. Do these hypothetical AGI-Hard problems have some relationship with each other? Whereas we know P is in NP, do we have an equivalent class to P and what would that mean other than "The class of problems solvable by AGI"? You can see this weakness when you look at the proposed examples of what is purportedly AGI-hard. What do they have in common? Mostly just that they are weakly-defined themselves. For example. How would you judge if an AGI derived from first principles something that isn't calculus but is equivalently important/innovative? Well you'd first need to define what that equivalence would mean.
geysersam 4y ago
The first example the author gives, adjusting language to the level of a student, does not seem unrealistic at all, provided there are examples of this in the training data.

Animats 4y ago

> This article is a bit too hand-wavy for my taste. From the article: "I think this is the sort of debate that makes good podcast fodder". So the article author admits that. It's hand-wavy on vision. Parts of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") What's inherently hard is always a good question. Aristotle thought that the human ability to do arithmetic was a key indicator of intelligence. Now we know just how few gates it takes to do arithmetic. A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI. As I've pointed out before, we're still nowhere on "common sense", defined as not screwing up in the next 30 seconds. This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong.

klipt 4y ago
> huge volumes of plausible blithering that's factually wrong That describes large parts of the internet, and we trained these models on that, so what did we really expect? ;-)
janalsncm 4y ago
Very well said. Just want to add one thing. > This is a big problem now that there are systems able to produce huge volumes of plausible blithering that's factually wrong. If because of some law of nature ChatGPT were the last iteration of automatic text generation we could ever create, it would be a big and permanent problem. Right now it’s just a big and temporary problem, with a huge opportunity for automatic fact-checking. (Or more precisely, validity and soundness checking.) Maybe the answer to the bullshit factory is a rebalancing of Brandolini’s law.
- mannykannot 4y ago
  What you call a huge opportunity is actually (or also, if you prefer) a major shortfall in the current LLMs. Whether it will turn out to be merely temporary, and more pertinently, how it might be solved and whether it can be solved with more of the same, are themselves unanswered questions at this time.
  - janalsncm 4y ago
    Right, it’s a fundamental disconnect between how an autoregressive statistical language model generates text and how people think about talking and writing. We have known this for a long time. It is the essence of the civil war between symbolic NLP folks and statistical language modeling folks. Now that ChatGPT is quasi-mainstream, the strengths and weaknesses of the latter are on display for all to see. My only point is that if we’re approaching a local maxima with purely statistical approaches, it raises the stakes for symbolic approaches to come through.
notahacker 4y ago
> of vision are not "AGI-hard". A key component of self-driving is something that can determine whether a volume of space is empty, occupied, or ambiguous. It's necessary to have something that does not report "empty" for occupied areas, and has a reasonably high accuracy rate on not emitting false reports of "occupied". There are sensor suites which can do that. LIDAR can do that. If you have data from a few viewpoints, you can do that. Musk has tried to turn it into an AGI problem, but that's his problem, not a property of the problem space. (If you look at California DMV autonomous vehicle crash reports, the successful companies have solved "is that area empty", and are now struggling with "what is that other car/pedestrian/bicycle going to do?") Isn't the latter part of vision the part that's "AGI" hard though, and why the article talks/handwaves about "perfect visual reasoning" and not basic classification and object tracking/predicting? LIDAR can determine there's an obstruction in the road, driving AI can determine that the obstruction appears to consist of people and their intent is not possible to predict with any confidence based on observations in its training set, a human applies higher level reasoning to infer - without having seen a visually similar group of people before - that they're protestors showing demonstrable hostility to vehicles that should be avoided, a rabble of drunks pouring out of a neighbouring venue that may part in response to forward movement and judicious use of the horn, or an organized event which probably has a set route and finishing time which can be ascertained by stopping to speak to them. Plus, of course, it can handle the non-vision based bits of that scenario without having done so before. > A big surprise, and shock to many, has been the discovery that punditry, and what passes for intellectualism in some quarters, can be done by a really good autocomplete with a big training set. This says more about human discourse than it does about AI I strongly agree with this. It's akin to Moravec's paradox. Turns out a machine can give mediocre-undergraduate level responses to complex abstract questions about human psychology purely by pattern matching what people have written (which isn't dissimilar to how mediocre undergraduates approach the question...) whilst possessing less actionable insight into human emotion than the average dog. And we've managed that without having a clue how to make a computer happy or angry.

swyx 4y ago

chess as a closed system is definitely solved by AI, including bootstrapping purely from self-play. the claim here is that applying physics/math/etc would be AGI-hard, because it involves the AI perfectly modeling a world it doesnt live in (the simulation problem) or understanding relationships in concepts without making jumps (obeying logic).

janalsncm 4y ago
It’s not clear to me that the bottleneck for simulating the universe is intelligence. Our measurement capabilities are limited, and our computational power is limited. When I say our measurement capabilities are limited, I mean that there are subsystems which are fundamentally chaotic. Small measurement inaccuracies will lead to large prediction inaccuracies, which in my view has little to do with intelligence.
- trashtester 4y ago
  Clearly, detailed long-term simulations of chaotic systems is impossible with any (non-quantum) turing machine with limited parallelism. I think the argument in the article is rather the other way around, ie that intelligence DEPENDS on a kind of simulation to develop proper intelligence. For instance AlphaZero was able to "solve" chess, go, etc, because it could simulate the game state perfectly. And, the "hard" part comes from the fact that perfect simulation is impossible (with or without AI). For an AI that will never interact with the physical world, this makes some sense. However, as I argued in my other response, while some kind of World Model (ref LeCun) may indeed be necessary for AGI, it doesn't need to be perfect. Predictive about essential aspects (those relevant for decisions and actions) is enough. https://syncedreview.com/2017/02/25/yann-le-cun-predicting-under-uncertainty-the-next-frontier-in-ai/ https://syncedreview.com/2017/02/25/yann-le-cun-predicting-u...
trashtester 4y ago
Any AGI that is supposed perform tasks in the physical world, will need some kind of world model. The model doesn't need to be perfect to be immensly useful. It just need to model elements that are necessary to make more accurate predictions in its domain. I would argue that exactly the same goes for the human brain. Consciousness seems to play the part of the World Model for us. Between the quantum wavefunctions of the world we live in and our conscious experience (or even our sub-conscious data processing) there are layers-upon-layars-upon-layers of data loss and data compression/abstraction. Our sensory endpoints probably receive more raw data in a second than our consciousness processes in a year. The reason it still works so well, I would argue, is a mix of darwinian learning that has been going on for hundreds of billions of years before humanity even appeared, where a data pipeline has been created in our hardware. This, combined with our ability to learn by interacting with the real world. Creating an AI with a "perfect" world model is impossible. It will require more compute power than exists in the universe. Rather, AGI will need a simpler world model that captures the parts that are essential for its functioning, and that can be automatically updated using its input data (ideally sensors). Just like us. And, it seems to me, this is already happening. A self driving car will create an approximation for a 6-dimensional state for all objects (position and velocity), possibly with acceleration, speculation about intent or other "mental states. It will then extrapolate this a few seconds (or more) into the future, while keeping track of the confidence of the predictions. This is a similar kind of world model that human brains seem to use, at least for the purpose of driving. Clearly, our brain needs a more complex model, since we make predictions much further into the future (such as the education of our children, or beyond) and we also interact with the world in more ways. When AI systems start to make use of world model of the same complexity (type and level) as our own, and is able to "train" it both "genetically" and by standard "deep learning", I suspect it will be increasingly good at displaying what we consider "common sense". And juding by recent developments, this could happens sooner than many think. If we're able to combine the features of ChatGPT, Tesla Self Driving, Stable Diffusion and perhaps a few other in a single model (that takes both visual data, sound and text as input, and which constructs a wold model that combines physical and abstract/textual/behavioral elements), this could happen within 10 years.

Existenceblinks 4y ago

The article works how it works: here is my provocative thought, comes to engage my stuff to change my mind, or criticize me that works too We should get used to this content creator thingy.

seanhunter 4y ago

Agree. I really dislike this terminology because like many analogies it tempts you into thinking it means more than it really does and thereby reasoning from it to things which are entirely unsupported by anything sound. NP-Hardness and NP-Completeness are rigorously-defined concepts. By proving a problem NP hard you establish some extremely important properties about its nature and its relationship with other problems. AGI-Hard sounds like the same sort of thing but really isn't. It's fundamentally just pure speculation. Of the three letters "AGI" it's arguable that we don't have a hard definition of what constitutes any of them. Then further to that we don't have any kind of idea what being in the set of AGI-Hard problems might mean other than you're in the set. Do these hypothetical AGI-Hard problems have some relationship with each other? Whereas we know P is in NP, do we have an equivalent class to P and what would that mean other than "The class of problems solvable by AGI"? You can see this weakness when you look at the proposed examples of what is purportedly AGI-hard. What do they have in common? Mostly just that they are weakly-defined themselves. For example. How would you judge if an AGI derived from first principles something that isn't calculus but is equivalently important/innovative? Well you'd first need to define what that equivalence would mean.

geysersam 4y ago

The first example the author gives, adjusting language to the level of a student, does not seem unrealistic at all, provided there are examples of this in the training data.