7 ms·

Are We Close to AGI?

31 points by mpwouden 4y ago

mpwouden 4y ago
How close are we to AGI and do people actually understand what that means? Too often people use the term as if General were 'ubiquitious' as in a broad application of AI instead of one that is inditinguishable from human intelligence.
- ftxbro 4y ago
  > do people actually understand what [AGI] means no because it means something different to each person (both its definition and its implications)
  - greyman 4y ago
    Exactly. For example John Carmack understands AGI as an simulation of 2-year old human brain, which we can teach anything. That's quite a good definition imo, but it seems to me this isn't achievable by evolution of LLMs. Other people thinks that by making current LLMs approach somewhat better, that could be called AGI. To sum it up, AGI isn't anything specific at this point.
- knallfrosch 4y ago
  As if we could agree on what 'human intelligence' is (or indeed, how to measure it.)
- ben_w 4y ago
  We definitely don't have consensus about either "general" or "intelligence" (and if mind-uploads ever enter the discussion, then also about "artificial"). For me, "general" is a sliding scale, not a yes-or-no boolean, so I think it's fine to say that GPT3, let alone later versions, is "general" in the domain of text — it has more general knowledge than I have, that's for sure! Likewise "intelligent", though for me that's a vector value: [i_0, i_1, …, i_m] where each i is the ability at some aspect of intelligence. In fact, I'd go further and say that you can define "generality" with respect to such a vector: if the intelligence vector is a normalised score 0-1 for each value, then generality could be easily defined by the statistics of that array. As even GPT-4 seems to be better than the layperson and worse than an expert at everything (textual) it has been tested on, I'm not sure if it's more or less general than a human by this standard. And you may wish to define intelligence by ability to learn from small data sets rather than large, at which point we then need to consider the difference between the initial training, the RLHF, and any fine-tuning; and how this compares to evolution, education, and work experience.
md_ 4y ago
Seems like a long-winded way of saying, "no". ;)
- jackmott42 4y ago
  It is telling though, that a year ago myself and I think most programmers would have just quickly said "nope definitely not", whereas now you kind of have to get long winded to say no, and many of us are not saying no but maybe AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it. What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such? Gonna be fun.
  - uoaei 4y ago
    > many of us are not saying no but maybe I'm not sure if this is a controversial stance, but not all opinions are equal. HN commenters span a wide range of competencies, but very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to.
    - UIUC_06 4y ago
      Gene Wilder meme: Tell me all about how your opinions are better than everyone else's!
      - uoaei 4y ago
        You will note I did not proffer any of my own. To avoid Dunning-Kruger, you must know what things you don't have sufficient knowledge about and avoid speaking as if you do. Also please don't "meme" on HN, it degrades discussion quality very quickly. There are other venues for that.
insane_dreamer 4y ago
No
- redeyedtreefrog 4y ago
  Betteridge's law of headlines
davewritescode 4y ago
We're not close and it probably doesn't really matter. I mean, yes I want a real world Jarvis but that's not really what's important. What's important is that AI will quickly replace toil in white collar work. Unfortunately, a lot of people make entire careers of that kind of work. "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing. We're going to have to reckon with the consequences of AI long before AGI becomes real.
- kevingadd 4y ago
  I won't soon forget how as a tools programmer at a game studio, I noticed a couple of the writers spent a big portion of their day churning text from the design team's word docs through excel macros to generate CSV files consumed by our build system. I looked at that and went "oh, I can just automate this", and replaced all that busywork with a tiny C# program that would just do it correctly all the time automatically so nobody had to do it by hand anymore. Only occurred to me much later that now they didn't have much work to do and were at a higher risk of being laid off! A good lesson to think about all the implications of the software I'm writing. We've seen AI-adjacent tooling getting leveraged to eliminate more of that category of work in games already, like automated generation of NPC "barks".
  - chii 4y ago
    > A good lesson to think about all the implications of the software I'm writing. only in so far as not reaping the rewards for having done more work than the company paid you for. Efficiency improvements are always good for society as a whole, despite it hurting some people who originally was paid due to the inefficiency.
    - mschuster91 4y ago
      > Efficiency improvements are always good for society as a whole, despite it hurting some people who originally was paid due to the inefficiency. When the economic/capital/wealth 1% end up hoarding all the profitability gains, society as a whole loses. I have come to loathe the term "disruption" as it has always meant externalizing costs to make a quick buck: either by technology replacing actual, real humans with real families who now have to struggle to survive or by breaking laws and regulations like Uber and AirBnB did, with society paying the costs associated with that - in the case of Uber, society picked up the tab for underinsured drivers or for their pension/social security contributions which don't apply for "freelancers", and in the case of AirBnB all the neighbors affected by the noise and literal other disruptions an AirBnB brings with it.
rcme 4y ago
The author's definition of "strong AI" is incredibly weak. > Theory of Mind: These AI systems are capable of understanding mental states and can simulate the behavior of other agents. GPT models can do this. > Self-Aware Why is that a requirement? > These AI systems are capable of learning any intellectual task that a human can. Again, GPT models appear to do this (at least for text-based tasks). That being said, I think GPT models do have some limitations that prevent them for learning any intellectual task a human can do. I wrote a blog post a few days ago that explored trying to "teach" ChatGPT to perform binary addition [0]. My conclusion was that, when ChatGPT appears to solve a logical problem, that solution is just a regurgitation of a low-entropy example generalized from its training data. But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems. 0: https://nobuteru.substack.com/p/can-chatgpt-learn-to-add https://nobuteru.substack.com/p/can-chatgpt-learn-to-add
- shagie 4y ago
  On the adding part - have you looked at how it tokenizes that string? https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer 10011111110100011100011011110010 00111010100101000100000101111011 is tokenized as: [100][1111][11][101][0001][11][0001][101][11][100][10] [00][11][101][01][001][01][0001][0000][01][01][11][101][1] It might be more trainable if the the bits where space delimited so that only the tokens [ 1] and [ 0] were in use. As it is, it doesn't really understand the contents of the tokens and manipulating [405, 1157, 8784, 486, 8298, 486, 18005, 2388, 486, 486, 1157, 8784, 16] is not easily trained - especially if there is a different bit sequence. Note also the misalignments of the tokens for bit length that can make it more difficult to work with.
  - rcme 4y ago
    I disagree that tokenization is important to consider. But just to test it out, I tried adding spaces between each bit and verified that each bit is tokenized separately: Let's add the two binary numbers provided, limiting the work on each bit to a single line: 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1 -------------------------------------------------------------- 1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1 So, the result of the addition is the binary number 10110110101001010111111001111101. The answer is still wrong. Think about it like this: even when tokenized into 1-5 digit binary chunks, there are still just a few tokens to consider. If ChatGPT could actually apply the rules of addition to a general, high-entropy example, it would be trivial to use these tokens to derive the correct answer. A human could easily do it.
cat_plus_plus 4y ago
Why do we need to be? Say human intelligence evolved so that we model real world events such a mammoth hunt mentally ahead of time and use simulated outcomes to craft weapons / preposition different hunters based on their skills. Would AI necessarily need to hunt or worry about energy intake/self preservation/producing offspring? It can be powered by direct photosynthesis (solar panels), hibernate when energy is in short supply and make backups to safeguard against future physical damage. Seems like we should think through the tasks that we need AI to help us with - climate change, medicine, space exploration, etc - and design systems for such tasks as efficiently as possible rather than trying to monkey ourselves. And sure, some day AI may develop its own purposes that are different from its origin, just like humans decided they like music. Doesn't mean that a caveman shouldn't have dedicated his attention to the next meal rather than worrying about being sophisticated.
- bbor 4y ago
  The simple answer is “go figure out how solve climate change” is a much easier direction to give an AI than figuring out what to do then building a bunch of little specific ones. Really all an AGI is an AI smart enough to build and use other smaller AIs, or at least that’s a very common guess as to the architecture that’ll end up working. I give it till Christmas
deleted 4y ago
[deleted]
williamcotton 4y ago
In practice, treating GPT as a weak AI and part of a larger system design is how to gain the most use from the tool. Also, I fully agree that cybernetics, human systems interactions, neuroscience and even epistemology are useful disciplines for implementing intelligent autonomous systems. I don’t have much more to add other than I like this article and the author’s point of view.
rvz 4y ago
No.
agentultra 4y ago
> AI is the development of machines with the ability to think and act like humans What we have today is machine learning... not this. LLM's are neat, transformers are a really useful innovation, etc. Some people like to leap to conclusions and claim that this is intelligence beyond human comprehension and it will learn to destroy is all. I think that's a bit much. If the goal of AGI is to be this we're nowhere near it. And I suspect there are plenty of people in this "AI" space that would not see it as a goal of the research. An expert system that can reason in a particular domain would be a huge advance and a useful tool. There's no need to also create autonomous agents that think and act like humans. We're pretty terrible at that on the whole.
titzer 4y ago
It seems like an AGI would just be a meta-machine-learning model, meaning it's a model whose whole goal is to think up new ways of training models. Eventually, it would figure out a way to either incorporate the smaller, specialized subtasks into itself and piecemeal upgrade itself. Oh, and it needs permission to use $MEGA_DOLLARS to run the computations necessary for training. So, it would probably need a side hussle. Like search ads or something. Right now, humans are either the bailing wire or the chewing gum in this situation, but they hold the purse strings and GPU farms. Humans are the ones thinking up new training strategies, curating the learning datasets, and building the cluster and cloud and datacenter systems that can handle that much data.
toteno 4y ago
I noticed that some people expect AGI to have some superhuman intelligence instead of just intelligence similar to that of humans. I have seen some posts arguing that an AI can only be called AGI if it is better than any human being on every task. It's nice to see that this article doesn't fall into that.
- deleted 4y ago
  [deleted]
throwaway22032 4y ago
Reasoning about whether we are close to solving something that is essentially a research problem is erroneous thinking to me. We could quite concievably be one or two discoveries away from improving the scaling of our current models by 10-1000x. But I don't think there is a way of determining progress on that research. It's an unknown unknown.
- bbor 4y ago
  But worth trying to guess! I think the only other project similar in character would be nuclear weapons, and I definitely think the world would’ve appreciated a heads up there. Just because it’s hard to predict doesn’t mean we shouldn’t reason about it and get some ballpark estimates, IMO
  - throwaway22032 4y ago
    Yes. I think on AI research in particular though, we could show that we're close, but not convincingly that we are far away. Not in the current world, anyway. With 1950s levels of compute, sure.
mindcrime 4y ago
I don't know, but I'd posit that anybody who answers "definitely yes" or "definitely no" doesn't know what they are talking about. And that's because we, collectively, really have no way of knowing how close to AGI we are. We struggle enough to define and measure human intelligence, and adding what is arguably a new "kind" of intelligence to the pot just complicates matters even more. So as of today, we don't really have any kind of simple metric like "IQ" that we can apply to AI systems, and even if we did have the metric, at this point we wouldn't really know it's range, whether to expect it to be a linear function or not, etc. That said... does it seem clear that we are making some kind of progress? I'd say yes, but with the admission that that's somewhat subjective. In particular, even though we know that current AI systems are getting better at doing whatever they do, we don't really know if that meaningfully counts as progress towards AGI. It is possible, for example, that LLM approaches will hit a ceiling in their development, and that the much feared "GPT 17" really won't even be a thing. And it may likewise turn out that the progress made on LLM's doesn't turn out to contribute much at all to whatever the eventual mechanism behind AGI turns out to be. Is that the way I expect things to play out? Not sure. I have a hunch that LLM's will prove to be insufficient to qualify as AGI on their own, but that they may well serve as an element of a modular / hybrid system that includes LLMs, reasoners using deduction, induction, and abduction, evolutionary algorithms, and FSM-knows what else.
- reitanqild 4y ago
  Related (and since you qualify the answers as "definitely yes" or "definitely no" I don't think it subtracts from what you say, but rather just adds to it: > When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong. (Clarkes first law, copied from https://en.wikipedia.org/wiki/Clarke%27s_three_laws https://en.wikipedia.org/wiki/Clarke%27s_three_laws)
  - mindcrime 4y ago
    > When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong. Agreed!
vsareto 4y ago
Virtual personal assistant isn't exactly a narrow definition. I'd expect that's a big role to play and we'll make AGI systems do that too. The strong AI definitions are also pretty biased towards human experiences. I wouldn't assume future innovations are necessarily human-like. It's hard to imagine a completely new thing until it gets here though. "Learning any task" is also, uh, quite expansive - does the AI lose its AGI badge if the internet finds something it sucks at? (spoiler: it will suck at something)
janalsncm 4y ago
What I would’ve preferred in this article which is largely long winded definitions and summaries of computing history that are much better explained on Wikipedia would be a workable, testable definition of AGI. How will we know when we’ve gotten there? Frankly, I don’t care about the Turing test. There’s several versions of it, and it’s not a general test of intelligence anyways. It’s a parlor game.
- pixl97 4y ago
  I don't think a single 'what is AGI' answer will satisfy everyone, someone will always have a complaint about something. I personally recommend we make a bunch of different test or 'classifications' that can be measured. Then we test each AI system against these classifications. This isn't to define if its AGI or not, but it's capabilities. For example, if a human took some of these tests they could also fail because the human body/mind is not capable. Just as an example, if we said "Flying is what birds do", then every plane would fail even though the capabilities of planes are far more useful to humans. I see this as a much better framework than the nebulous goal of 'AGI' itself.
v4dok 4y ago
I find this posts being word-noise. We can't agree what is "intelligence" and the goalposts are moving depending on who you ask and when. I think then more interesting question would be "If AGI came, would we know?"
nologic01 4y ago
Its possible that the AGI obsession will fizzle. The analogy is fruitful to some extent as an inspiration. But it is too remote and ambitious to work as a roadmap. There is precedence. People called certain algorithms neural networks, but they are very far from real neurons and they are not getting closer. Why should they? People will keep developing "AI" algorithms but it will be along the natural pathways that underlies these mathematical structures (which are rather simplistic) and focusing on problems that people find important.
tikkun 4y ago
Without looking it up, when you see or hear AGI, what definition do you assume most people to be using?
sashank_1509 4y ago
Even if say we are close to “AGI” it’s going to be something significantly different from human and animal intelligence. One of the most obvious differences that I don’t see people bring up: 1. ChatGPT can only respond to an input. If you left it alone it would literally do nothing. It cannot, generate, create thoughts and choose what stimuli to respond to. This posits that the human brain is like a dynamical system. You switch it on and it keeps going on forever, there is also no hardline between learning and inference like DL. ChatGPT and others etc, feel very much like a digital system. It can only respond to inputs provided, there is a clear demarcation between the learning stage and the inference stage Note: it is still possible that these AI systems will reach human like performance in a variety of tasks. But they will seem very weird and different from the intelligent systems we are exposed to in Nature
slackfan 4y ago
Nah. We haven't built Positronic Brains yet.
passion__desire 4y ago
Wrt "No True Autonomy Fallacy" and the examples provided underneath, I am reminded of this Tuned Deck magic trick example used by Daniel Dennett to explain consciousness. https://jessegalef.com/2011/01/09/the-tuned-deck/ https://jessegalef.com/2011/01/09/the-tuned-deck/ https://www.youtube.com/watch?v=LVg23QRJD70 https://www.youtube.com/watch?v=LVg23QRJD70
logicallee 4y ago
Past it. AGI was publicly unveiled on November 30, 2022 and currently has two hundred million users. It is capable of general abstract thinking on any subject regardless of whether it's already in its training set or not. It meets my definitions of AGI.
ballenf 4y ago
> This captures the crux of the problem in that we claimed we’d have True AI when a computer could beat a human in chess. But when that occurred with Deep Blue, and we saw the code, it was explainable and therefore not true AI. Then we moved the goalpost and said that if a computer could beat a human in Go, then we’d have True AI. When that occurred with AlphaGo and we saw the code, we moved the goalposts yet again. We believed that chess and go proficiency were good proxies for true AI. They weren't. They were proxies for advancement, but the underlying issue is that we don't know how much we don't know about the human mind. Every AI advancement helps us learn more about our minds, but the amount we don't know is still unknown. The moving of the goalposts was not done because "we saw the code" it was because Deep Blue and Alpha Go were obviously not AI. Again, we were just wrong about those tasks being good proxies.
- jackmott42 4y ago
  I'm not sure we were wrong about Go. The general approach that conquered Go just a few years later led to something passing the Turing test. Give it a few more years, see what happens.
- byyyy 4y ago
  No those were never official goal posts. The one goal post was official and famous was the Turing test that measured if a human could tell the difference between a text chat generated by an AI or one generated by a human. https://en.m.wikipedia.org/wiki/Turing_test https://en.m.wikipedia.org/wiki/Turing_test Literally the inventor of the Turing machine came up with this test so this was always the north star goal post since computing began. It's a way higher bar then any of the nameless goal posts you mentioned. Unfortunately this goal post was also just moved recently with LLMs.
  - dS0rrow 4y ago
    I dont think its fair to say that LLMs have passed the Turing test.
    - byyyy 4y ago
retrac 4y ago
We struggle to define the traits, characteristics, properties of intelligence, sentience or awareness. Nor can we locate the physical processes, exactly, by which these things come into being in a human brain. Some voodoo to do with information processing in an embodied system as an agent trying to survive in the world, maybe, if you're a materialist. We know approximately nothing about the configuration that a state of matter would need, to attain intelligent, sentient, aware properties like we have. Other than that something extremely like a human brain (including non-human animal brains) seem to possess these properties in some way. Certain configurations in machines are starting to exhibit the first trait, at least, intelligence. Arguably. I don't think we can even rule out that AGI might, kind of just accidentally happen, in some sufficiently complex self-feedback machine learning-based information-processing task. Since that's one of the main hypotheses for how we came about to think. Just a gut feeling tells me nothing we've built is anywhere near large enough for that. But the supposition that intelligence, sentience, awareness, are tied to big data, big processing, and biological parallelism -- is just that, a supposition. What if the secret of it all boils down to something that can be described mathematically on one page (my other gut feeling), which happens to be physically realized in some way in our heads? Can't be ruled out, either. I can't really see how to set any bounds on anything with these questions. How many bits does it take to hold a mind?
- aanya_dawkins 4y ago
  One of the big issues is that we anthropomorphize those things onto AI, not that AI exhibits it.
  - pixl97 4y ago
    We trained AI on the sum of human knowledge so there is going to be some implicit anthropomorphization in the model itself.
aanya_dawkins 4y ago
[dead]
Workaccount2 4y ago
Mass media has so heavily conveyed the fact that humans are transcendent of the laws of physics, the weeping tear that brings back the dying or the true love bond that defeats the machines. This trope is so tired, but man do our narcissistic meat brains love it. AGI or not, I think people conditioned their whole lives that humans are transcendent are starting to get smashed through the wringer of reality, making it hard to discern honest takes from desperate coping takes.
Ekaros 4y ago
What are we meaning by AGI? Something that does same job as current "AIs" but for well everything? Maybe. For actual self-"aware" or thinking system, probably not.
GreedClarifies 4y ago
Preface, most predictions are worthless. 3 years ago would the author have predicted a chat bot that could understand and write language fluently? Ha. I doubt it. My prediction, FWIW (not much): we are very close to AGI (for some definitions of AGI).
mikewarot 4y ago
My current model of AI is that we're applying massive amounts of compute to force a function approximator to learn a function. To solve this function, there is likely a huge mismatch between the learned results and the internal logic used to generate those results. I believe that it is possible that the mismatch is several orders of magnitude, and thus it might be that the cognitive core could be quite a superintelligence hobbled by the layers around it. It is quite possible that there are cognitive strategies at use that haven't been invented yet in the outside world. At this point, it would require better training data, or even more compute, to let that intelligence reach the outside world unfiltered. Only time will tell if any of this is true, it's just speculation on my part at present.
opyate 4y ago
If you observe toddlers, at the early stages of their development you can see the "inner voice" at play, their lips moving as they reason through scenarios whilst playing with toys (and deaf/mute children similarly signing with their hands under the table). The same work is being done now with LLMs, where you can prompt-engineer models to reason about its generated text. See https://youtu.be/wVzuvf9D9BU https://youtu.be/wVzuvf9D9BU This video captures some of the state-of-the-art currently. (E.g. see the video description links) So, decoders are getting their own little inner voices. A small little step towards AGI, for sure.

31 points by mpwouden 4y ago

How close are we to AGI and do people actually understand what that means? Too often people use the term as if General were 'ubiquitious' as in a broad application of AI instead of one that is inditinguishable from human intelligence.

ftxbro 4y ago
> do people actually understand what [AGI] means no because it means something different to each person (both its definition and its implications)
- greyman 4y ago
  Exactly. For example John Carmack understands AGI as an simulation of 2-year old human brain, which we can teach anything. That's quite a good definition imo, but it seems to me this isn't achievable by evolution of LLMs. Other people thinks that by making current LLMs approach somewhat better, that could be called AGI. To sum it up, AGI isn't anything specific at this point.
knallfrosch 4y ago
As if we could agree on what 'human intelligence' is (or indeed, how to measure it.)
ben_w 4y ago
We definitely don't have consensus about either "general" or "intelligence" (and if mind-uploads ever enter the discussion, then also about "artificial"). For me, "general" is a sliding scale, not a yes-or-no boolean, so I think it's fine to say that GPT3, let alone later versions, is "general" in the domain of text — it has more general knowledge than I have, that's for sure! Likewise "intelligent", though for me that's a vector value: [i_0, i_1, …, i_m] where each i is the ability at some aspect of intelligence. In fact, I'd go further and say that you can define "generality" with respect to such a vector: if the intelligence vector is a normalised score 0-1 for each value, then generality could be easily defined by the statistics of that array. As even GPT-4 seems to be better than the layperson and worse than an expert at everything (textual) it has been tested on, I'm not sure if it's more or less general than a human by this standard. And you may wish to define intelligence by ability to learn from small data sets rather than large, at which point we then need to consider the difference between the initial training, the RLHF, and any fine-tuning; and how this compares to evolution, education, and work experience.

md_ 4y ago

Seems like a long-winded way of saying, "no". ;)

jackmott42 4y ago
It is telling though, that a year ago myself and I think most programmers would have just quickly said "nope definitely not", whereas now you kind of have to get long winded to say no, and many of us are not saying no but maybe AI has progressed in fits and starts. We could be one more breakthrough away from AGI, maybe many breakthroughs are needed. Nobody knows for sure, or how quickly those will come. I think things could get wild really quicky, given how effective a really primitive "Guess the next word" neural network is by just throwing a ton of data at it. What will we get with more nuanced ideas and multiple networks wired together with feedback loops and such? Gonna be fun.
- uoaei 4y ago
  > many of us are not saying no but maybe I'm not sure if this is a controversial stance, but not all opinions are equal. HN commenters span a wide range of competencies, but very few of them have been in the trenches with ML (not just using, but designing and building the actual architectures, working on theory, or writing cogent philosophy about it) long enough to have good opinions about it worth listening to.
  - UIUC_06 4y ago
    Gene Wilder meme: Tell me all about how your opinions are better than everyone else's!
    - uoaei 4y ago
      You will note I did not proffer any of my own. To avoid Dunning-Kruger, you must know what things you don't have sufficient knowledge about and avoid speaking as if you do. Also please don't "meme" on HN, it degrades discussion quality very quickly. There are other venues for that.

insane_dreamer 4y ago

redeyedtreefrog 4y ago
Betteridge's law of headlines

davewritescode 4y ago

We're not close and it probably doesn't really matter. I mean, yes I want a real world Jarvis but that's not really what's important. What's important is that AI will quickly replace toil in white collar work. Unfortunately, a lot of people make entire careers of that kind of work. "Paper pusher" type jobs are the most at risk but so are the low end of lots of creative jobs like copywriting, graphic design and even script writing. We're going to have to reckon with the consequences of AI long before AGI becomes real.

kevingadd 4y ago
I won't soon forget how as a tools programmer at a game studio, I noticed a couple of the writers spent a big portion of their day churning text from the design team's word docs through excel macros to generate CSV files consumed by our build system. I looked at that and went "oh, I can just automate this", and replaced all that busywork with a tiny C# program that would just do it correctly all the time automatically so nobody had to do it by hand anymore. Only occurred to me much later that now they didn't have much work to do and were at a higher risk of being laid off! A good lesson to think about all the implications of the software I'm writing. We've seen AI-adjacent tooling getting leveraged to eliminate more of that category of work in games already, like automated generation of NPC "barks".
- chii 4y ago
  > A good lesson to think about all the implications of the software I'm writing. only in so far as not reaping the rewards for having done more work than the company paid you for. Efficiency improvements are always good for society as a whole, despite it hurting some people who originally was paid due to the inefficiency.
  - mschuster91 4y ago
    > Efficiency improvements are always good for society as a whole, despite it hurting some people who originally was paid due to the inefficiency. When the economic/capital/wealth 1% end up hoarding all the profitability gains, society as a whole loses. I have come to loathe the term "disruption" as it has always meant externalizing costs to make a quick buck: either by technology replacing actual, real humans with real families who now have to struggle to survive or by breaking laws and regulations like Uber and AirBnB did, with society paying the costs associated with that - in the case of Uber, society picked up the tab for underinsured drivers or for their pension/social security contributions which don't apply for "freelancers", and in the case of AirBnB all the neighbors affected by the noise and literal other disruptions an AirBnB brings with it.

rcme 4y ago

The author's definition of "strong AI" is incredibly weak. > Theory of Mind: These AI systems are capable of understanding mental states and can simulate the behavior of other agents. GPT models can do this. > Self-Aware Why is that a requirement? > These AI systems are capable of learning any intellectual task that a human can. Again, GPT models appear to do this (at least for text-based tasks). That being said, I think GPT models do have some limitations that prevent them for learning any intellectual task a human can do. I wrote a blog post a few days ago that explored trying to "teach" ChatGPT to perform binary addition [0]. My conclusion was that, when ChatGPT appears to solve a logical problem, that solution is just a regurgitation of a low-entropy example generalized from its training data. But the model fails to apply even basic logical rules as entropy increases, indicating that GPT models have a fundamental limit of applying logic to high entropy problems. 0: https://nobuteru.substack.com/p/can-chatgpt-learn-to-add https://nobuteru.substack.com/p/can-chatgpt-learn-to-add

shagie 4y ago
On the adding part - have you looked at how it tokenizes that string? https://platform.openai.com/tokenizer https://platform.openai.com/tokenizer 10011111110100011100011011110010 00111010100101000100000101111011 is tokenized as: [100][1111][11][101][0001][11][0001][101][11][100][10] [00][11][101][01][001][01][0001][0000][01][01][11][101][1] It might be more trainable if the the bits where space delimited so that only the tokens [ 1] and [ 0] were in use. As it is, it doesn't really understand the contents of the tokens and manipulating [405, 1157, 8784, 486, 8298, 486, 18005, 2388, 486, 486, 1157, 8784, 16] is not easily trained - especially if there is a different bit sequence. Note also the misalignments of the tokens for bit length that can make it more difficult to work with.
- rcme 4y ago
  I disagree that tokenization is important to consider. But just to test it out, I tried adding spaces between each bit and verified that each bit is tokenized separately: Let's add the two binary numbers provided, limiting the work on each bit to a single line: 1 0 0 1 1 1 1 1 1 1 0 1 0 0 0 1 1 1 0 0 0 1 1 0 1 1 1 1 0 0 1 0 + 0 0 1 1 1 0 1 0 1 0 0 1 0 1 0 0 0 1 0 0 0 0 0 1 0 1 1 1 1 0 1 1 -------------------------------------------------------------- 1 0 1 1 0 1 1 0 1 0 0 1 0 0 0 1 0 1 1 1 1 1 0 0 0 1 1 1 1 1 0 1 So, the result of the addition is the binary number 10110110101001010111111001111101. The answer is still wrong. Think about it like this: even when tokenized into 1-5 digit binary chunks, there are still just a few tokens to consider. If ChatGPT could actually apply the rules of addition to a general, high-entropy example, it would be trivial to use these tokens to derive the correct answer. A human could easily do it.

cat_plus_plus 4y ago

Why do we need to be? Say human intelligence evolved so that we model real world events such a mammoth hunt mentally ahead of time and use simulated outcomes to craft weapons / preposition different hunters based on their skills. Would AI necessarily need to hunt or worry about energy intake/self preservation/producing offspring? It can be powered by direct photosynthesis (solar panels), hibernate when energy is in short supply and make backups to safeguard against future physical damage. Seems like we should think through the tasks that we need AI to help us with - climate change, medicine, space exploration, etc - and design systems for such tasks as efficiently as possible rather than trying to monkey ourselves. And sure, some day AI may develop its own purposes that are different from its origin, just like humans decided they like music. Doesn't mean that a caveman shouldn't have dedicated his attention to the next meal rather than worrying about being sophisticated.

bbor 4y ago
The simple answer is “go figure out how solve climate change” is a much easier direction to give an AI than figuring out what to do then building a bunch of little specific ones. Really all an AGI is an AI smart enough to build and use other smaller AIs, or at least that’s a very common guess as to the architecture that’ll end up working. I give it till Christmas

deleted 4y ago

[deleted]

williamcotton 4y ago

In practice, treating GPT as a weak AI and part of a larger system design is how to gain the most use from the tool. Also, I fully agree that cybernetics, human systems interactions, neuroscience and even epistemology are useful disciplines for implementing intelligent autonomous systems. I don’t have much more to add other than I like this article and the author’s point of view.

rvz 4y ago

No.

agentultra 4y ago

> AI is the development of machines with the ability to think and act like humans What we have today is machine learning... not this. LLM's are neat, transformers are a really useful innovation, etc. Some people like to leap to conclusions and claim that this is intelligence beyond human comprehension and it will learn to destroy is all. I think that's a bit much. If the goal of AGI is to be this we're nowhere near it. And I suspect there are plenty of people in this "AI" space that would not see it as a goal of the research. An expert system that can reason in a particular domain would be a huge advance and a useful tool. There's no need to also create autonomous agents that think and act like humans. We're pretty terrible at that on the whole.

titzer 4y ago

It seems like an AGI would just be a meta-machine-learning model, meaning it's a model whose whole goal is to think up new ways of training models. Eventually, it would figure out a way to either incorporate the smaller, specialized subtasks into itself and piecemeal upgrade itself. Oh, and it needs permission to use $MEGA_DOLLARS to run the computations necessary for training. So, it would probably need a side hussle. Like search ads or something. Right now, humans are either the bailing wire or the chewing gum in this situation, but they hold the purse strings and GPU farms. Humans are the ones thinking up new training strategies, curating the learning datasets, and building the cluster and cloud and datacenter systems that can handle that much data.

toteno 4y ago

I noticed that some people expect AGI to have some superhuman intelligence instead of just intelligence similar to that of humans. I have seen some posts arguing that an AI can only be called AGI if it is better than any human being on every task. It's nice to see that this article doesn't fall into that.

deleted 4y ago
[deleted]

throwaway22032 4y ago

Reasoning about whether we are close to solving something that is essentially a research problem is erroneous thinking to me. We could quite concievably be one or two discoveries away from improving the scaling of our current models by 10-1000x. But I don't think there is a way of determining progress on that research. It's an unknown unknown.

bbor 4y ago
But worth trying to guess! I think the only other project similar in character would be nuclear weapons, and I definitely think the world would’ve appreciated a heads up there. Just because it’s hard to predict doesn’t mean we shouldn’t reason about it and get some ballpark estimates, IMO
- throwaway22032 4y ago
  Yes. I think on AI research in particular though, we could show that we're close, but not convincingly that we are far away. Not in the current world, anyway. With 1950s levels of compute, sure.

mindcrime 4y ago

I don't know, but I'd posit that anybody who answers "definitely yes" or "definitely no" doesn't know what they are talking about. And that's because we, collectively, really have no way of knowing how close to AGI we are. We struggle enough to define and measure human intelligence, and adding what is arguably a new "kind" of intelligence to the pot just complicates matters even more. So as of today, we don't really have any kind of simple metric like "IQ" that we can apply to AI systems, and even if we did have the metric, at this point we wouldn't really know it's range, whether to expect it to be a linear function or not, etc. That said... does it seem clear that we are making some kind of progress? I'd say yes, but with the admission that that's somewhat subjective. In particular, even though we know that current AI systems are getting better at doing whatever they do, we don't really know if that meaningfully counts as progress towards AGI. It is possible, for example, that LLM approaches will hit a ceiling in their development, and that the much feared "GPT 17" really won't even be a thing. And it may likewise turn out that the progress made on LLM's doesn't turn out to contribute much at all to whatever the eventual mechanism behind AGI turns out to be. Is that the way I expect things to play out? Not sure. I have a hunch that LLM's will prove to be insufficient to qualify as AGI on their own, but that they may well serve as an element of a modular / hybrid system that includes LLMs, reasoners using deduction, induction, and abduction, evolutionary algorithms, and FSM-knows what else.

reitanqild 4y ago
Related (and since you qualify the answers as "definitely yes" or "definitely no" I don't think it subtracts from what you say, but rather just adds to it: > When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong. (Clarkes first law, copied from https://en.wikipedia.org/wiki/Clarke%27s_three_laws https://en.wikipedia.org/wiki/Clarke%27s_three_laws)
- mindcrime 4y ago
  > When a distinguished but elderly scientist states that something is possible, he is almost certainly right. When he states that something is impossible, he is very probably wrong. Agreed!

vsareto 4y ago

Virtual personal assistant isn't exactly a narrow definition. I'd expect that's a big role to play and we'll make AGI systems do that too. The strong AI definitions are also pretty biased towards human experiences. I wouldn't assume future innovations are necessarily human-like. It's hard to imagine a completely new thing until it gets here though. "Learning any task" is also, uh, quite expansive - does the AI lose its AGI badge if the internet finds something it sucks at? (spoiler: it will suck at something)

janalsncm 4y ago

What I would’ve preferred in this article which is largely long winded definitions and summaries of computing history that are much better explained on Wikipedia would be a workable, testable definition of AGI. How will we know when we’ve gotten there? Frankly, I don’t care about the Turing test. There’s several versions of it, and it’s not a general test of intelligence anyways. It’s a parlor game.

pixl97 4y ago
I don't think a single 'what is AGI' answer will satisfy everyone, someone will always have a complaint about something. I personally recommend we make a bunch of different test or 'classifications' that can be measured. Then we test each AI system against these classifications. This isn't to define if its AGI or not, but it's capabilities. For example, if a human took some of these tests they could also fail because the human body/mind is not capable. Just as an example, if we said "Flying is what birds do", then every plane would fail even though the capabilities of planes are far more useful to humans. I see this as a much better framework than the nebulous goal of 'AGI' itself.

v4dok 4y ago

I find this posts being word-noise. We can't agree what is "intelligence" and the goalposts are moving depending on who you ask and when. I think then more interesting question would be "If AGI came, would we know?"

nologic01 4y ago

Its possible that the AGI obsession will fizzle. The analogy is fruitful to some extent as an inspiration. But it is too remote and ambitious to work as a roadmap. There is precedence. People called certain algorithms neural networks, but they are very far from real neurons and they are not getting closer. Why should they? People will keep developing "AI" algorithms but it will be along the natural pathways that underlies these mathematical structures (which are rather simplistic) and focusing on problems that people find important.

tikkun 4y ago

Without looking it up, when you see or hear AGI, what definition do you assume most people to be using?

sashank_1509 4y ago

Even if say we are close to “AGI” it’s going to be something significantly different from human and animal intelligence. One of the most obvious differences that I don’t see people bring up: 1. ChatGPT can only respond to an input. If you left it alone it would literally do nothing. It cannot, generate, create thoughts and choose what stimuli to respond to. This posits that the human brain is like a dynamical system. You switch it on and it keeps going on forever, there is also no hardline between learning and inference like DL. ChatGPT and others etc, feel very much like a digital system. It can only respond to inputs provided, there is a clear demarcation between the learning stage and the inference stage Note: it is still possible that these AI systems will reach human like performance in a variety of tasks. But they will seem very weird and different from the intelligent systems we are exposed to in Nature

slackfan 4y ago

Nah. We haven't built Positronic Brains yet.

passion__desire 4y ago

Wrt "No True Autonomy Fallacy" and the examples provided underneath, I am reminded of this Tuned Deck magic trick example used by Daniel Dennett to explain consciousness. https://jessegalef.com/2011/01/09/the-tuned-deck/ https://jessegalef.com/2011/01/09/the-tuned-deck/ https://www.youtube.com/watch?v=LVg23QRJD70 https://www.youtube.com/watch?v=LVg23QRJD70

logicallee 4y ago

Past it. AGI was publicly unveiled on November 30, 2022 and currently has two hundred million users. It is capable of general abstract thinking on any subject regardless of whether it's already in its training set or not. It meets my definitions of AGI.

ballenf 4y ago

> This captures the crux of the problem in that we claimed we’d have True AI when a computer could beat a human in chess. But when that occurred with Deep Blue, and we saw the code, it was explainable and therefore not true AI. Then we moved the goalpost and said that if a computer could beat a human in Go, then we’d have True AI. When that occurred with AlphaGo and we saw the code, we moved the goalposts yet again. We believed that chess and go proficiency were good proxies for true AI. They weren't. They were proxies for advancement, but the underlying issue is that we don't know how much we don't know about the human mind. Every AI advancement helps us learn more about our minds, but the amount we don't know is still unknown. The moving of the goalposts was not done because "we saw the code" it was because Deep Blue and Alpha Go were obviously not AI. Again, we were just wrong about those tasks being good proxies.

jackmott42 4y ago
I'm not sure we were wrong about Go. The general approach that conquered Go just a few years later led to something passing the Turing test. Give it a few more years, see what happens.
byyyy 4y ago
No those were never official goal posts. The one goal post was official and famous was the Turing test that measured if a human could tell the difference between a text chat generated by an AI or one generated by a human. https://en.m.wikipedia.org/wiki/Turing_test https://en.m.wikipedia.org/wiki/Turing_test Literally the inventor of the Turing machine came up with this test so this was always the north star goal post since computing began. It's a way higher bar then any of the nameless goal posts you mentioned. Unfortunately this goal post was also just moved recently with LLMs.
- dS0rrow 4y ago
  I dont think its fair to say that LLMs have passed the Turing test.
  - byyyy 4y ago

retrac 4y ago

We struggle to define the traits, characteristics, properties of intelligence, sentience or awareness. Nor can we locate the physical processes, exactly, by which these things come into being in a human brain. Some voodoo to do with information processing in an embodied system as an agent trying to survive in the world, maybe, if you're a materialist. We know approximately nothing about the configuration that a state of matter would need, to attain intelligent, sentient, aware properties like we have. Other than that something extremely like a human brain (including non-human animal brains) seem to possess these properties in some way. Certain configurations in machines are starting to exhibit the first trait, at least, intelligence. Arguably. I don't think we can even rule out that AGI might, kind of just accidentally happen, in some sufficiently complex self-feedback machine learning-based information-processing task. Since that's one of the main hypotheses for how we came about to think. Just a gut feeling tells me nothing we've built is anywhere near large enough for that. But the supposition that intelligence, sentience, awareness, are tied to big data, big processing, and biological parallelism -- is just that, a supposition. What if the secret of it all boils down to something that can be described mathematically on one page (my other gut feeling), which happens to be physically realized in some way in our heads? Can't be ruled out, either. I can't really see how to set any bounds on anything with these questions. How many bits does it take to hold a mind?

aanya_dawkins 4y ago
One of the big issues is that we anthropomorphize those things onto AI, not that AI exhibits it.
- pixl97 4y ago
  We trained AI on the sum of human knowledge so there is going to be some implicit anthropomorphization in the model itself.

aanya_dawkins 4y ago

[dead]

Workaccount2 4y ago

Mass media has so heavily conveyed the fact that humans are transcendent of the laws of physics, the weeping tear that brings back the dying or the true love bond that defeats the machines. This trope is so tired, but man do our narcissistic meat brains love it. AGI or not, I think people conditioned their whole lives that humans are transcendent are starting to get smashed through the wringer of reality, making it hard to discern honest takes from desperate coping takes.

Ekaros 4y ago

What are we meaning by AGI? Something that does same job as current "AIs" but for well everything? Maybe. For actual self-"aware" or thinking system, probably not.

GreedClarifies 4y ago

Preface, most predictions are worthless. 3 years ago would the author have predicted a chat bot that could understand and write language fluently? Ha. I doubt it. My prediction, FWIW (not much): we are very close to AGI (for some definitions of AGI).

mikewarot 4y ago

My current model of AI is that we're applying massive amounts of compute to force a function approximator to learn a function. To solve this function, there is likely a huge mismatch between the learned results and the internal logic used to generate those results. I believe that it is possible that the mismatch is several orders of magnitude, and thus it might be that the cognitive core could be quite a superintelligence hobbled by the layers around it. It is quite possible that there are cognitive strategies at use that haven't been invented yet in the outside world. At this point, it would require better training data, or even more compute, to let that intelligence reach the outside world unfiltered. Only time will tell if any of this is true, it's just speculation on my part at present.

opyate 4y ago

If you observe toddlers, at the early stages of their development you can see the "inner voice" at play, their lips moving as they reason through scenarios whilst playing with toys (and deaf/mute children similarly signing with their hands under the table). The same work is being done now with LLMs, where you can prompt-engineer models to reason about its generated text. See https://youtu.be/wVzuvf9D9BU https://youtu.be/wVzuvf9D9BU This video captures some of the state-of-the-art currently. (E.g. see the video description links) So, decoders are getting their own little inner voices. A small little step towards AGI, for sure.