8 ms·

I don't understand how most of the comments here seem to be along the lines of "these interview questions are useless now" or "we need to rethink education, it

I don't understand how most of the comments here seem to be along the lines of "these interview questions are useless now" or "we need to rethink education, it hasn't kept up". These all seem absurdly myopic to me. What we're seeing is the first instance, still very limited and imperfect, of AGI. This is not going to make some interview questions obsolete, or give students more tools to cheat with their homework. It is effectively proving that acquiring knowledge and professional skills is becoming useless, for good. In a few years (3, 5, 10 at most) this is going to defeat the entire purpose of hiring, and therefore of completing most forms of professional education. At the current rate of progress, most intellectual professions will be obsolete before today's sixth-graders are ready to enter the job market. I can't even picture a functional world where humans are cut out of most professions that don't involve manual work; where any amount of acquired knowledge and skills will be surpassed by machines that can produce better results at a thousandth of the cost in a thousandth of the time a human can. And even if such a world can function, I can't imagine a smooth transition to that world from its current state. I'm worried, does it show? :)

haolez 4y ago
I don't know if this is an AGI-like experiment, because LLM are trained on human knowledge. I'd expect that real AGI wouldn't need such a thing and would improve on its own. That's the moment where we become obsolete.
- hombre_fatal 4y ago
  "you can invent human cognition from first principles via billion years of parallel evolution. or since it’s already been invented, applied, recorded at scale you can just observe its behavior to learn it"
- pixl97 4y ago
  Mammals did not evolve from single cell creatures in order to replace the dinosaurs.
kurisufag 4y ago
>What we're seeing is the first instance, still very limited and imperfect, of AGI absolutely not. language-model text generation is actually about as non-general as it gets -- they are fundamentally incapable of understanding anything at all, ever. they can't do math, work through basic logic problems, or produce any output that isn't just an assumed logical continuation of the input.
- throw310822 4y ago
  I don't agree. Besides the fact that I can't do much math either, except with a lot of effort and tools, at this point there are masses of examples of GPTs doing basic reasoning and logic and solving problems. Of course you can say that they do it by "finding the most probable continuation"- and you're right. But that doesn't change the fact that it works. Simply because to find the most probable continuation ultimately you need semantic understanding, and somewhere, back-propagating from the training text, the NNs have managed to build a decent model of the world. There's no other way to explain their performance, these are not markov chain text generators.
- adastra22 4y ago
  Have you actually used any of these products? GPT et al are perfectly capable of taking knowledge from any one domain and applying it towards the solution of any other problem domain, through various kinds of data abstraction, reasoning by analogy, and other techniques similar to what humans do. It makes plenty of goofups along the way, just like humans do. But if your requirement is that it performs absolutely perfectly in new domains without making any mistake or needing any supervision... well that is certainly a requirement no human could ever meet, either.
  - RhysU 4y ago
    To be completely ridiculous, we'll have AGI when it can convince a human to mate with it over any other human followed by raising its offspring to a successful adulthood. This is the game we're all playing.
    - 4y ago
sanderjd 4y ago
This all seems very unlikely to me. Better tools have always made humans more useful, just doing different things. I do think it's interesting to consider whether there is some singularity where that trend dramatically reverses (Vonnegut's Player Piano is one of my favorite books...), but I think the better prior is that this is just another step up the abstraction ladder for humanity.
krembanan 4y ago
Software development isn't really about just acquiring and memorising knowledge to later spit out. You need reasoning, logic and creativity. That AGI can develop complex new software (not just some code that has been scraped of the internet years ago) in 10 years is something I doubt.
- deleted 4y ago
  [deleted]
- grugagag 4y ago
  We don’t have AGI as of now but it could spark anytime and its acceleration could be extremely fast. It could almost spark and yet never really get there, just get closer and closer and quite never hit the mark. But this is not something that will simply stop a massive disruption in most professions and livelihoods, LLMs could do that easily
jacobjjacob 4y ago
I think what an LLM is best at is fooling people into thinking it’s intelligent. It is really good at saying things in a natural sounding way, and statistically often getting it right, because certain strings of tokens are encoded. But it’s clear when you start poking that it just as easily tells you that 5/2=3, or that 2+2 != 4. it doesn’t model math or any sort of knowledge at all.
- babyshake 4y ago
  Something that I don't quite understand is why the tendency of ChatGPT to be inaccurate sometimes is a fundamental flaw rather than something that can be improved on iteratively if its just a matter of improving the statistical likelihood of accuracy. The question of whether its AGI or not is, to paraphrase the famous quote, a bit like the question of whether a submarine can swim.
  - JohnFen 4y ago
    Because ChatGPT isn't thinking. It's not reasoning at all. It's assembling sentences that are statistically predicted from using existing writings as the template. Accuracy isn't a part of the process except in terms of how accurate the training data is. ChatGPT is not making any sort of truth or accuracy determination, let alone doing so poorly.
    - zeckalpha 4y ago
      Can you point to where the thinking happens in a human?
      - JohnFen 4y ago
        Nope. But I don't have to do that to understand that LLMs do not assess truth or accuracy of anything.
        jstanley 4y ago
        But you don't know that humans don't reason by stringing words together and seeing how statistically likely they seem. Related to this topic, see "Babble and Prune": https://www.lesswrong.com/s/pC6DYFLPMTCbEwH8W https://www.lesswrong.com/s/pC6DYFLPMTCbEwH8W
stuckinhell 4y ago
I agree 1000000%. Watching all the ART AI's change so radically fast, Watching ChatGPT being adopted by so many people at my company. Watching people say art will shift to prompt engineering, then watching prompt engineering being automated away already! Watching the rise of stable diffusion tooling like invokeai, controlnet, and more. Watching youtube videos of people integrating openai and stable diffusion together. Hearing my old university has created an emergency council to deal with AI and its implications on education and students. Seeing artists on twitter say their commissions have dropped dramatically. Hearing stories that 4channers have been trying to steal Anime creation AI's. Reading articles that chatgpt passed Wharton MBA tests, and various Lawyer tests. An article on hackernews TODAY talks about running this giant models on a home pc! The speed of AI adoption with its immediate practical usefulness and watching the the speed of innovation around only increase faster than ever makes me nervous extremely nervous about the future, extremely nervous. This is a global paradigm shift.
- grugagag 4y ago
  It is a paradigm shift and whoever happens to be on the good side of this may survive just a bit longer before they too become redundant.
  - analog31 4y ago
    Do we know if anything, other than luck, will determine who's on which side? Or, if any decisive action on anybody's part is more likely to pay off than just staying mentally flexible, watching, and waiting?
comment_ran 4y ago
It's a good idea to think about some aspects of humans. Most people never be content with what they have. Once progress was made, we get used to the old tool and begin to build new norms. The pain is always there, you just run faster.
JohnFen 4y ago
> What we're seeing is the first instance, still very limited and imperfect, of AGI. None of this is AGI. This is Eliza on steroids.
- sacrosancty 4y ago
  [dead]
- barking_biscuit 4y ago
  It seems disingenuous to say that this is Eliza on steroids. They operate in fundamentally different ways, no? I definitely agree with the premise that the current state of the art isn't at all AGI, but it seems almost self-evident to me that LLMs are a key piece of the puzzle on the road to AGI. Eliza was never going to have that kind of trajectory, but LLMs I think you could certainly make that argument for.
Aperocky 4y ago
This take is widely prevalent in every single ChatGPT discussion. I'll quote myself for discussion for a previous article: https://news.ycombinator.com/item?id=34746348 https://news.ycombinator.com/item?id=34746348 > I've find two groups of people on this subject, first one has taken a graduate level stat course and/or ML course, and may have work experience in machine learning/data science. The latter camp is more numerous and did not do those things. > The latter group are far more hyped about ChatGPT et al, despite explanation by the first group. > Don't get me wrong, ChatGPT is very exciting, just not in the way it is frequently portrait to be, in particular, development on this model will not lead to General Intelligence, which is not a data training/stat problem altogether. With how the ML field is shaping up to be today, it doesn't even look to be a Machine Learning problem. And let me just add a ridiculous comparison, since you are assuming that it is a path towards AGI: > iPhone 4 isn't fusion energy. It's incredibly useful and a tectonic shift in the cellphone industry, but it's not producing energy. The analogy might sound completely insane, but that's how different machine learning of today and general intelligence is. While AI of today feels like a blackbox, at every stage of its training we know to a degree what it is doing, and once trained, it is as deterministic as a few billion interconnected if else connected to a random seed generator. It could do a lot of things, acquiring a mind of its own isn't one of it. I wouldn't be worried just yet, because AGI is something that research have barely had an idea on how it's going to look outside of our brain, or even inside. I'm not saying that it will never come, just that I highly doubt it will be invented in my lifetime. In the meanwhile, I have something that all of bots today won't have, initiative.
- circuit10 4y ago
  But from what I’ve seen many skilled AI researchers who have even reimplemented these models themselves do in fact believe that a scaled up ChatGPT could be AGI, or at least that it’s a step towards AGI
  - Aperocky 4y ago
    > many skilled citation needed, I'm only aware of one case which might be characterized as such (though I digress): https://www.engadget.com/blake-lemoide-fired-google-lamda-sentient-001746197.html https://www.engadget.com/blake-lemoide-fired-google-lamda-se...
EamonnMR 4y ago
Butlerian Jihad when?
cmrdporcupine 4y ago
I'm just going to quit my programming job and take up flint knapping, make some quality stone tools -- that's what homo genus did for 2 million years before computers, I figure I can always fall back on that.
breuleux 4y ago
> I can't even picture a functional world where humans are cut out of most professions that don't involve manual work Many people do manual work. It may very well be the case that general fine motor skills are a far more complex and difficult operation than the entire edifice of human intellect. Philosophically, it would be an immense blow, the mother of all existential crises. But regardless, it suggests that the first AGI would be incapable of independent survival and we'd still be relevant and in control for a while. > where any amount of acquired knowledge and skills will be surpassed by machines that can produce better results at a thousandth of the cost in a thousandth of the time a human can. I don't think we can necessarily extrapolate it to be that cheap. It could be. But it is also possible that the increases in cost and resources to scale these models bigger and bigger will outstrip hardware progress and that this technology will run into a dead end. To put it differently, I think it is far from clear that the kind of hardware that we build is actually better than an organic substrate for this sort of computation. Imagine an optimized organic neural implementation of ChatGPT, for instance. Would it be slower or more expensive than ChatGPT? Perhaps not. Likewise, the very best that the current paradigm can offer may not be faster or cheaper than humans are at quite many valuable tasks.

haolez 4y ago

I don't know if this is an AGI-like experiment, because LLM are trained on human knowledge. I'd expect that real AGI wouldn't need such a thing and would improve on its own. That's the moment where we become obsolete.

hombre_fatal 4y ago
"you can invent human cognition from first principles via billion years of parallel evolution. or since it’s already been invented, applied, recorded at scale you can just observe its behavior to learn it"
pixl97 4y ago
Mammals did not evolve from single cell creatures in order to replace the dinosaurs.

kurisufag 4y ago

>What we're seeing is the first instance, still very limited and imperfect, of AGI absolutely not. language-model text generation is actually about as non-general as it gets -- they are fundamentally incapable of understanding anything at all, ever. they can't do math, work through basic logic problems, or produce any output that isn't just an assumed logical continuation of the input.

throw310822 4y ago
I don't agree. Besides the fact that I can't do much math either, except with a lot of effort and tools, at this point there are masses of examples of GPTs doing basic reasoning and logic and solving problems. Of course you can say that they do it by "finding the most probable continuation"- and you're right. But that doesn't change the fact that it works. Simply because to find the most probable continuation ultimately you need semantic understanding, and somewhere, back-propagating from the training text, the NNs have managed to build a decent model of the world. There's no other way to explain their performance, these are not markov chain text generators.
adastra22 4y ago
Have you actually used any of these products? GPT et al are perfectly capable of taking knowledge from any one domain and applying it towards the solution of any other problem domain, through various kinds of data abstraction, reasoning by analogy, and other techniques similar to what humans do. It makes plenty of goofups along the way, just like humans do. But if your requirement is that it performs absolutely perfectly in new domains without making any mistake or needing any supervision... well that is certainly a requirement no human could ever meet, either.
- RhysU 4y ago
  To be completely ridiculous, we'll have AGI when it can convince a human to mate with it over any other human followed by raising its offspring to a successful adulthood. This is the game we're all playing.
  - 4y ago

sanderjd 4y ago

This all seems very unlikely to me. Better tools have always made humans more useful, just doing different things. I do think it's interesting to consider whether there is some singularity where that trend dramatically reverses (Vonnegut's Player Piano is one of my favorite books...), but I think the better prior is that this is just another step up the abstraction ladder for humanity.

krembanan 4y ago

Software development isn't really about just acquiring and memorising knowledge to later spit out. You need reasoning, logic and creativity. That AGI can develop complex new software (not just some code that has been scraped of the internet years ago) in 10 years is something I doubt.

deleted 4y ago
[deleted]
grugagag 4y ago
We don’t have AGI as of now but it could spark anytime and its acceleration could be extremely fast. It could almost spark and yet never really get there, just get closer and closer and quite never hit the mark. But this is not something that will simply stop a massive disruption in most professions and livelihoods, LLMs could do that easily

jacobjjacob 4y ago

I think what an LLM is best at is fooling people into thinking it’s intelligent. It is really good at saying things in a natural sounding way, and statistically often getting it right, because certain strings of tokens are encoded. But it’s clear when you start poking that it just as easily tells you that 5/2=3, or that 2+2 != 4. it doesn’t model math or any sort of knowledge at all.

babyshake 4y ago
Something that I don't quite understand is why the tendency of ChatGPT to be inaccurate sometimes is a fundamental flaw rather than something that can be improved on iteratively if its just a matter of improving the statistical likelihood of accuracy. The question of whether its AGI or not is, to paraphrase the famous quote, a bit like the question of whether a submarine can swim.
- JohnFen 4y ago
  Because ChatGPT isn't thinking. It's not reasoning at all. It's assembling sentences that are statistically predicted from using existing writings as the template. Accuracy isn't a part of the process except in terms of how accurate the training data is. ChatGPT is not making any sort of truth or accuracy determination, let alone doing so poorly.
  - zeckalpha 4y ago
    Can you point to where the thinking happens in a human?
    - JohnFen 4y ago
      Nope. But I don't have to do that to understand that LLMs do not assess truth or accuracy of anything.
      - jstanley 4y ago
        But you don't know that humans don't reason by stringing words together and seeing how statistically likely they seem. Related to this topic, see "Babble and Prune": https://www.lesswrong.com/s/pC6DYFLPMTCbEwH8W https://www.lesswrong.com/s/pC6DYFLPMTCbEwH8W

stuckinhell 4y ago

I agree 1000000%. Watching all the ART AI's change so radically fast, Watching ChatGPT being adopted by so many people at my company. Watching people say art will shift to prompt engineering, then watching prompt engineering being automated away already! Watching the rise of stable diffusion tooling like invokeai, controlnet, and more. Watching youtube videos of people integrating openai and stable diffusion together. Hearing my old university has created an emergency council to deal with AI and its implications on education and students. Seeing artists on twitter say their commissions have dropped dramatically. Hearing stories that 4channers have been trying to steal Anime creation AI's. Reading articles that chatgpt passed Wharton MBA tests, and various Lawyer tests. An article on hackernews TODAY talks about running this giant models on a home pc! The speed of AI adoption with its immediate practical usefulness and watching the the speed of innovation around only increase faster than ever makes me nervous extremely nervous about the future, extremely nervous. This is a global paradigm shift.

grugagag 4y ago
It is a paradigm shift and whoever happens to be on the good side of this may survive just a bit longer before they too become redundant.
- analog31 4y ago
  Do we know if anything, other than luck, will determine who's on which side? Or, if any decisive action on anybody's part is more likely to pay off than just staying mentally flexible, watching, and waiting?

comment_ran 4y ago

It's a good idea to think about some aspects of humans. Most people never be content with what they have. Once progress was made, we get used to the old tool and begin to build new norms. The pain is always there, you just run faster.

JohnFen 4y ago

> What we're seeing is the first instance, still very limited and imperfect, of AGI. None of this is AGI. This is Eliza on steroids.

sacrosancty 4y ago
[dead]
barking_biscuit 4y ago
It seems disingenuous to say that this is Eliza on steroids. They operate in fundamentally different ways, no? I definitely agree with the premise that the current state of the art isn't at all AGI, but it seems almost self-evident to me that LLMs are a key piece of the puzzle on the road to AGI. Eliza was never going to have that kind of trajectory, but LLMs I think you could certainly make that argument for.

Aperocky 4y ago

This take is widely prevalent in every single ChatGPT discussion. I'll quote myself for discussion for a previous article: https://news.ycombinator.com/item?id=34746348 https://news.ycombinator.com/item?id=34746348 > I've find two groups of people on this subject, first one has taken a graduate level stat course and/or ML course, and may have work experience in machine learning/data science. The latter camp is more numerous and did not do those things. > The latter group are far more hyped about ChatGPT et al, despite explanation by the first group. > Don't get me wrong, ChatGPT is very exciting, just not in the way it is frequently portrait to be, in particular, development on this model will not lead to General Intelligence, which is not a data training/stat problem altogether. With how the ML field is shaping up to be today, it doesn't even look to be a Machine Learning problem. And let me just add a ridiculous comparison, since you are assuming that it is a path towards AGI: > iPhone 4 isn't fusion energy. It's incredibly useful and a tectonic shift in the cellphone industry, but it's not producing energy. The analogy might sound completely insane, but that's how different machine learning of today and general intelligence is. While AI of today feels like a blackbox, at every stage of its training we know to a degree what it is doing, and once trained, it is as deterministic as a few billion interconnected if else connected to a random seed generator. It could do a lot of things, acquiring a mind of its own isn't one of it. I wouldn't be worried just yet, because AGI is something that research have barely had an idea on how it's going to look outside of our brain, or even inside. I'm not saying that it will never come, just that I highly doubt it will be invented in my lifetime. In the meanwhile, I have something that all of bots today won't have, initiative.

circuit10 4y ago
But from what I’ve seen many skilled AI researchers who have even reimplemented these models themselves do in fact believe that a scaled up ChatGPT could be AGI, or at least that it’s a step towards AGI
- Aperocky 4y ago
  > many skilled citation needed, I'm only aware of one case which might be characterized as such (though I digress): https://www.engadget.com/blake-lemoide-fired-google-lamda-sentient-001746197.html https://www.engadget.com/blake-lemoide-fired-google-lamda-se...

EamonnMR 4y ago

Butlerian Jihad when?

cmrdporcupine 4y ago

I'm just going to quit my programming job and take up flint knapping, make some quality stone tools -- that's what homo genus did for 2 million years before computers, I figure I can always fall back on that.

breuleux 4y ago

> I can't even picture a functional world where humans are cut out of most professions that don't involve manual work Many people do manual work. It may very well be the case that general fine motor skills are a far more complex and difficult operation than the entire edifice of human intellect. Philosophically, it would be an immense blow, the mother of all existential crises. But regardless, it suggests that the first AGI would be incapable of independent survival and we'd still be relevant and in control for a while. > where any amount of acquired knowledge and skills will be surpassed by machines that can produce better results at a thousandth of the cost in a thousandth of the time a human can. I don't think we can necessarily extrapolate it to be that cheap. It could be. But it is also possible that the increases in cost and resources to scale these models bigger and bigger will outstrip hardware progress and that this technology will run into a dead end. To put it differently, I think it is far from clear that the kind of hardware that we build is actually better than an organic substrate for this sort of computation. Imagine an optimized organic neural implementation of ChatGPT, for instance. Would it be slower or more expensive than ChatGPT? Perhaps not. Likewise, the very best that the current paradigm can offer may not be faster or cheaper than humans are at quite many valuable tasks.