12 ms·

The Inherent Limitations of GPT-3

88 points by andreyk 5y ago

Flankk 5y ago
65 years of research and our cutting-edge AI doesn't have a memory? Excuse me if I'm not excited. It's likely that most of the functions of the human brain were selected for intelligence. Such a focus on learning when problem solving and creativity are far more interesting.
- PaulHoule 5y ago
  I'm trying to put my finger on the source of moral decay that led to so many people behaving as if the GPT-3 emperor wears clothes. In 1966 it was clear to everyone that this program https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA parasitically depends on the hunger for meaning that people have. Recently GPT-3 was held back from the public on the pretense that it was "dangerous" but in reality it held back because it is too expensive to run and the public would quickly learn that it can answer any question at all... if you don't mind if the answer is right. There is this page https://nlp.stanford.edu/projects/glove/ https://nlp.stanford.edu/projects/glove/ under which "2. Linear Substructures" there are four projections of the 50-dimensional vector space that would project out just as well from a random matrix because, well, projecting 20 generic points in a 50-dimensional space to 2-dimensions you can make the points fall exactly where you want in 2 dimensions. Nobody holds them to account over this. The closest thing I see to the GPT-3 cult is that a Harvard professor said that this thing https://en.wikipedia.org/wiki/%CA%BBOumuamua https://en.wikipedia.org/wiki/%CA%BBOumuamua was an alien spacecraft. It's sad and a little scary that people can get away with that, the media picks it up, and they don't face consequences. I am more afraid of that than I am afraid that GPT-99381387 will take over the world. (e.g. growing up in the 1970s I could look to Einstein for inspiration that intelligence could understand the Universe. Somebody today might as well look forward to being a comic book writer like Stan Lee.)
  - deleted 5y ago
    [deleted]
  - bangkoksbest 5y ago
    It's a legitimate practice in science to speculate. Having heard the Harvard guy explain more fully the Oumuamua thing, it's struck me as perfectly fine activity for some scientist to look into. His hypothesis is almost certainly going to be untrue, but it's fine to investigate a bit of a moonshot idea. You don't want half the field doing this, but you absolutely need different little pockets of speculative work going on in order to keep scientific inquiry open, dynamic, and diverse.
  - wwweston 5y ago
    > the public would quickly learn that it can answer any question at all... if you don't mind if the answer is right. There appear to be an awful lot of conversations in which people care about other things much, much more than what is objectively correct. And any technology that can greatly amplify engagement in that kind of conversation probably is dangerous.
  - canjobear 5y ago
    GPT3 and its cousins do things that no previous language model could do; it is qualitatively different from Eliza in its capabilities. As for your argument about random projections in the evaluation of GLoVE, comparisons with random projections are now routine. See for example https://aclanthology.org/N19-1419/ https://aclanthology.org/N19-1419/
    - NoGravitas 5y ago
      Why do you say it is qualitatively different from Eliza in its capabilities?
      - PaulHoule 5y ago
        It does something totally different. However that totally different still depends on people being desperate to see intelligence inside it. It’s like how people see a face in a cut stem or on Mars.
        NoGravitas 5y ago
        Is it because it does something totally different that you came to me?
        rytill 5y ago
        You're trying to prove some kind of point where you respond as ELIZA would have to show how "even back then we could pass for conversation". The truth is that GPT-3 is actually, totally qualitatively different and if you played with it enough you'd realize.
        not2b 5y ago
        The difference is quantitative, rather than qualitative, as compared to primitive Markov models that have been used in the past. It's just a numerical model with a very large number of parameters that extends a text token sequence. The parameter size is so large that it has in essence memorized its training data, so if the right answer was already present in the training data you'll get it, same if the answer is closely related to the training data in a way that lets the model predict it. If the wrong answer was present in the training data you may well get that.
  - thedorkknight 5y ago
    Confused. If professor Loeb tries to at least open discourse to the idea that ET space junk might be flying around like our space junk in a desire to reduce the giggle factor around that hypothesis, what sort of "consequences" do you think he should face for that?
  - CrazyStat 5y ago
    > there are four projections of the 50-dimensional vector space that would project out just as well from a random matrix because, well, projecting 20 generic points in a 50-dimensional space to 2-dimensions you can make the points fall exactly where you want in 2 dimensions. Well, no. Linear projections follow a bunch of rules that enforce conservation of various linear structures. You can't manipulate things arbitrarily. For example, if three points are colinear in the original space, they will be colinear in every projection. Perhaps more relevant to the GloVe examples, if a-b = c-d in the original space then the same equality holds in every projection. Since the projections are Lipschitz continuous, we can also say that is a-b is close to c-d in the original space then they are also close in every projection. Anyway, the point that these figures illustrate is easily confirmed by downloading the embeddings yourself, so insinuating that the authors are getting away with something that they should be held to account for is silly.
- gibsonf1 5y ago
  In addition to that subtle memory issue, it has no reference at all to the space/time world we people model mentally to think with. So, basically, there is no I in the GPT-3 AI, just A.
  - PaulHoule 5y ago
    One can point to many necessary structural features that it is missing. Consider Ashby's law of requisite variety: https://www.edge.org/response-detail/27150 https://www.edge.org/response-detail/27150 Many GPT-3 cultists are educated in computer science so they should know better. GPT-3's "one pass" processing means that a fixed amount of resources are always used. Thus it can't sort a list of items unless the fixed time it uses is humongous. You might boil the oceans that way but you won't attain AGI. There are numerous arguments along the line of Turing's halting problem that restrict what that kind of thing can do. As it uses a finite amount of time it can't do anything that could require an unbounded time to complete or that could potentially not terminate. GPT-3 has no model for dealing with ambiguity or uncertainty. (Other than shooting in the dark.) Practically this requires some ability to backtrack either automatically or as a result of user feedback. The current obscurantism is that you need to have 20 PhD students work for 2 years to write a paper that makes the model "explainable" in some narrow domain. With this insight you can spend another $30 million training a new model that might get the answer right. A practical system needs to be told that "you did it wrong" and why and then be able to correct itself on the next pass if possible, otherwise in a few passes. Of course a system like that would be a real piece of engineering that people would become familiar with, not a outlet for their religious feelings that is kept on a pedestal.
    - gibsonf1 5y ago
      The big issue is that it literally knows nothing - there is no reference to a model of the real world such as humans use when thinking about the real world. It is a very advanced pattern matching parrot, and in using words like a parrot, knows nothing about what those words mean.
      - 5y ago
- andreyk 5y ago
  This is very specific to GPT-3 and not generally true though. And GPT-3 is not an agent per se but rather a passive model (it received input and produces output, and does not continuously interact with its environment). So it makes sense in this context, and just goes to show GPT-3 needs to be understood for what it is.
- naasking 5y ago
  > It's likely that most of the functions of the human brain were selected for intelligence. That doesn't seem correct. Intelligence came much later than when most of our brain evolved.
  - PaulHoule 5y ago
    Intelligence involves many layers. Planaria can move towards and away from things and even learn. Bees work collectively to harvest nectar from flowers and build hives. Mammals have a "theory of mind" and are very good at reasoning about what other beings think about what other beings think. For that matter birds are pretty smart in terms of ability to navigate 1000 miles and find the same nest. People make tools, use language, play chess, bullshit each other and make cults around rationalism and GPT-3.
    - naasking 5y ago
      "Adaptation" is not synonymous with "intelligence". The latter is a much more narrowly defined phenomenon.
  - pfortuny 5y ago
    memory is something shared by… one might even say plants. But let us keep to animals: almost anyone, including worms.
- manojlds 5y ago
  Do our aeroplanes flap their wings like the birds do? GPT-3 is obviously not the AI end goal, but we are on the path and the end might lead to aeroplanes than flapping machines.
  - ska 5y ago
    > but we are on the path This isn't actually clear; with things like this we are on a path but it may not lead anywhere that fundamental (at least when we are talking "AI", especially general AI).
  - Flankk 5y ago
    Birds don't need 150,000 litres of jet fuel to fly across the ocean. Given that the development of airplanes was made by studying birds I'm not sure I see your point. The 1889 book "Birdflight as the Basis of Aviation" is one example.
- Groxx 5y ago
  The current leading purchase-able extremely-over-hyped-by-non-technicals language model has no memory, yes. You see the same thing in all popular reporting about science and tech. Endless battery breakthroughs that will quadruple or 10x capacity become a couple percent improvement in practice. New gravity models mean we might have practical warp drives in 50 years. Fusion that's perpetually 20 years away. Flying cars and personal jetpacks. Moon bases, when we haven't been on the moon since the 70s. AI reporting and hype is no different. Maybe slightly worse because it's touching on "intelligence", which we still have no clear definition of.
andreyk 5y ago
Author here, would love feedback / thoughts / corrections!
- skybrian 5y ago
  Another limitation to be aware of is that it generates text by randomly choosing the next word from a probability distribution. If you turn that off, it tends to go into a loop. The random choices improve text generation from an artistic perspective, but if you want to know why it chose one word rather than another, the answer is sometimes that it chose a low-probability word at random. So there is a built-in error rate (assuming not all completions are valid), and the choice of one completion versus another is clearly not made based on meaning. (It can be artistically interesting anyway since a human can pick the best completions based on their knowledge of meanings.) On the other hand, going into a loop (if you always choose the highest probability next word) also demonstrates pretty clearly that it doesn’t know what it’s saying.
  - andreyk 5y ago
    Good point, thanks!
- mojuba 5y ago
  Prove us you are not GPT-3 ;)
  - andreyk 5y ago
    Why would I write an article about my own limitations :P
Jack000 5y ago
GPT3 is a huge language model, no more and no less. If you expect it to be AGI you're going to be dissapointed. I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam..
- andreyk 5y ago
  yep, pretty much what i'm saying here. Though not all language models are built the same, eg the inference cost is unique to it due to its size. Still, most of this applies to any typical language model.
- PaulHoule 5y ago
  Works to accomplish what useful task?
  - deleted 5y ago
    [deleted]
  - deleted 5y ago
    [deleted]
  - modeless 5y ago
    Github Copilot? It may not be perfect but I think it can definitely be useful.
    - PaulHoule 5y ago
      It is useful if you don't care if the product is right. Most engineering managers would think "this is great!" but the customer won't agree. The CEO will agree until the customers revolt.
      - deleted 5y ago
        [deleted]
      - rpedela 5y ago
        There are several use cases where ML can help even if it isn't perfect or even just better than random. Here is one example in NLP/search. Let's say you have a product search engine and you analyzed the logged queries. What you find is a very long tail of queries that are only searched once or twice. In most cases, the queries are either misspellings, synonyms that aren't in the product text, or long queries that describe the product with generic keywords. And the queries either return zero results or junk. If text classification for the product category is applied to these long tail queries, then the search results will improve and likely yield a boost in sales because users can find what they searched for. Even if the model is only 60% accurate, it will still help because more queries are returning useful results than before. However you don't apply ML with 60% accuracy to your top N queries because it could ruin the results and reduce sales. Knowing when to use ML is just as important as improving its accuracy.
nonameiguess 5y ago
I can't prove it, but I suspect there is a more fundamental limitation to any language model that is purely a language model in the sense of a probability distribution over possible words given the precedent of other words. Gaining any meaningful level of understanding without an awareness that things other than words even exist seems like it won't happen. The most obvious limitation is you can't develop a language that way. Language is a compression of reality or of some other intermediate model of reality to either an audio stream or symbol stream, so not having access to the less abstracted models, let alone to reality itself, means you can never understand anything except the existing corpus. That isn't a criticism of GPT-3 by any stretch, as comments like this seem to often get interpreted that way, but the "taking all possible jobs AGI" hype seems a bit out of control given it is just a language model. Even something with the unambiguous intellect of a human, say an actual human, but with no ability to move, no senses other than hearing, that never heard anything except speech, would not be expected by anyone to dominate all job markets and advance the intellectual frontier. This, of course, goes beyond fundamental limitations of GPT-3, as I see this as a fundamental limitation of any language model whatsoever. On its own, it isn't enough. At some point, AI research is going to have to figure out how to fuse models from many domains and get them to cooperatively model all of the various ways to explore and sense reality. That includes the corpus of existing human written knowledge, but it isn't just that.
- astrange 5y ago
  > That isn't a criticism of GPT-3 by any stretch, as comments like this seem to often get interpreted that way, but the "taking all possible jobs AGI" hype seems a bit out of control given it is just a language model. Note the "taking all possible jobs" scenario is not all that… possible. Strict supply and demand analysis doesn't really work for jobs, for instance because labor creates its own demand, and you usually run into fallacies like Luddism and "lump of labor". At worst, comparative advantage means you'd have a job being human, something AGI can't do.
- sweezyjeezy 5y ago
  Interesting example is the "black sheep problem" - language models don't "understand" that sheep are overwhelmingly white. Why? Because we tend not to specify the colour of a sheep in writing if it's white, and because of the associated idiom about misfits ("black sheep" outnumbers "white sheep" 25:1 in written English).
thamer 5y ago
Something I've noticed that both GPT-2 and GPT-3 tend to do is get stuck in a loop, repeating the same thing over and over again. As if the system was relying on recent text/concepts to go to the next utterance, only getting into a state where the next sentence or block of code being produced is one that has already been generated. It's not exactly uncommon. What causes this? I'm curious to know what triggers this behavior. Here's an example of GPT-2 posting on Reddit, getting stuck on "below minimum wage" or equivalent: https://reddit.com/r/SubSimulatorGPT2/comments/engt9v/my_former_boss_has_been_paying_me_less_than/ https://reddit.com/r/SubSimulatorGPT2/comments/engt9v/my_for... (edit) another example from the GPT-2 subreddit: https://reddit.com/r/SubSimulatorGPT2/comments/en1sy0/im_going_through_a_period_of_questioning_identity/fdt4arh/ https://reddit.com/r/SubSimulatorGPT2/comments/en1sy0/im_goi... With GPT-3, I saw GitHub Copilot generate the same line or block of code over and over a couple of times.
- not2b 5y ago
  Limited memory, as the article points out. It doesn't remember what it said beyond a certain point. It's a bit like the lead character in the film "Memento". A very long time ago (early 1990s) I wrote a much simpler text generator: it digested Usenet postings and built a Markov chain model based on the previous two tokens. It produced reasonable sentences but would go into loops. Same issue at a smaller scale.
- Abrownn 5y ago
  This is exactly why we stopped using it. Even after fine tuning the parameters and picking VERY good input text, it still got stuck in loops or repeated itself too much even after 2 or 3 tries. It's neat as-is, but not useful for us. Maybe GPT-4 will fix the "looping" issue.
  - xwolfi 5y ago
    Have you talked to an angry redneck in a dank pub ? He'll do the same. And not just him, but whomever is expected to speak but has no high value things to add, and is a little bit dumb/drunk to count how many times he said something and stop at 3. Maybe information, to be interesting to us, has to be novel, while GPT-3 may not model for listener's interest (like you when you re drunk) and only produces the best it can express in a given input context ? And sometimes, maybe repeating 34 times the same thing is good if no new input changes the fundamentals, just not very interesting for a signal dampener like our brain who starts losing focus when novelty disappears from the signal? It s like imagine a political debate around building a bridge between a truck driver who wants to go faster and a bird watcher who wants birds to keep their habitat close to his home. There's no input that can change the fundamentals and it would be expected that after a few loops, no brain could find anything to add and just repeat forever the same thing: but the birds must be close to me or I lose my life's meaning, but the bridge must be built there or I cant optimize my route. The only thing we do is put a time stop and say "ok we got it, now everyone in the public can map their own constraint to the discussion and vote".
emmelaich 5y ago
> "The Inherent Limitations of GPT-3 as is"
- evrydayhustling 5y ago
  Came here for this... there isn't anything clearly "inherent" about these limitations, except to the degree that they are part of how some folks define the label "GPT-3". If one packed GPT-3's structural architecture into a reinforcement learning system to grant memory, or pushed it through a compression system that made it cheaper to train or run, would you say GPT-3 had transcended its limitations, or just that you created something new? The fact that this question is semantic and uninteresting is why "Method / Model X doesn't do everything" posts don't progress the scientific conversation.
da39a3ee 5y ago
> GPT-3 lacks any form of memory. Is that true? How is it able to have the conversation shown in the article about python programming, if it can’t remember that the premise of the later questions it is being asked is that it said it has Python programming experience?
- YeGoblynQueenne 5y ago
  It is fed each answer it generates as part of the next input. That's why it can't keep that up for too long: the size of its input grows well-kinda-sorta- exponentially quickly until it hits the limit of 2048 tokens. Which is one rason why only diminshing returns can come from increasing computational resources, btw. For inference, anyway. You can see examples of that kind of back-to-back querying and examples of code to do it in one of the links of the article above: https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-language-model https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-lang...

88 points by andreyk 5y ago

65 years of research and our cutting-edge AI doesn't have a memory? Excuse me if I'm not excited. It's likely that most of the functions of the human brain were selected for intelligence. Such a focus on learning when problem solving and creativity are far more interesting.

PaulHoule 5y ago
I'm trying to put my finger on the source of moral decay that led to so many people behaving as if the GPT-3 emperor wears clothes. In 1966 it was clear to everyone that this program https://en.wikipedia.org/wiki/ELIZA https://en.wikipedia.org/wiki/ELIZA parasitically depends on the hunger for meaning that people have. Recently GPT-3 was held back from the public on the pretense that it was "dangerous" but in reality it held back because it is too expensive to run and the public would quickly learn that it can answer any question at all... if you don't mind if the answer is right. There is this page https://nlp.stanford.edu/projects/glove/ https://nlp.stanford.edu/projects/glove/ under which "2. Linear Substructures" there are four projections of the 50-dimensional vector space that would project out just as well from a random matrix because, well, projecting 20 generic points in a 50-dimensional space to 2-dimensions you can make the points fall exactly where you want in 2 dimensions. Nobody holds them to account over this. The closest thing I see to the GPT-3 cult is that a Harvard professor said that this thing https://en.wikipedia.org/wiki/%CA%BBOumuamua https://en.wikipedia.org/wiki/%CA%BBOumuamua was an alien spacecraft. It's sad and a little scary that people can get away with that, the media picks it up, and they don't face consequences. I am more afraid of that than I am afraid that GPT-99381387 will take over the world. (e.g. growing up in the 1970s I could look to Einstein for inspiration that intelligence could understand the Universe. Somebody today might as well look forward to being a comic book writer like Stan Lee.)
- deleted 5y ago
  [deleted]
- bangkoksbest 5y ago
  It's a legitimate practice in science to speculate. Having heard the Harvard guy explain more fully the Oumuamua thing, it's struck me as perfectly fine activity for some scientist to look into. His hypothesis is almost certainly going to be untrue, but it's fine to investigate a bit of a moonshot idea. You don't want half the field doing this, but you absolutely need different little pockets of speculative work going on in order to keep scientific inquiry open, dynamic, and diverse.
- wwweston 5y ago
  > the public would quickly learn that it can answer any question at all... if you don't mind if the answer is right. There appear to be an awful lot of conversations in which people care about other things much, much more than what is objectively correct. And any technology that can greatly amplify engagement in that kind of conversation probably is dangerous.
- canjobear 5y ago
  GPT3 and its cousins do things that no previous language model could do; it is qualitatively different from Eliza in its capabilities. As for your argument about random projections in the evaluation of GLoVE, comparisons with random projections are now routine. See for example https://aclanthology.org/N19-1419/ https://aclanthology.org/N19-1419/
  - NoGravitas 5y ago
    Why do you say it is qualitatively different from Eliza in its capabilities?
    - PaulHoule 5y ago
      It does something totally different. However that totally different still depends on people being desperate to see intelligence inside it. It’s like how people see a face in a cut stem or on Mars.
      - NoGravitas 5y ago
        Is it because it does something totally different that you came to me?
        rytill 5y ago
        You're trying to prove some kind of point where you respond as ELIZA would have to show how "even back then we could pass for conversation". The truth is that GPT-3 is actually, totally qualitatively different and if you played with it enough you'd realize.
        not2b 5y ago
        The difference is quantitative, rather than qualitative, as compared to primitive Markov models that have been used in the past. It's just a numerical model with a very large number of parameters that extends a text token sequence. The parameter size is so large that it has in essence memorized its training data, so if the right answer was already present in the training data you'll get it, same if the answer is closely related to the training data in a way that lets the model predict it. If the wrong answer was present in the training data you may well get that.
- thedorkknight 5y ago
  Confused. If professor Loeb tries to at least open discourse to the idea that ET space junk might be flying around like our space junk in a desire to reduce the giggle factor around that hypothesis, what sort of "consequences" do you think he should face for that?
- CrazyStat 5y ago
  > there are four projections of the 50-dimensional vector space that would project out just as well from a random matrix because, well, projecting 20 generic points in a 50-dimensional space to 2-dimensions you can make the points fall exactly where you want in 2 dimensions. Well, no. Linear projections follow a bunch of rules that enforce conservation of various linear structures. You can't manipulate things arbitrarily. For example, if three points are colinear in the original space, they will be colinear in every projection. Perhaps more relevant to the GloVe examples, if a-b = c-d in the original space then the same equality holds in every projection. Since the projections are Lipschitz continuous, we can also say that is a-b is close to c-d in the original space then they are also close in every projection. Anyway, the point that these figures illustrate is easily confirmed by downloading the embeddings yourself, so insinuating that the authors are getting away with something that they should be held to account for is silly.
gibsonf1 5y ago
In addition to that subtle memory issue, it has no reference at all to the space/time world we people model mentally to think with. So, basically, there is no I in the GPT-3 AI, just A.
- PaulHoule 5y ago
  One can point to many necessary structural features that it is missing. Consider Ashby's law of requisite variety: https://www.edge.org/response-detail/27150 https://www.edge.org/response-detail/27150 Many GPT-3 cultists are educated in computer science so they should know better. GPT-3's "one pass" processing means that a fixed amount of resources are always used. Thus it can't sort a list of items unless the fixed time it uses is humongous. You might boil the oceans that way but you won't attain AGI. There are numerous arguments along the line of Turing's halting problem that restrict what that kind of thing can do. As it uses a finite amount of time it can't do anything that could require an unbounded time to complete or that could potentially not terminate. GPT-3 has no model for dealing with ambiguity or uncertainty. (Other than shooting in the dark.) Practically this requires some ability to backtrack either automatically or as a result of user feedback. The current obscurantism is that you need to have 20 PhD students work for 2 years to write a paper that makes the model "explainable" in some narrow domain. With this insight you can spend another $30 million training a new model that might get the answer right. A practical system needs to be told that "you did it wrong" and why and then be able to correct itself on the next pass if possible, otherwise in a few passes. Of course a system like that would be a real piece of engineering that people would become familiar with, not a outlet for their religious feelings that is kept on a pedestal.
  - gibsonf1 5y ago
    The big issue is that it literally knows nothing - there is no reference to a model of the real world such as humans use when thinking about the real world. It is a very advanced pattern matching parrot, and in using words like a parrot, knows nothing about what those words mean.
    - 5y ago
andreyk 5y ago
This is very specific to GPT-3 and not generally true though. And GPT-3 is not an agent per se but rather a passive model (it received input and produces output, and does not continuously interact with its environment). So it makes sense in this context, and just goes to show GPT-3 needs to be understood for what it is.
naasking 5y ago
> It's likely that most of the functions of the human brain were selected for intelligence. That doesn't seem correct. Intelligence came much later than when most of our brain evolved.
- PaulHoule 5y ago
  Intelligence involves many layers. Planaria can move towards and away from things and even learn. Bees work collectively to harvest nectar from flowers and build hives. Mammals have a "theory of mind" and are very good at reasoning about what other beings think about what other beings think. For that matter birds are pretty smart in terms of ability to navigate 1000 miles and find the same nest. People make tools, use language, play chess, bullshit each other and make cults around rationalism and GPT-3.
  - naasking 5y ago
    "Adaptation" is not synonymous with "intelligence". The latter is a much more narrowly defined phenomenon.
- pfortuny 5y ago
  memory is something shared by… one might even say plants. But let us keep to animals: almost anyone, including worms.
manojlds 5y ago
Do our aeroplanes flap their wings like the birds do? GPT-3 is obviously not the AI end goal, but we are on the path and the end might lead to aeroplanes than flapping machines.
- ska 5y ago
  > but we are on the path This isn't actually clear; with things like this we are on a path but it may not lead anywhere that fundamental (at least when we are talking "AI", especially general AI).
- Flankk 5y ago
  Birds don't need 150,000 litres of jet fuel to fly across the ocean. Given that the development of airplanes was made by studying birds I'm not sure I see your point. The 1889 book "Birdflight as the Basis of Aviation" is one example.
Groxx 5y ago
The current leading purchase-able extremely-over-hyped-by-non-technicals language model has no memory, yes. You see the same thing in all popular reporting about science and tech. Endless battery breakthroughs that will quadruple or 10x capacity become a couple percent improvement in practice. New gravity models mean we might have practical warp drives in 50 years. Fusion that's perpetually 20 years away. Flying cars and personal jetpacks. Moon bases, when we haven't been on the moon since the 70s. AI reporting and hype is no different. Maybe slightly worse because it's touching on "intelligence", which we still have no clear definition of.

andreyk 5y ago

Author here, would love feedback / thoughts / corrections!

skybrian 5y ago
Another limitation to be aware of is that it generates text by randomly choosing the next word from a probability distribution. If you turn that off, it tends to go into a loop. The random choices improve text generation from an artistic perspective, but if you want to know why it chose one word rather than another, the answer is sometimes that it chose a low-probability word at random. So there is a built-in error rate (assuming not all completions are valid), and the choice of one completion versus another is clearly not made based on meaning. (It can be artistically interesting anyway since a human can pick the best completions based on their knowledge of meanings.) On the other hand, going into a loop (if you always choose the highest probability next word) also demonstrates pretty clearly that it doesn’t know what it’s saying.
- andreyk 5y ago
  Good point, thanks!
mojuba 5y ago
Prove us you are not GPT-3 ;)
- andreyk 5y ago
  Why would I write an article about my own limitations :P

Jack000 5y ago

GPT3 is a huge language model, no more and no less. If you expect it to be AGI you're going to be dissapointed. I find some of these negative comments to be overly hyperbolic though. It clearly works and is not some kind of scam..

andreyk 5y ago
yep, pretty much what i'm saying here. Though not all language models are built the same, eg the inference cost is unique to it due to its size. Still, most of this applies to any typical language model.
PaulHoule 5y ago
Works to accomplish what useful task?
- deleted 5y ago
  [deleted]
- deleted 5y ago
  [deleted]
- modeless 5y ago
  Github Copilot? It may not be perfect but I think it can definitely be useful.
  - PaulHoule 5y ago
    It is useful if you don't care if the product is right. Most engineering managers would think "this is great!" but the customer won't agree. The CEO will agree until the customers revolt.
    - deleted 5y ago
      [deleted]
    - rpedela 5y ago
      There are several use cases where ML can help even if it isn't perfect or even just better than random. Here is one example in NLP/search. Let's say you have a product search engine and you analyzed the logged queries. What you find is a very long tail of queries that are only searched once or twice. In most cases, the queries are either misspellings, synonyms that aren't in the product text, or long queries that describe the product with generic keywords. And the queries either return zero results or junk. If text classification for the product category is applied to these long tail queries, then the search results will improve and likely yield a boost in sales because users can find what they searched for. Even if the model is only 60% accurate, it will still help because more queries are returning useful results than before. However you don't apply ML with 60% accuracy to your top N queries because it could ruin the results and reduce sales. Knowing when to use ML is just as important as improving its accuracy.

nonameiguess 5y ago

I can't prove it, but I suspect there is a more fundamental limitation to any language model that is purely a language model in the sense of a probability distribution over possible words given the precedent of other words. Gaining any meaningful level of understanding without an awareness that things other than words even exist seems like it won't happen. The most obvious limitation is you can't develop a language that way. Language is a compression of reality or of some other intermediate model of reality to either an audio stream or symbol stream, so not having access to the less abstracted models, let alone to reality itself, means you can never understand anything except the existing corpus. That isn't a criticism of GPT-3 by any stretch, as comments like this seem to often get interpreted that way, but the "taking all possible jobs AGI" hype seems a bit out of control given it is just a language model. Even something with the unambiguous intellect of a human, say an actual human, but with no ability to move, no senses other than hearing, that never heard anything except speech, would not be expected by anyone to dominate all job markets and advance the intellectual frontier. This, of course, goes beyond fundamental limitations of GPT-3, as I see this as a fundamental limitation of any language model whatsoever. On its own, it isn't enough. At some point, AI research is going to have to figure out how to fuse models from many domains and get them to cooperatively model all of the various ways to explore and sense reality. That includes the corpus of existing human written knowledge, but it isn't just that.

astrange 5y ago
> That isn't a criticism of GPT-3 by any stretch, as comments like this seem to often get interpreted that way, but the "taking all possible jobs AGI" hype seems a bit out of control given it is just a language model. Note the "taking all possible jobs" scenario is not all that… possible. Strict supply and demand analysis doesn't really work for jobs, for instance because labor creates its own demand, and you usually run into fallacies like Luddism and "lump of labor". At worst, comparative advantage means you'd have a job being human, something AGI can't do.
sweezyjeezy 5y ago
Interesting example is the "black sheep problem" - language models don't "understand" that sheep are overwhelmingly white. Why? Because we tend not to specify the colour of a sheep in writing if it's white, and because of the associated idiom about misfits ("black sheep" outnumbers "white sheep" 25:1 in written English).

thamer 5y ago

Something I've noticed that both GPT-2 and GPT-3 tend to do is get stuck in a loop, repeating the same thing over and over again. As if the system was relying on recent text/concepts to go to the next utterance, only getting into a state where the next sentence or block of code being produced is one that has already been generated. It's not exactly uncommon. What causes this? I'm curious to know what triggers this behavior. Here's an example of GPT-2 posting on Reddit, getting stuck on "below minimum wage" or equivalent: https://reddit.com/r/SubSimulatorGPT2/comments/engt9v/my_former_boss_has_been_paying_me_less_than/ https://reddit.com/r/SubSimulatorGPT2/comments/engt9v/my_for... (edit) another example from the GPT-2 subreddit: https://reddit.com/r/SubSimulatorGPT2/comments/en1sy0/im_going_through_a_period_of_questioning_identity/fdt4arh/ https://reddit.com/r/SubSimulatorGPT2/comments/en1sy0/im_goi... With GPT-3, I saw GitHub Copilot generate the same line or block of code over and over a couple of times.

not2b 5y ago
Limited memory, as the article points out. It doesn't remember what it said beyond a certain point. It's a bit like the lead character in the film "Memento". A very long time ago (early 1990s) I wrote a much simpler text generator: it digested Usenet postings and built a Markov chain model based on the previous two tokens. It produced reasonable sentences but would go into loops. Same issue at a smaller scale.
Abrownn 5y ago
This is exactly why we stopped using it. Even after fine tuning the parameters and picking VERY good input text, it still got stuck in loops or repeated itself too much even after 2 or 3 tries. It's neat as-is, but not useful for us. Maybe GPT-4 will fix the "looping" issue.
- xwolfi 5y ago
  Have you talked to an angry redneck in a dank pub ? He'll do the same. And not just him, but whomever is expected to speak but has no high value things to add, and is a little bit dumb/drunk to count how many times he said something and stop at 3. Maybe information, to be interesting to us, has to be novel, while GPT-3 may not model for listener's interest (like you when you re drunk) and only produces the best it can express in a given input context ? And sometimes, maybe repeating 34 times the same thing is good if no new input changes the fundamentals, just not very interesting for a signal dampener like our brain who starts losing focus when novelty disappears from the signal? It s like imagine a political debate around building a bridge between a truck driver who wants to go faster and a bird watcher who wants birds to keep their habitat close to his home. There's no input that can change the fundamentals and it would be expected that after a few loops, no brain could find anything to add and just repeat forever the same thing: but the birds must be close to me or I lose my life's meaning, but the bridge must be built there or I cant optimize my route. The only thing we do is put a time stop and say "ok we got it, now everyone in the public can map their own constraint to the discussion and vote".

emmelaich 5y ago

> "The Inherent Limitations of GPT-3 as is"

evrydayhustling 5y ago
Came here for this... there isn't anything clearly "inherent" about these limitations, except to the degree that they are part of how some folks define the label "GPT-3". If one packed GPT-3's structural architecture into a reinforcement learning system to grant memory, or pushed it through a compression system that made it cheaper to train or run, would you say GPT-3 had transcended its limitations, or just that you created something new? The fact that this question is semantic and uninteresting is why "Method / Model X doesn't do everything" posts don't progress the scientific conversation.

da39a3ee 5y ago

> GPT-3 lacks any form of memory. Is that true? How is it able to have the conversation shown in the article about python programming, if it can’t remember that the premise of the later questions it is being asked is that it said it has Python programming experience?

YeGoblynQueenne 5y ago
It is fed each answer it generates as part of the next input. That's why it can't keep that up for too long: the size of its input grows well-kinda-sorta- exponentially quickly until it hits the limit of 2048 tokens. Which is one rason why only diminshing returns can come from increasing computational resources, btw. For inference, anyway. You can see examples of that kind of back-to-back querying and examples of code to do it in one of the links of the article above: https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-language-model https://www.twilio.com/blog/ultimate-guide-openai-gpt-3-lang...