5 ms·

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "u

I would love to see a real critique of the potential of transformer models that doesn't use the words "semantic", "syntactic", "symbolic", "know", "meaning", "understand" or "think(ing)/thought". Predicting what it can and can't do, or might and might not be able to do, lets us productively talk about potential limitations.

azinman2 6y ago
Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.
- mordymoop 6y ago
  I don’t recall any strong argument that Searle’s Chinese room can’t be an AGI, just that it can’t be conscious.
  - azinman2 6y ago
    Certainly consciousness and understand are central to Searle’s argument. However from my perspective (particularly someone who is critical of all the recent DNN advances as being some harbinger of AGI), if all you’re doing is looking up replies from a dictionary, then you have no capacity to generalize, learn new things, adapt, empathize, have memory that isn’t already pre-computed and pre-allocated. Now there are ways in which thru fine tuning etc that you can take GPT-3 and have it adapt in some way, thru such adaption or “context/attention networks” give it “memory,” in practice neither of these looks anything like AGI because right now these pages of this Chinese dictionary don’t have any relationship to each other. It becomes clear both in this article (you shouldn’t need some “correct” prompt to get it to make sense), and clear with long form generation that there isn’t a deeper understanding of the meaning of these words. I will say it’s very impressive what it can do when it does make sense, shockingly so, but we are very much in the position where we didn’t expect “chinese” to come out of the machine at all and thus we are projecting onto its outputs an anthropomorphisis that is unwarranted.
- Isinlor 6y ago
  To me you are also just a very good version of a Chinese room. How would you show me that you are not?
  - azinman2 6y ago
    You can look up my comment history and find a continuity and a perspective mired in personal experience that a machine cannot (at least currently) replicate.
    - Isinlor 6y ago
      That proves that you are not GPT-3, but you could still be a Chinese room. You may be even a biological human and I still won't know if you are a Chinese room. Maybe since birth you are just sort of sleepwalking? I wouldn't be surprised if there were fully functional people without qualia.
      - azinman2 6y ago
        If I am, I’m the best Chinese room created yet.
        Isinlor 6y ago
        I guess you are :) . I will tell you a secret. I suspect I'm a Chinese room too ^^ ... There are also people with so called total locked-in syndrome i.e. people who can not do any voluntary movements even with their eyes, so there is no way to communicate with them. How would you decide if they are conscious? It may be literally matter of live and death for them. How should we know if they even want to live in their condition?
chubot 6y ago
What are the models useful for? As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong. Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have trouble using? That would be a trillion dollar business. As best as I can tell that's completely beyond GPT-3 and requires a huge breakthrough, which may or may not happen.
- mordymoop 6y ago
  GPT-3 involves zero fine-tuning or customization to any purpose. It’s not attempting to be a product, but rather a platform people can use to explore the possibilities of products. With fine-tuning it could do all sorts of domain-specific things that it can only passably do now. We suspect this to be true because we’ve seen how other systems behave with and without fine-tuning.
  - ClumsyPilot 6y ago
    you are sidestepping the question above - what can it be used for? What kind of fine-tuning is required? Because to me it looks like 'fixing catastrophic errors' not fine-tuning - as anywhere where the text can deliver value, it has to be correct. AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?
    - voces 6y ago
      Language models can help the disabled communicate better and faster (fine-tune on their online output, then the BCI can offer better continuations, so the patient does not have to type as much). Fine-tune on educational materials, and a language model could be a 24/7 assistant to students. Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better ELIZA, and "socialize with" - and "support" people with depression or trauma. Fine-tune on etiquette and social norms, and autistic people could ask questions without being ashamed. These are some positive use cases. There are also neutral use cases (seeding a new social platform with autogenerated comments), and negative use cases (SEO spam, fake reviews, scaling up disinformation campaigns). Edit: not complaining, but no idea why this was downvoted. Would be helpful to state why, so I don't make this mistake in the future (I am optimizing for useful replies).

azinman2 6y ago

Because when people say “AGI is near, just look at GPT-3,” it’d clear that we’re in a really good version of Searle’s chinese room. The lack of understanding is the important point.

mordymoop 6y ago
I don’t recall any strong argument that Searle’s Chinese room can’t be an AGI, just that it can’t be conscious.
- azinman2 6y ago
  Certainly consciousness and understand are central to Searle’s argument. However from my perspective (particularly someone who is critical of all the recent DNN advances as being some harbinger of AGI), if all you’re doing is looking up replies from a dictionary, then you have no capacity to generalize, learn new things, adapt, empathize, have memory that isn’t already pre-computed and pre-allocated. Now there are ways in which thru fine tuning etc that you can take GPT-3 and have it adapt in some way, thru such adaption or “context/attention networks” give it “memory,” in practice neither of these looks anything like AGI because right now these pages of this Chinese dictionary don’t have any relationship to each other. It becomes clear both in this article (you shouldn’t need some “correct” prompt to get it to make sense), and clear with long form generation that there isn’t a deeper understanding of the meaning of these words. I will say it’s very impressive what it can do when it does make sense, shockingly so, but we are very much in the position where we didn’t expect “chinese” to come out of the machine at all and thus we are projecting onto its outputs an anthropomorphisis that is unwarranted.
Isinlor 6y ago
To me you are also just a very good version of a Chinese room. How would you show me that you are not?
- azinman2 6y ago
  You can look up my comment history and find a continuity and a perspective mired in personal experience that a machine cannot (at least currently) replicate.
  - Isinlor 6y ago
    That proves that you are not GPT-3, but you could still be a Chinese room. You may be even a biological human and I still won't know if you are a Chinese room. Maybe since birth you are just sort of sleepwalking? I wouldn't be surprised if there were fully functional people without qualia.
    - azinman2 6y ago
      If I am, I’m the best Chinese room created yet.
      - Isinlor 6y ago
        I guess you are :) . I will tell you a secret. I suspect I'm a Chinese room too ^^ ... There are also people with so called total locked-in syndrome i.e. people who can not do any voluntary movements even with their eyes, so there is no way to communicate with them. How would you decide if they are conscious? It may be literally matter of live and death for them. How should we know if they even want to live in their condition?

chubot 6y ago

What are the models useful for? As best I can tell the only thing would be some kind of GMail-like auto-reply which a human consciously edits, and which has little consequence if it's wrong. Are the models useful for customer service? Like reading a manual or knowledgebase and then answering a customer's questions about a product, and troubleshooting problems? Like about your Android phone, which lots of people have trouble using? That would be a trillion dollar business. As best as I can tell that's completely beyond GPT-3 and requires a huge breakthrough, which may or may not happen.

mordymoop 6y ago
GPT-3 involves zero fine-tuning or customization to any purpose. It’s not attempting to be a product, but rather a platform people can use to explore the possibilities of products. With fine-tuning it could do all sorts of domain-specific things that it can only passably do now. We suspect this to be true because we’ve seen how other systems behave with and without fine-tuning.
- ClumsyPilot 6y ago
  you are sidestepping the question above - what can it be used for? What kind of fine-tuning is required? Because to me it looks like 'fixing catastrophic errors' not fine-tuning - as anywhere where the text can deliver value, it has to be correct. AWS/Azure are a platform, and they work reliably. What kind of platform 'sometimes works'?
  - voces 6y ago
    Language models can help the disabled communicate better and faster (fine-tune on their online output, then the BCI can offer better continuations, so the patient does not have to type as much). Fine-tune on educational materials, and a language model could be a 24/7 assistant to students. Fine-tune on psychology or medicine data, and a language model could tell you of all medicine interactions, or act as a better ELIZA, and "socialize with" - and "support" people with depression or trauma. Fine-tune on etiquette and social norms, and autistic people could ask questions without being ashamed. These are some positive use cases. There are also neutral use cases (seeding a new social platform with autogenerated comments), and negative use cases (SEO spam, fake reviews, scaling up disinformation campaigns). Edit: not complaining, but no idea why this was downvoted. Would be helpful to state why, so I don't make this mistake in the future (I am optimizing for useful replies).