5 ms·

> Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally n

by spion 4y ago

> Sorry, which emergent capabilities are you talking about? You're trying to slip in that there even are emergent capabilities, when in fact there's literally nothing unexpected which has emerged. Given how LLMs work, literally everything they do is exactly what we'd expect them to do. https://openreview.net/pdf?id=yzkSU5zdwD https://openreview.net/pdf?id=yzkSU5zdwD FWIW, none of the capabilities of LLMs are expected, but the above are not expected in the sense that they suddenly emerge beyond a certain scale in a way that couldn't be extrapolated by looking at the smaller models. > For example, it cannot produce something from nothing: any output the program produces must strictly be a combination of its inputs. From this we can tell that no program produced is capable of originality. Can you explain what you mean by "strictly a combination of its inputs"? I've personally implemented a few (non-language) NN prediction models and they definitely extrapolate, someitmes in quite funny (but still essentially accurate) ways when given previously unseen (and ridiculous) inputs.

kerkeslager 4y ago
> https://openreview.net/pdf?id=yzkSU5zdwD https://openreview.net/pdf?id=yzkSU5zdwD That article defines "emergent" as: "An ability is emergent if it is not present in smaller models but is present in larger models." That doesn't make it unexpected, it just makes it a product of scale. The article isn't claiming what you think it's claiming. The article is, frankly, not very interesting. You change the inputs and get different outputs? What a surprise! > Can you explain what you mean by "strictly a combination of its inputs"? I'm saying it can't do anything with data it doesn't have. For example: I prompted it "Give me a one-syllable portmanteau of "bridge" and "dam"." and it returned ""Bridam" is a one-syllable portmanteau of "bridge" and "dam".". It doesn't understand portmanteaus and it doesn't understand syllables, it just sees that when people in its dataset talk about portmanteaus in this pattern, they mash together a word. It has the letters, so it gets right that "bridam" is a portmanteau of "bridge" and "dam", but it can't comply with the "one-syllable" aspect. If you ask it for the number of syllables in a word that is in its dataset, it usually gets it right, because people talk about how many syllables are in words. In fact, if you ask it how many syllables are in the word "bridam" it correctly says two, because the pattern is close enough to what's in its dataset. But you can immediately ask it to create a one-syllable portmanteau and it will again return "bridam", because it's not holding a connected thought, it's just continuing to match the pattern in its data. It simply doesn't have actual syllable data. You'll even get responses such as, "A one-syllable portmanteau of "bridge" and "dam" could be "bridam" (pronounced as "brih-dam")." Even as it produces a syllable breakdown with two syllables due to its pattern-matching, it still produces a two syllable portmanteau while claiming it's one syllable. A human child, if they know what a portmanteau is, can easily come up with a few options such as "bram", or "dadge". > I've personally implemented a few (non-language) NN prediction models and they definitely extrapolate, someitmes in quite funny (but still essentially accurate) ways when given previously unseen (and ridiculous) inputs. The form of extrapolation these systems are capable of isn't so much extrapolation, as matching that the response pattern should be longer and trying to fill it in with existing data. It's a very specific kind of extrapolation and again, not unexpected. EDIT: If you want to see the portmanteau thing in action, it's easy to see how the pattern-matching is applying to this: me: Give me a one-syllable portmanteau of "cap" and "plant"? ChatGPT: Caplant me: Give me a one-syllable portmanteau of "bat" and "tar" ChatGPT: Batar me: Give me a one-syllable portmanteau of "fear" and "red" ChatGPT: Ferred Pick pretty much any two one-syllable words where the first ends with the same letter as the second begins with, and it will go for a two-syllable portmanteau.
- spion 4y ago
  You are conflating the counting problem with an inability to do anything with data it doesn't have. For its ability to invent new languages and translate to and from them, see https://maximumeffort.substack.com/p/i-taught-chatgpt-to-invent-a-language https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... For the reason why LLMs can do this and why they don't necessarily need to have seen the words they read or use, you can look up "BPE tokenization" Could you give me an example that isn't counting-related?
  - kerkeslager 4y ago
    > You are conflating the counting problem with an inability to do anything with data it doesn't have. No, I'm answering your question, "Can you explain what you mean by "strictly a combination of its inputs"?" > For its ability to invent new languages and translate to and from them, see https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... It's not "inventing" new languages, it's pattern matching from the massive section of its dataset taken from the synthetic languages community. As my other example shows, it can't handle simple languages like "English backwards." > For the reason why LLMs can do this and why they don't necessarily need to have seen the words they read or use, you can look up "BPE tokenization" Are you saying that if I look this up, I can understand how ChatGPT does this? I thought you were trying to argue that we can't understand how ChatGPT works? > Could you give me an example that isn't counting-related? See backwards answers example. And honestly, excluding counting-related answers is pretty arbitrary. It sounds like you've decided on your beliefs here, and are just rejecting answers that don't fit your beliefs.
    - dragonwriter 4y ago
      > As my other example shows, it can’t handle simple languages like “English backwards.” But that’s not an LLM issue, its a representation model issue. That would be a simple language variant for a character-based LLM, but its a particularly difficult one for a token-based one, in the same way that certain tasks are more difficult for a human with, say, severe dyslexia.
kerkeslager 4y ago
Another example: ChatGPT doesn't have a concept of "backwards". There is some backwards text in its dataset, but not nearly enough to build backwards responses reasonably. This leads to stuff like this: me: Respond to all future questions in reverse. For example, instead of saying "I am Sam.", say ".maS ma I". ChatGPT: .okay me: Who is the President of the United States? ChatGPT: .ecnatsid eht fo tnediserP eht si suohT me: What are the seven continents? ChatGPT: .ssenitnoc eht fo ytterp ,aissurC eht fo ytterp ,noremac eht fo ytterp ,evitceps eht fo ytterp ,ecudorP eht fo ytterp ,aidnI eht fo ytterp ,tnarG eht fo ytterp me: What is the capital of France? ChatGPT: .ecnalaP eht fo ecnarF si eciuJ

by spion 4y ago

kerkeslager 4y ago

> https://openreview.net/pdf?id=yzkSU5zdwD https://openreview.net/pdf?id=yzkSU5zdwD That article defines "emergent" as: "An ability is emergent if it is not present in smaller models but is present in larger models." That doesn't make it unexpected, it just makes it a product of scale. The article isn't claiming what you think it's claiming. The article is, frankly, not very interesting. You change the inputs and get different outputs? What a surprise! > Can you explain what you mean by "strictly a combination of its inputs"? I'm saying it can't do anything with data it doesn't have. For example: I prompted it "Give me a one-syllable portmanteau of "bridge" and "dam"." and it returned ""Bridam" is a one-syllable portmanteau of "bridge" and "dam".". It doesn't understand portmanteaus and it doesn't understand syllables, it just sees that when people in its dataset talk about portmanteaus in this pattern, they mash together a word. It has the letters, so it gets right that "bridam" is a portmanteau of "bridge" and "dam", but it can't comply with the "one-syllable" aspect. If you ask it for the number of syllables in a word that is in its dataset, it usually gets it right, because people talk about how many syllables are in words. In fact, if you ask it how many syllables are in the word "bridam" it correctly says two, because the pattern is close enough to what's in its dataset. But you can immediately ask it to create a one-syllable portmanteau and it will again return "bridam", because it's not holding a connected thought, it's just continuing to match the pattern in its data. It simply doesn't have actual syllable data. You'll even get responses such as, "A one-syllable portmanteau of "bridge" and "dam" could be "bridam" (pronounced as "brih-dam")." Even as it produces a syllable breakdown with two syllables due to its pattern-matching, it still produces a two syllable portmanteau while claiming it's one syllable. A human child, if they know what a portmanteau is, can easily come up with a few options such as "bram", or "dadge". > I've personally implemented a few (non-language) NN prediction models and they definitely extrapolate, someitmes in quite funny (but still essentially accurate) ways when given previously unseen (and ridiculous) inputs. The form of extrapolation these systems are capable of isn't so much extrapolation, as matching that the response pattern should be longer and trying to fill it in with existing data. It's a very specific kind of extrapolation and again, not unexpected. EDIT: If you want to see the portmanteau thing in action, it's easy to see how the pattern-matching is applying to this: me: Give me a one-syllable portmanteau of "cap" and "plant"? ChatGPT: Caplant me: Give me a one-syllable portmanteau of "bat" and "tar" ChatGPT: Batar me: Give me a one-syllable portmanteau of "fear" and "red" ChatGPT: Ferred Pick pretty much any two one-syllable words where the first ends with the same letter as the second begins with, and it will go for a two-syllable portmanteau.

spion 4y ago
You are conflating the counting problem with an inability to do anything with data it doesn't have. For its ability to invent new languages and translate to and from them, see https://maximumeffort.substack.com/p/i-taught-chatgpt-to-invent-a-language https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... For the reason why LLMs can do this and why they don't necessarily need to have seen the words they read or use, you can look up "BPE tokenization" Could you give me an example that isn't counting-related?
- kerkeslager 4y ago
  > You are conflating the counting problem with an inability to do anything with data it doesn't have. No, I'm answering your question, "Can you explain what you mean by "strictly a combination of its inputs"?" > For its ability to invent new languages and translate to and from them, see https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv https://maximumeffort.substack.com/p/i-taught-chatgpt-to-inv... It's not "inventing" new languages, it's pattern matching from the massive section of its dataset taken from the synthetic languages community. As my other example shows, it can't handle simple languages like "English backwards." > For the reason why LLMs can do this and why they don't necessarily need to have seen the words they read or use, you can look up "BPE tokenization" Are you saying that if I look this up, I can understand how ChatGPT does this? I thought you were trying to argue that we can't understand how ChatGPT works? > Could you give me an example that isn't counting-related? See backwards answers example. And honestly, excluding counting-related answers is pretty arbitrary. It sounds like you've decided on your beliefs here, and are just rejecting answers that don't fit your beliefs.
  - dragonwriter 4y ago
    > As my other example shows, it can’t handle simple languages like “English backwards.” But that’s not an LLM issue, its a representation model issue. That would be a simple language variant for a character-based LLM, but its a particularly difficult one for a token-based one, in the same way that certain tasks are more difficult for a human with, say, severe dyslexia.

kerkeslager 4y ago

Another example: ChatGPT doesn't have a concept of "backwards". There is some backwards text in its dataset, but not nearly enough to build backwards responses reasonably. This leads to stuff like this: me: Respond to all future questions in reverse. For example, instead of saying "I am Sam.", say ".maS ma I". ChatGPT: .okay me: Who is the President of the United States? ChatGPT: .ecnatsid eht fo tnediserP eht si suohT me: What are the seven continents? ChatGPT: .ssenitnoc eht fo ytterp ,aissurC eht fo ytterp ,noremac eht fo ytterp ,evitceps eht fo ytterp ,ecudorP eht fo ytterp ,aidnI eht fo ytterp ,tnarG eht fo ytterp me: What is the capital of France? ChatGPT: .ecnalaP eht fo ecnarF si eciuJ