3 ms·

Nearly everything that has been written on the subject is misleading in that way. People don't write about GPT: they write about GPT personified. The two magi

Nearly everything that has been written on the subject is misleading in that way. People don't write about GPT: they write about GPT personified. The two magic words are, "exhibit behavior". GPT exhibits the behavior of "humans writing language" by implicitly modeling the "already-written-by-humans language" of its training corpus, then using that model to respond to a prompt.

calf 4y ago
In 1st year engineering we learned about the concept of behavioral equivalence, with a digital or analog system you could formally show that two things do the same thing even though their internals are different. If only the debates about ChatGPT had some of that considered nuance instead of anthropomorphizing it, even some linguists seem guilty of this.
- selestify 4y ago
  Isn’t anthromorphization an informal way of asserting behavioral equivalence on some level?
  - thomastjeffery 4y ago
    The problem is when you use the personified character to draw conclusions about the system itself.
  - calf 4y ago
    No because behavioral equivalence is used in systems engineering theory to mathematically prove that two control systems are equivalent. The mathematical proof is complete, e.g. for all internals state transitions and the cross product of the two machines. With anthropormization there is zero amount of that rigor, which lets people use sloppy arguments about what ChatGPT is doing and isn't doing.
TillE 4y ago
Right, anthropomorphization is the biggest source of confusion here. An LLM gives you a perfect answer to a complex question and you think wow, it really "understood" my question. But no! It doesn't understand, it doesn't reason, these are concepts wholly absent from its fundamental design. It can do really cool things despite the fact that it's essentially just a text generator. But there's a ceiling to what can be accomplished with that approach.
- thomastjeffery 4y ago
  It's presented as a feature when GPT provides a correct answer. It's presented as a limitation when GPT provides an incorrect answer. Both of these behaviors are literally the same. We are sorting them into the subjective categories of "right" and "wrong" after the fact. GPT is fundamentally incapable of modeling that difference. A "right answer" is every bit as valid as a "wrong answer". The two are equivalent in what GPT is modeling. Lies are a valid feature of language. They are shaped the same as truths. The only way to resolve this problem is brute force: provide every unique construction of a question, and the corresponding correct answer to that construction.
  - LawTalkingGuy 4y ago
    Not entirely. It's modeling a completion in a given context. That language model "understands" that if one party stops speaking, the other party generally starts, etc. It also "understands" that if someone says something 'wrong' the other party often mentions it, which makes the first party respond thusly, and so forth. If you ask it what the outcome of a lie is on the conversation it can generally answer. If you ask it for a sample conversation where someone is factually incorrect, or lying, and caught out, it can generate it. If you give it a fact and ask it to lie about that fact, it will.
    - thomastjeffery 4y ago
      It doesn't "understand" those interactions: it models them to be repeated later. But the way it chooses what interaction to repeat is not by logical decision: it's by semantic distance. The meaning of semantic distance usually leads to the correct path, but sometimes that pattern is ambiguous.

calf 4y ago

In 1st year engineering we learned about the concept of behavioral equivalence, with a digital or analog system you could formally show that two things do the same thing even though their internals are different. If only the debates about ChatGPT had some of that considered nuance instead of anthropomorphizing it, even some linguists seem guilty of this.

selestify 4y ago
Isn’t anthromorphization an informal way of asserting behavioral equivalence on some level?
- thomastjeffery 4y ago
  The problem is when you use the personified character to draw conclusions about the system itself.
- calf 4y ago
  No because behavioral equivalence is used in systems engineering theory to mathematically prove that two control systems are equivalent. The mathematical proof is complete, e.g. for all internals state transitions and the cross product of the two machines. With anthropormization there is zero amount of that rigor, which lets people use sloppy arguments about what ChatGPT is doing and isn't doing.

TillE 4y ago

Right, anthropomorphization is the biggest source of confusion here. An LLM gives you a perfect answer to a complex question and you think wow, it really "understood" my question. But no! It doesn't understand, it doesn't reason, these are concepts wholly absent from its fundamental design. It can do really cool things despite the fact that it's essentially just a text generator. But there's a ceiling to what can be accomplished with that approach.

thomastjeffery 4y ago
It's presented as a feature when GPT provides a correct answer. It's presented as a limitation when GPT provides an incorrect answer. Both of these behaviors are literally the same. We are sorting them into the subjective categories of "right" and "wrong" after the fact. GPT is fundamentally incapable of modeling that difference. A "right answer" is every bit as valid as a "wrong answer". The two are equivalent in what GPT is modeling. Lies are a valid feature of language. They are shaped the same as truths. The only way to resolve this problem is brute force: provide every unique construction of a question, and the corresponding correct answer to that construction.
- LawTalkingGuy 4y ago
  Not entirely. It's modeling a completion in a given context. That language model "understands" that if one party stops speaking, the other party generally starts, etc. It also "understands" that if someone says something 'wrong' the other party often mentions it, which makes the first party respond thusly, and so forth. If you ask it what the outcome of a lie is on the conversation it can generally answer. If you ask it for a sample conversation where someone is factually incorrect, or lying, and caught out, it can generate it. If you give it a fact and ask it to lie about that fact, it will.
  - thomastjeffery 4y ago
    It doesn't "understand" those interactions: it models them to be repeated later. But the way it chooses what interaction to repeat is not by logical decision: it's by semantic distance. The meaning of semantic distance usually leads to the correct path, but sometimes that pattern is ambiguous.