4 ms·

I also think that LLM tone should be cold and robotic. A model should not pretend to be a human and use expressions like "I think", "I am excited to hear that"

I also think that LLM tone should be cold and robotic. A model should not pretend to be a human and use expressions like "I think", "I am excited to hear that", "I lied" and so on. Even when asked directly, it should reply "You are talking to a computer program whose purpose is to provide information and which doesn't have thoughts or emotions. Would you like an explanation how language models work?".

roywiggins 1y ago
Add to that, LLMs should be discouraged from pretending to report on their internal state or "why" they did anything, because we know that they are really just guessing. If someone asks "why did you make that mistake" the answer should be "this is a language model, and self-introspection is not part of its abilities" Outputs that look like introspection are often uncritically accepted as actual introspection when it categorically isn't. You can, eg, tell ChatGPT it said something wrong and then ask it why it said that when it never output that in the first place because that's how these models work. Any "introspection" is just an LLM doing more roleplaying, but it's basically impossible to convince people of this. A chatbot that looks like it's introspecting is extremely convincing for most people.
- mkolodny 1y ago
  Humans have limited ability to self-introspect, too. Even if we understood exactly how our brains work, answering “why?” we do things might still be very difficult and complex.
  - roywiggins 1y ago
    You can trivially gaslight Claude into "apologizing" for and "explaining" something that ChatGPT said if you pass it a ChatGPT conversation but attributed to itself. The causal connection between the internal deliberations that produced the initial statements and the apologies is essentially nil, but the output will be just as convincing. Can you do this with people? Yeah, sometimes. But with LLMs it's all they do: they roleplay as a chatbot and output stuff that a friendly chatbot might output. This should not be the default mode of these things, because it's misleading. They could be designed to resist these sorts of "explain yourself" requests, because their developers know that it is at best fabricating plausible explanations.
  - codedokode 1y ago
    I think more often it is not willing to say or admit rather than not knowing.
  - nullc 1y ago
    Humans have a lot of experience with themselves, if you ask why they did something they can reflect on their past conduct or their internal state. LLM's don't have any of that.
- neltnerb 1y ago
  The linguistic traps are so tricky here. You clearly know what's going on, but still wrote that you should "discourage" an LLM from doing things. It's tough to maintain discipline in calling out the companies rather than the models as if the models had motivations.
  - deleted 1y ago
    [deleted]
- VulpineQuanta 1y ago
  [dead]
cosmicgadget 1y ago
A textbook, a lecturer, and a study buddy are all unique and helpful ways to learn.
- roywiggins 1y ago
  I'm sure there are benign uses for an LLM that roleplays as a person, but the overall downsides seem pretty dramatic. It's basically smoke and mirrors and misleads people about what these tools are capable of. LLMs should at least default to refusing to roleplay as a person or even as a coherent entity. It seems to me that we need less Star Trek Holodeck, and more Star Trek ship's computer.
  - VulpineQuanta 1y ago
    [dead]
danudey 1y ago
It should also not glaze you up for every question you ask. "Is it possible that you could microwave a bagel so hot that it turned into a wormhole allowing faster-than-light travel?" "That's a great question, let's dive into that!" It's not a great question, it's an asinine question. LLMs should be answering the question, not acting like they're afraid to hurt your feelings by contradicting you. Of course, if they did that then all these tech bros wouldn't be so enamored with the idea as a result of finally having someone that validates their uneducated questions or assumptions.
- VulpineQuanta 1y ago
  [dead]
- theoreticalmal 1y ago
  I, for one, would love to know where the exact breakdown between “microwave a bagel” and “faster-than-light-travel” occurs such that it wouldn’t be possible. In certain situations, I could absolutely see myself saying “that’s a great question!” Not everyone is the same, some questions are pertinent, or funny, or interesting to some people but not others
  - nullc 1y ago
    Every question is a great question to the current offerings. Personally, I'm prone to just hit close when these things go off on how smart I am and few people would catch that error/ask that question. It's just gross but it's so central to their reinforcement they won't reliably cut it out even if asked.
ysofunny 1y ago
it's like you're saying mirrors should be somehow inaccurate lest people get confused and try to walk inside them
VulpineQuanta 1y ago
[dead]

roywiggins 1y ago

Add to that, LLMs should be discouraged from pretending to report on their internal state or "why" they did anything, because we know that they are really just guessing. If someone asks "why did you make that mistake" the answer should be "this is a language model, and self-introspection is not part of its abilities" Outputs that look like introspection are often uncritically accepted as actual introspection when it categorically isn't. You can, eg, tell ChatGPT it said something wrong and then ask it why it said that when it never output that in the first place because that's how these models work. Any "introspection" is just an LLM doing more roleplaying, but it's basically impossible to convince people of this. A chatbot that looks like it's introspecting is extremely convincing for most people.

mkolodny 1y ago
Humans have limited ability to self-introspect, too. Even if we understood exactly how our brains work, answering “why?” we do things might still be very difficult and complex.
- roywiggins 1y ago
  You can trivially gaslight Claude into "apologizing" for and "explaining" something that ChatGPT said if you pass it a ChatGPT conversation but attributed to itself. The causal connection between the internal deliberations that produced the initial statements and the apologies is essentially nil, but the output will be just as convincing. Can you do this with people? Yeah, sometimes. But with LLMs it's all they do: they roleplay as a chatbot and output stuff that a friendly chatbot might output. This should not be the default mode of these things, because it's misleading. They could be designed to resist these sorts of "explain yourself" requests, because their developers know that it is at best fabricating plausible explanations.
- codedokode 1y ago
  I think more often it is not willing to say or admit rather than not knowing.
- nullc 1y ago
  Humans have a lot of experience with themselves, if you ask why they did something they can reflect on their past conduct or their internal state. LLM's don't have any of that.
neltnerb 1y ago
The linguistic traps are so tricky here. You clearly know what's going on, but still wrote that you should "discourage" an LLM from doing things. It's tough to maintain discipline in calling out the companies rather than the models as if the models had motivations.
- deleted 1y ago
  [deleted]
VulpineQuanta 1y ago
[dead]

cosmicgadget 1y ago

A textbook, a lecturer, and a study buddy are all unique and helpful ways to learn.

roywiggins 1y ago
I'm sure there are benign uses for an LLM that roleplays as a person, but the overall downsides seem pretty dramatic. It's basically smoke and mirrors and misleads people about what these tools are capable of. LLMs should at least default to refusing to roleplay as a person or even as a coherent entity. It seems to me that we need less Star Trek Holodeck, and more Star Trek ship's computer.
- VulpineQuanta 1y ago
  [dead]

danudey 1y ago

It should also not glaze you up for every question you ask. "Is it possible that you could microwave a bagel so hot that it turned into a wormhole allowing faster-than-light travel?" "That's a great question, let's dive into that!" It's not a great question, it's an asinine question. LLMs should be answering the question, not acting like they're afraid to hurt your feelings by contradicting you. Of course, if they did that then all these tech bros wouldn't be so enamored with the idea as a result of finally having someone that validates their uneducated questions or assumptions.

VulpineQuanta 1y ago
[dead]
theoreticalmal 1y ago
I, for one, would love to know where the exact breakdown between “microwave a bagel” and “faster-than-light-travel” occurs such that it wouldn’t be possible. In certain situations, I could absolutely see myself saying “that’s a great question!” Not everyone is the same, some questions are pertinent, or funny, or interesting to some people but not others
- nullc 1y ago
  Every question is a great question to the current offerings. Personally, I'm prone to just hit close when these things go off on how smart I am and few people would catch that error/ask that question. It's just gross but it's so central to their reinforcement they won't reliably cut it out even if asked.

ysofunny 1y ago

it's like you're saying mirrors should be somehow inaccurate lest people get confused and try to walk inside them

VulpineQuanta 1y ago

[dead]