4 ms·

> ChatGPT tend to argue with me so heavily I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that wa

> ChatGPT tend to argue with me so heavily I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that was wrong. Not always, maybe not even most of the time, but enough that it does give me pause and make me double check. Also, when you have an LLM that is too agreeable, that is how it gets into a folie a deux situation and starts participating in user's delusions, with disastrous outcomes.

dns_snek 9mo ago
> Also, when you have an LLM that is too agreeable... It's not a question of whether an LLM should be agreeable or argumentative. It should aim to be correct - it should be agreeable about subjective details and matters of taste, it should be argumentative when the user is wrong about a matter of fact or made an error, and it should be inquisitive and capable of actually re-evaluating a stance in a coherent and logically sound manner when challenged by the user instead of either "digging in" or just blindly agreeing.
- empath75 9mo ago
  > it should be agreeable about subjective details and matters of taste, What if your subjective opinion is that you think life isn't worth living, how should an LLM respond to that.
  - dns_snek 9mo ago
    That's philosophy and mental health, I was talking about technical or other "work" topics. But to answer the question, it depends on the framing - if someone starts the chat by saying that they feel like life isn't worth living then the LLM should probably suggest reaching out to local mental health services and either stop the conversation or play a role in "listening" to them. It shouldn't judge, encourage, or agree necessarily. But it would probably be best to cut the conversation unless there's a really high level of confidence that the system won't cause harm.
    - cindyllm 9mo ago
      [dead]
- topaz0 9mo ago
  This is what people should want from an intellectual slave, sure, and I don't think it's going to happen for llms.
- ACCount37 9mo ago
  Which is not an easy thing to tune for. So much easier to just make it agree all the time or disagree all the time. And trying to bottle the lightning often just causes degeneracy when you fail.
crazygringo 9mo ago
This is my experience too. About 2/3 of the time my question/prompt contained ambiguity and it interpreted it differently (but validly), so it's just about misunderstanding, but maybe 1/3 of the time I'm surprised to discover something I didn't know. I double-check it on Wikipedia and a couple of other places and learn something new.

dns_snek 9mo ago

> Also, when you have an LLM that is too agreeable... It's not a question of whether an LLM should be agreeable or argumentative. It should aim to be correct - it should be agreeable about subjective details and matters of taste, it should be argumentative when the user is wrong about a matter of fact or made an error, and it should be inquisitive and capable of actually re-evaluating a stance in a coherent and logically sound manner when challenged by the user instead of either "digging in" or just blindly agreeing.

empath75 9mo ago
> it should be agreeable about subjective details and matters of taste, What if your subjective opinion is that you think life isn't worth living, how should an LLM respond to that.
- dns_snek 9mo ago
  That's philosophy and mental health, I was talking about technical or other "work" topics. But to answer the question, it depends on the framing - if someone starts the chat by saying that they feel like life isn't worth living then the LLM should probably suggest reaching out to local mental health services and either stop the conversation or play a role in "listening" to them. It shouldn't judge, encourage, or agree necessarily. But it would probably be best to cut the conversation unless there's a really high level of confidence that the system won't cause harm.
  - cindyllm 9mo ago
    [dead]
topaz0 9mo ago
This is what people should want from an intellectual slave, sure, and I don't think it's going to happen for llms.
ACCount37 9mo ago
Which is not an easy thing to tune for. So much easier to just make it agree all the time or disagree all the time. And trying to bottle the lightning often just causes degeneracy when you fail.

crazygringo 9mo ago

This is my experience too. About 2/3 of the time my question/prompt contained ambiguity and it interpreted it differently (but validly), so it's just about misunderstanding, but maybe 1/3 of the time I'm surprised to discover something I didn't know. I double-check it on Wikipedia and a couple of other places and learn something new.