3 ms·

While I haven't experienced LLMs correcting most (or any) of the problems listed fully and consistently, I do agree that consistent use of LLMs and dealing with

While I haven't experienced LLMs correcting most (or any) of the problems listed fully and consistently, I do agree that consistent use of LLMs and dealing with their frustrations has worn my patience for conversations with people who exhibit the same issues when talking. It's kind of depressing. I just want the LLM to be a bot that responds to what I say with a useful response. However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things making it even more grating to interact with them which chews away at my normal interpersonal patience which, as someone on the spectrum, was already limited.

acedTrex 9mo ago
This is why i simply do not bother with them unless the task i need is so specific that theres no room for argument, like yesterday i asked it to generate me a bash script that ran aws ssm commands for all the following instance IDs. It did that as a two shot. But long conversations are never worth it.
empath75 9mo ago
> ChatGPT tend to argue with me so heavily I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that was wrong. Not always, maybe not even most of the time, but enough that it does give me pause and make me double check. Also, when you have an LLM that is too agreeable, that is how it gets into a folie a deux situation and starts participating in user's delusions, with disastrous outcomes.
- dns_snek 9mo ago
  > Also, when you have an LLM that is too agreeable... It's not a question of whether an LLM should be agreeable or argumentative. It should aim to be correct - it should be agreeable about subjective details and matters of taste, it should be argumentative when the user is wrong about a matter of fact or made an error, and it should be inquisitive and capable of actually re-evaluating a stance in a coherent and logically sound manner when challenged by the user instead of either "digging in" or just blindly agreeing.
  - empath75 9mo ago
    > it should be agreeable about subjective details and matters of taste, What if your subjective opinion is that you think life isn't worth living, how should an LLM respond to that.
    - dns_snek 9mo ago
      That's philosophy and mental health, I was talking about technical or other "work" topics. But to answer the question, it depends on the framing - if someone starts the chat by saying that they feel like life isn't worth living then the LLM should probably suggest reaching out to local mental health services and either stop the conversation or play a role in "listening" to them. It shouldn't judge, encourage, or agree necessarily. But it would probably be best to cut the conversation unless there's a really high level of confidence that the system won't cause harm.
      - cindyllm 9mo ago
        [dead]
  - topaz0 9mo ago
    This is what people should want from an intellectual slave, sure, and I don't think it's going to happen for llms.
  - ACCount37 9mo ago
    Which is not an easy thing to tune for. So much easier to just make it agree all the time or disagree all the time. And trying to bottle the lightning often just causes degeneracy when you fail.
- crazygringo 9mo ago
  This is my experience too. About 2/3 of the time my question/prompt contained ambiguity and it interpreted it differently (but validly), so it's just about misunderstanding, but maybe 1/3 of the time I'm surprised to discover something I didn't know. I double-check it on Wikipedia and a couple of other places and learn something new.
deleted 9mo ago
[deleted]
bicepjai 9mo ago
>>> … However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things … This is something I have not experienced. Can you provide examples ?
agloe_dreams 9mo ago
Yeah this is exactly opposite my issue with LLMs. They often take what you say as the truth when it absolutely could not be.
rguzman 9mo ago
> However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things do you have examples of this? asking because this is not what happens to me. one of the main things i worry about when interacting with the llm is that they agree with me too easily.
mikasisiki 9mo ago
There was a period when coding agents would always agree with you, even if you gave them a really bad idea. They’d always start with something like, “You’re right — I should…”. Back then, what we actually wanted was for them to push back and argue with us.
GuB-42 9mo ago
I have taken the stance to not argue with LLMs, don't give them any clues, and don't ask them to roleplay. Tell them no more than what they need to know. And if they get the answer wrong, don't try to correct them or guide them, there is a high chance they don't have the answer and what follow will be hallucinations. You can ask for details, but don't try to go against it, it will just assume you are right (even if you are not) and hallucinate around that. Keep what you already know to yourself. As for the "you are an expert" prompts, it will mostly just make the LLM speak more authoritatively, but it doesn't mean it will be more correct. My strategy is now to give the LLM as much freedom as it can get, it may not be the best way to extract all the knowledge it has, but it helps spot hallucinations. You can argue with actual people, if both of you are open enough, something greater make come out of it, but if not, it is useless, and with LLMs it is always useless, they are pretrained, they won't get better in the future because that little conversation sparked their interest. And on your side, you will just have your own points rephrased and sent back to you, and that will just put you deeper in your own bubble.
- eru 9mo ago
  What's the purpose of your stance? What are you trying to achieve?
okwhateverdude 9mo ago
> However, for some reason, both Gemini and ChatGPT tend to argue with me The trick here is: "Be succinct. No commentary." And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified" Also generally the very first time it expresses any of that weird shit, your context is toast. So even correcting it is reinforcing. Just regenerate the response.
- CamperBob2 9mo ago
  And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified" Last time I bawled out an LLM and forced it to change its mind, I later realized that the LLM was right the first time. One of those "Who am I and how did I end up in this hole in the ground, and where did all these carrots and brightly-colored eggs come from?" moments, of the sort that seem to be coming more and more frequently lately.
  - Aerbil313 9mo ago
    Yeah, same. Lately almost every time I think "Oh no way, this is not the correct way/not the optimal way/it's a hallucination" it later turns out that it's actually the correct way/the optimal way/it's not a hallucination. I now think twice before doing anything differently than what the LLM tells me unless I'm an expert on the subject and can already spot mistakes easily. It seems like they really figured out grounding and the like in the last couple of months.
    - eru 9mo ago
      I wouldn't worry too much about these false negatives: your human friends might be cross if you constantly accuse them of being wrong when they are actually right, but the LLMs are too polite to hold a grudge.

acedTrex 9mo ago

This is why i simply do not bother with them unless the task i need is so specific that theres no room for argument, like yesterday i asked it to generate me a bash script that ran aws ssm commands for all the following instance IDs. It did that as a two shot. But long conversations are never worth it.

empath75 9mo ago

> ChatGPT tend to argue with me so heavily I have found that quite often when ChatGPT digs in on something, that it is in fact right, and I was the one that was wrong. Not always, maybe not even most of the time, but enough that it does give me pause and make me double check. Also, when you have an LLM that is too agreeable, that is how it gets into a folie a deux situation and starts participating in user's delusions, with disastrous outcomes.

dns_snek 9mo ago
> Also, when you have an LLM that is too agreeable... It's not a question of whether an LLM should be agreeable or argumentative. It should aim to be correct - it should be agreeable about subjective details and matters of taste, it should be argumentative when the user is wrong about a matter of fact or made an error, and it should be inquisitive and capable of actually re-evaluating a stance in a coherent and logically sound manner when challenged by the user instead of either "digging in" or just blindly agreeing.
- empath75 9mo ago
  > it should be agreeable about subjective details and matters of taste, What if your subjective opinion is that you think life isn't worth living, how should an LLM respond to that.
  - dns_snek 9mo ago
    That's philosophy and mental health, I was talking about technical or other "work" topics. But to answer the question, it depends on the framing - if someone starts the chat by saying that they feel like life isn't worth living then the LLM should probably suggest reaching out to local mental health services and either stop the conversation or play a role in "listening" to them. It shouldn't judge, encourage, or agree necessarily. But it would probably be best to cut the conversation unless there's a really high level of confidence that the system won't cause harm.
    - cindyllm 9mo ago
      [dead]
- topaz0 9mo ago
  This is what people should want from an intellectual slave, sure, and I don't think it's going to happen for llms.
- ACCount37 9mo ago
  Which is not an easy thing to tune for. So much easier to just make it agree all the time or disagree all the time. And trying to bottle the lightning often just causes degeneracy when you fail.
crazygringo 9mo ago
This is my experience too. About 2/3 of the time my question/prompt contained ambiguity and it interpreted it differently (but validly), so it's just about misunderstanding, but maybe 1/3 of the time I'm surprised to discover something I didn't know. I double-check it on Wikipedia and a couple of other places and learn something new.

deleted 9mo ago

[deleted]

bicepjai 9mo ago

>>> … However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things … This is something I have not experienced. Can you provide examples ?

agloe_dreams 9mo ago

Yeah this is exactly opposite my issue with LLMs. They often take what you say as the truth when it absolutely could not be.

rguzman 9mo ago

> However, for some reason, both Gemini and ChatGPT tend to argue with me so heavily and inject their own weird stupid ideas on things do you have examples of this? asking because this is not what happens to me. one of the main things i worry about when interacting with the llm is that they agree with me too easily.

mikasisiki 9mo ago

There was a period when coding agents would always agree with you, even if you gave them a really bad idea. They’d always start with something like, “You’re right — I should…”. Back then, what we actually wanted was for them to push back and argue with us.

GuB-42 9mo ago

I have taken the stance to not argue with LLMs, don't give them any clues, and don't ask them to roleplay. Tell them no more than what they need to know. And if they get the answer wrong, don't try to correct them or guide them, there is a high chance they don't have the answer and what follow will be hallucinations. You can ask for details, but don't try to go against it, it will just assume you are right (even if you are not) and hallucinate around that. Keep what you already know to yourself. As for the "you are an expert" prompts, it will mostly just make the LLM speak more authoritatively, but it doesn't mean it will be more correct. My strategy is now to give the LLM as much freedom as it can get, it may not be the best way to extract all the knowledge it has, but it helps spot hallucinations. You can argue with actual people, if both of you are open enough, something greater make come out of it, but if not, it is useless, and with LLMs it is always useless, they are pretrained, they won't get better in the future because that little conversation sparked their interest. And on your side, you will just have your own points rephrased and sent back to you, and that will just put you deeper in your own bubble.

eru 9mo ago
What's the purpose of your stance? What are you trying to achieve?

okwhateverdude 9mo ago

> However, for some reason, both Gemini and ChatGPT tend to argue with me The trick here is: "Be succinct. No commentary." And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified" Also generally the very first time it expresses any of that weird shit, your context is toast. So even correcting it is reinforcing. Just regenerate the response.

CamperBob2 9mo ago
And sometimes a healthy dose of expressing frustration or anger (cursing, berating, threatening) also gets them to STFU and do the thing. As in literally: "I don't give a fuck about your stupid fucking opinions on the matter. Do it exactly as I specified" Last time I bawled out an LLM and forced it to change its mind, I later realized that the LLM was right the first time. One of those "Who am I and how did I end up in this hole in the ground, and where did all these carrots and brightly-colored eggs come from?" moments, of the sort that seem to be coming more and more frequently lately.
- Aerbil313 9mo ago
  Yeah, same. Lately almost every time I think "Oh no way, this is not the correct way/not the optimal way/it's a hallucination" it later turns out that it's actually the correct way/the optimal way/it's not a hallucination. I now think twice before doing anything differently than what the LLM tells me unless I'm an expert on the subject and can already spot mistakes easily. It seems like they really figured out grounding and the like in the last couple of months.
  - eru 9mo ago
    I wouldn't worry too much about these false negatives: your human friends might be cross if you constantly accuse them of being wrong when they are actually right, but the LLMs are too polite to hold a grudge.