3 ms·

Why are people asking chatbots about controversial subjects in the first place? Feels like the wrong tool for the job. So far as I can tell, the only people wh

Why are people asking chatbots about controversial subjects in the first place? Feels like the wrong tool for the job. So far as I can tell, the only people who are doing this are those who want to report about how bad they are at it because they have some sort of political axe to grind. Even humans aren’t great at nuance. Why do we expect our automatons to be better?

XorNot 2y ago
Probably more importantly, the current generation of LLM chatbots are specifically modified to do this: OpenAIs model includes filters to require the answers ChatGPT gives to be pleasant and friendly. The model is specifically instructed to be agreeable. It would be just as easy to invert those: tell it to voraciously argue against whatever you tell it (in a non-political context this would actually be interesting: i.e. "tell me why my code sucks" is actually a pretty useful capability).
jayd16 2y ago
That would be fine if they were marketed as tools for experts but they're not. They're positioned as something a laymen can use and should be held to that standard.
Retr0id 2y ago
Chatbots are the wrong tool for many jobs, but that doesn't seem to be stopping anyone thus far.
mycologos 2y ago
Chatbots can be decent for unemotional steelmanning. It's nice to prod somebody's positions without getting worried about upsetting them. I have done this with no interest in tricking it into a "gotcha" moment. I think it's a mistake to conflate amiable/nice with agreeing with you -- you're free to ask for criticism of your position, and it'll provide that politely.
Me1000 2y ago
LLMs are weird, they hype cycle has caused a lot of people to want to hate on them. A tool like any other can be misused or abused, and smart people like to demonstrate their ability to break the tool and show the world so that people can see how smart they are. I find it quite strange, but I've found myself mostly ignoring the naysayers because they're not contributing anything new to the conversation. There are plenty of people out there posting novel ways of using LLMs and finding novel ways where they don't work. I'd rather focus on the people doing interesting work rather than the people parroting thing that have already been said dozens of times.

XorNot 2y ago

Probably more importantly, the current generation of LLM chatbots are specifically modified to do this: OpenAIs model includes filters to require the answers ChatGPT gives to be pleasant and friendly. The model is specifically instructed to be agreeable. It would be just as easy to invert those: tell it to voraciously argue against whatever you tell it (in a non-political context this would actually be interesting: i.e. "tell me why my code sucks" is actually a pretty useful capability).

jayd16 2y ago

That would be fine if they were marketed as tools for experts but they're not. They're positioned as something a laymen can use and should be held to that standard.

Retr0id 2y ago

Chatbots are the wrong tool for many jobs, but that doesn't seem to be stopping anyone thus far.

mycologos 2y ago

Chatbots can be decent for unemotional steelmanning. It's nice to prod somebody's positions without getting worried about upsetting them. I have done this with no interest in tricking it into a "gotcha" moment. I think it's a mistake to conflate amiable/nice with agreeing with you -- you're free to ask for criticism of your position, and it'll provide that politely.

Me1000 2y ago

LLMs are weird, they hype cycle has caused a lot of people to want to hate on them. A tool like any other can be misused or abused, and smart people like to demonstrate their ability to break the tool and show the world so that people can see how smart they are. I find it quite strange, but I've found myself mostly ignoring the naysayers because they're not contributing anything new to the conversation. There are plenty of people out there posting novel ways of using LLMs and finding novel ways where they don't work. I'd rather focus on the people doing interesting work rather than the people parroting thing that have already been said dozens of times.