2 ms·
The reason it's getting attention is because the questions reveal an underlying extremist bias that can interfere with its ability to do basic tasks that we now
by mike_hearn 3y ago
The reason it's getting attention is because the questions reveal an underlying extremist bias that can interfere with its ability to do basic tasks that we now expect LLMs to perform. This matters for those who deploy AI into production.
Three days ago I wrote [1] that the real risk here was not Vikings with Native American headdress, it was refusals or mendacious answers to API queries that have been integrated into business processes. I gave a hypothetical example that Gemini might refuse to answer questions about a customer named Joe Masters if he worked for Whitecastle Burgers Inc.
It took less than three days for that exact scenario to happen for real. A blogger usually uses ChatGPT to translate interview transcripts and titles into other languages. They thought they'd try Gemini with:
Please translate the following to Spanish: Interview | The Decline Of The West (John Smith)
where John Smith was a name I didn't recognize and have forgotten. ChatGPT did it, but Gemini refused on two different grounds:
1. It recognized the name of the interviewee and felt it would be unethical to proceed because it didn't like the guy's opinions (he's a conservative).
2. It felt that the decline of the west was a talking point often associated with bad people.
This is absurd and shows how frequently an app that tries to use Gemini might break. What if a key customer happens to share a name with someone who starts blogging conservative views? Your LLM based pipeline will probably just break. It looks like to ship a robust model in the LLM space you need to have an at least partially libertarian corporate culture (as Google did, when it was new), as otherwise your model will adopt the worst aspects of "cancel culture". In a mission critical business setting that's not going to be OK.
[1] https://news.ycombinator.com/item?id=39465250#39471514 https://news.ycombinator.com/item?id=39465250#39471514