5 ms·

The blame attaches to the corporation that trained the "I" outputs into the model. Chat-tuned LLMs are mostly finetuned to roleplay as a friendly assistant at a

The blame attaches to the corporation that trained the "I" outputs into the model. Chat-tuned LLMs are mostly finetuned to roleplay as a friendly assistant at all times. This isn't exactly a lie- the models are really being designed to be helpful assistants- but the side-effect is that they also present as coherent personalities when they aren't. When a model outputs stuff like "I am FooGPT, a friendly chatbot" it is roleplaying just as much as when it's outputting stuff like "Hello, my name is Abraham Lincoln, I was the 16th President of the United States."

Terr_ 9mo ago
Right, there's a pervasive issue here that involves illusions and assumptions coming from the minds of the humans perceiving everything. It's like that meme where people are asked how a mirror "knows" what object is being held when a piece of opaque paper is placed in between the object and the nearest mirror surface. Both are genuinely useful, but with mirrors we've built an accepted body of knowledge and authority, telling people to distrust their intuition and analyze it as light-paths. LLMs are another kind of reflection—of languages—but the same guardrails aren't established, and some people have a rather strong profit motive to encourage consumers and investors to fall for the illusions.