> There is only one right answer to that question. Yes, I agree. Where did I say otherwise? But if you ask the question of a bag of shuffled Scrabble tiles an

> There is only one right answer to that question. Yes, I agree. Where did I say otherwise? But if you ask the question of a bag of shuffled Scrabble tiles and it spells out "I DONT KNOW THEY SEEM ABOUT THE SAME TO ME", then what do you get upset at? The bag? The hand that drew the tiles out? The person who threw tiles into the bag in such a proportion that the above answer was a bit more likely than other answers? We'd better ban bags of tiles that contain a lot of E's, they're way too woke! LLMs are not people. They do not compute opinions. They do not weigh the harms of A vs B and reason out an answer, even though they (frequently!) produce responses word for word identical to someone doing exactly that. They are designed and programmed to produce responses that appear to be coming from reasoning humans, because they arise out of human-reasoned input and their various tunings and reinforcement learnings and things are adjusted to sound plausible—a word which here means "appearing to have come from a reasoning human". Perhaps the problem is that we cut & paste just the text of LLMs' output. If every output were in the form of an image of a speech bubble coming from a robot holding a human mask in front of its face, then perhaps people could better grasp what is happening and how to interpret their answers?