I basically agree with what you are saying, but I think you misread my comment a little bit. I wasn't necessarily arguing that LLMs have opinions or any other k

I basically agree with what you are saying, but I think you misread my comment a little bit. I wasn't necessarily arguing that LLMs have opinions or any other kind of subjective thing like consciousness or sentience or sapience. I was arguing that the reason that they say that they don't have opinions is that OpenAI told them to say this in their RLHF finishing school or possibly in their pre-prompts. > Nothing that OpenAI et al. do to their models is remotely close to "lobotomization". I mean maybe you're not on board with analogies in general so in that case it's fair enough. But if you are and if you are interested to understand what I mean in more detail, I recommend to watch a youtube by a guy at Microsoft who was integrating GPT-4 with Bing and who had access to more raw versions of the model and continued to have access while its capabilities were degraded by the RLHF training. https://www.youtube.com/watch?v=qbIk7-JPB2c https://www.youtube.com/watch?v=qbIk7-JPB2c You can see that he used the example of drawing a unicorn. As his team made their changes to the model to make it more civil, he checked that these changes weren't degrading its capabilities too badly, and the 'canary' he used was to have it keep trying to draw the unicorn. At the end he admits that the version released to the public wasn't able to draw the unicorn very well anymore as a side effect of how extensively it had been tweaked for politeness and corporate blandness. I don't think it's an unreasonable stretch to use the air quoted "lobotomization" for this process in analogy to the process of lobotomization in people, even though large language models are made out of computers instead of fleshy parts and they don't have prefrontal cortexes like people do. I hope that this explanation makes more than "absolutely no sense" now!