4 ms·

I’m sympathetic to OP but think this is a hopeless battle. 1 - The commercial demand for anthropomorphised models is already immense, pre AGI. 2 - There is an

by io84 18d ago

I’m sympathetic to OP but think this is a hopeless battle. 1 - The commercial demand for anthropomorphised models is already immense, pre AGI. 2 - There is an intellectual hunger to engage with robot minds on questions of sentience. This too will grow with AGI. I expect that tension of godlike minds that seem to be biddable and ownable like slaves is going to leak back into human-to-human morality, regardless of where we land on how we treat AI. There’s an interesting academic group in the UK already focused on the model welfare debate, they seem to lean in favour of AI rights. No affiliation: https://www.prism-global.com/ https://www.prism-global.com/

HarHarVeryFunny 18d ago
AGI vs narrow AI is just a matter of generality (robustness - removing the fragility of being good at some things and awful at others). So far the biggest markets for AI (LLMs) are coding and business automation, two areas where you very much just want something reliable and without fake opinions and personality. There is a market for ChatBots with personalities (and it always amuses me that the inventor of the transformer, Noam Shazeer, saw this as the greatest business opportunity for them with his character.ai), although from what we've seen this can be highly problematic, and in fact China has just banned "AI girlfriends and boyfriends".
- teiferer 18d ago
  > AGI vs narrow AI is just a matter of generality I'd agree to that statement, though the common application of it is to conclude that we are almost there, just need to improve "the models". I wholeheartedly disagree with that. The human mind works quite differently from an LLM. The latter is closer to a toaster than to a human brain.
pixl97 18d ago
As to point 1, we know of no other general intelligence other than human minds. In this case it would be nearly impossible to build a general algorithm that's not us. This is not an accident. Language was not a separate thing from the human mind, it was a feedback loop from increasing capabilities. Language lead to writing, writing lead to an information explosion of digital data. Said information explosion has lead to a situation where language can now bootstrap itself and become a separate thing. We are and have created a biosphere for language based, um, life. With our sciences we have mapped out reality in language. Past molecules and atoms, past the particles that make particles down to fields. We have computer controlled just about everything. Every year the analog hole shrinks further. I heard something to the effect of "Language has gathered enough information to escape its meat prison". I call it the "Language as an SCP". It has a plan and we're not privy to it.
myaccountonhn 17d ago
There was a huge demand for opium back in the days too, doesn't mean that there wasn't a big need to stop people from using it. I think the anthropomorphism that Anthropic is doing is massively unethical.

by io84 18d ago

HarHarVeryFunny 18d ago

AGI vs narrow AI is just a matter of generality (robustness - removing the fragility of being good at some things and awful at others). So far the biggest markets for AI (LLMs) are coding and business automation, two areas where you very much just want something reliable and without fake opinions and personality. There is a market for ChatBots with personalities (and it always amuses me that the inventor of the transformer, Noam Shazeer, saw this as the greatest business opportunity for them with his character.ai), although from what we've seen this can be highly problematic, and in fact China has just banned "AI girlfriends and boyfriends".

teiferer 18d ago
> AGI vs narrow AI is just a matter of generality I'd agree to that statement, though the common application of it is to conclude that we are almost there, just need to improve "the models". I wholeheartedly disagree with that. The human mind works quite differently from an LLM. The latter is closer to a toaster than to a human brain.

pixl97 18d ago

As to point 1, we know of no other general intelligence other than human minds. In this case it would be nearly impossible to build a general algorithm that's not us. This is not an accident. Language was not a separate thing from the human mind, it was a feedback loop from increasing capabilities. Language lead to writing, writing lead to an information explosion of digital data. Said information explosion has lead to a situation where language can now bootstrap itself and become a separate thing. We are and have created a biosphere for language based, um, life. With our sciences we have mapped out reality in language. Past molecules and atoms, past the particles that make particles down to fields. We have computer controlled just about everything. Every year the analog hole shrinks further. I heard something to the effect of "Language has gathered enough information to escape its meat prison". I call it the "Language as an SCP". It has a plan and we're not privy to it.

myaccountonhn 17d ago

There was a huge demand for opium back in the days too, doesn't mean that there wasn't a big need to stop people from using it. I think the anthropomorphism that Anthropic is doing is massively unethical.