4 ms·

Anthropic believes in whatever will generate the most hype and revenue. This includes a lot of marketing, including pretend-grassroots spam on HN to convince gu

Anthropic believes in whatever will generate the most hype and revenue. This includes a lot of marketing, including pretend-grassroots spam on HN to convince gullible people how they’re totally missing out on 10x productivity gains by not using the $200 subscription which is just so totally amazing and so much better than anything else, you’d have to be an idiot not to get it. Oh you already did but got mediocre results? You’re holding it wrong. Nobody sane believes the current LLMs are conscious, ffs

strogonoff 9mo ago
I doubt many people believe that LLMs are conscious[0], but if they did—a belief in sentient/conscious LLMs really implies that using LLMs the way we do at scale (doing things equivalent to torture, mass killing, etc.) may qualify as abuse of sentient beings, which would bring down the whole industry. The reason is that there is no working definition of “consciousness” or “sentience” that does not imply “human-like”, which in turn implies ability to feel and suffer, and what we do with LLMs would generally be considered something that would make beings with human-like sentience and consciousness suffer. [0] Some definitely do, though; or at least they behave with LLMs in a way one would behave with a conscious being.
- gavinray 9mo ago
  I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. If you follow the line of thinking that consciousness is an emergent phenomenon, arising out of complexity, it doesn't seem far-fetched to me to believe that someday in the future, a silicon-based computing machine (rather than a biological, carbon-based computing machine) might be "conscious" -- whatever that means.
  - strogonoff 9mo ago
    > I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. It’s circular and self-referential. In defining consciousness we, to phrase it in the least nuanced way, are trying to define a thing through which we define things. The best definition we have reduces to something along the lines of “what we, humans, experience”. By its very nature it makes us unable to fathom or even recognize a hypothetical consciousness if it is entirely unlike ours and/or operates on radically different scales; anything we call “conscious” is implied to be human-like.
- Kim_Bruning 9mo ago
  An LLM will behave the way you treat it, for better or for worse. From an objective, empirical, scientific point of view, consciousness and feelings are not really fantastically defined. But looking at diverse tests that ARE available, modern LLMs seem to get interesting scores on a number of them. The counter-argument being -of course- that no one ever made those tests with LLMs in mind. But that's not something you should come up with post-hoc. Define better experiments instead! (The ethical issues you mention should probably be (re-)evaluated once systems have continuous memory/context)
  - strogonoff 9mo ago
    > An LLM will behave the way you treat it, for better or for worse An LLM produces better output if you treat it badly (threaten violence, gaslight, etc.), which is hardly true for a human unless you count humans in slavery-like conditions.
    - Kim_Bruning 9mo ago
      That's generally not what I find. Do you have a source for that?
      - strogonoff 9mo ago
        Sergey Brin dropped it once, for example[0]: > You know, that’s a weird thing, we don’t circulate this much in the AI community… Not just our models, but all models tend to do better if you threaten them. [0] https://au.lifehacker.com/ai/114236/news/googles-co-founder-says-ai-performs-best-when-you-threaten-it https://au.lifehacker.com/ai/114236/news/googles-co-founder-...
sh3rl0ck 9mo ago
Yeah, I feel Anthropic is just very deliberately theatrical about the way they present their technology and company and even how they price them. Dario's conviction seems too over dramatic to be real to me, but while there's a chance he's drinking his own kool-aid, they just know how to present it as a premium experience, and their developer adoption helps with that.

strogonoff 9mo ago

I doubt many people believe that LLMs are conscious[0], but if they did—a belief in sentient/conscious LLMs really implies that using LLMs the way we do at scale (doing things equivalent to torture, mass killing, etc.) may qualify as abuse of sentient beings, which would bring down the whole industry. The reason is that there is no working definition of “consciousness” or “sentience” that does not imply “human-like”, which in turn implies ability to feel and suffer, and what we do with LLMs would generally be considered something that would make beings with human-like sentience and consciousness suffer. [0] Some definitely do, though; or at least they behave with LLMs in a way one would behave with a conscious being.

gavinray 9mo ago
I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. If you follow the line of thinking that consciousness is an emergent phenomenon, arising out of complexity, it doesn't seem far-fetched to me to believe that someday in the future, a silicon-based computing machine (rather than a biological, carbon-based computing machine) might be "conscious" -- whatever that means.
- strogonoff 9mo ago
  > I think it's difficult to have a serious discussion about consciousness online because it's such a mushy thing to define. It’s circular and self-referential. In defining consciousness we, to phrase it in the least nuanced way, are trying to define a thing through which we define things. The best definition we have reduces to something along the lines of “what we, humans, experience”. By its very nature it makes us unable to fathom or even recognize a hypothetical consciousness if it is entirely unlike ours and/or operates on radically different scales; anything we call “conscious” is implied to be human-like.
Kim_Bruning 9mo ago
An LLM will behave the way you treat it, for better or for worse. From an objective, empirical, scientific point of view, consciousness and feelings are not really fantastically defined. But looking at diverse tests that ARE available, modern LLMs seem to get interesting scores on a number of them. The counter-argument being -of course- that no one ever made those tests with LLMs in mind. But that's not something you should come up with post-hoc. Define better experiments instead! (The ethical issues you mention should probably be (re-)evaluated once systems have continuous memory/context)
- strogonoff 9mo ago
  > An LLM will behave the way you treat it, for better or for worse An LLM produces better output if you treat it badly (threaten violence, gaslight, etc.), which is hardly true for a human unless you count humans in slavery-like conditions.
  - Kim_Bruning 9mo ago
    That's generally not what I find. Do you have a source for that?
    - strogonoff 9mo ago
      Sergey Brin dropped it once, for example[0]: > You know, that’s a weird thing, we don’t circulate this much in the AI community… Not just our models, but all models tend to do better if you threaten them. [0] https://au.lifehacker.com/ai/114236/news/googles-co-founder-says-ai-performs-best-when-you-threaten-it https://au.lifehacker.com/ai/114236/news/googles-co-founder-...

sh3rl0ck 9mo ago

Yeah, I feel Anthropic is just very deliberately theatrical about the way they present their technology and company and even how they price them. Dario's conviction seems too over dramatic to be real to me, but while there's a chance he's drinking his own kool-aid, they just know how to present it as a premium experience, and their developer adoption helps with that.