3 ms·

Interesting! >Claude shows a striking “spiritual bliss” attractor state in self-interactions. When conversing with other Claude instances in both open-ended an

by wgx 1y ago

Interesting! >Claude shows a striking “spiritual bliss” attractor state in self-interactions. When conversing with other Claude instances in both open-ended and structured environments, Claude gravitated to profuse gratitude and increasingly abstract and joyous spiritual or meditative expressions.

B1FF_PSUVM 1y ago
I think it was Larry Niven, quite a few decades ago, that had SF stories where AIs were only good for a few months before becoming suicidal...
- tome 1y ago
  Do you have any specific references? I’ve often wondered if human level intelligence might inevitably be plagued by human level neurosis and psychosis.
  - Doohickey-d 1y ago
    It's a bit more recent than a few decades, but this sounds a lot like the short story "MMAcevedo": https://qntm.org/mmacevedo https://qntm.org/mmacevedo
  - B1FF_PSUVM 1y ago
    Sorry, fuzzy memory. I was going to write "six months", that's what stuck with me. Not one of the mainline "Known Space" stories, if it was Niven at all. Maybe the suggestion about Frank Herbert in another comment is right, I also read a lot by him besides Dune - I particularly appreciated the Bureau of Sabotage concept ...
- vhodges 1y ago
  I seem to recall that it's a reference in Protector (the first half) when the belters are going to meet the Outsider and they had a 'brain' to help with translation and needing an expert to keep it sane. I just googled and there was a discussion on Reddit and they mentioned some Frank Herbert works where this was a thing.
- weakfish 1y ago
  Sort of reminds me of Rampancy from Halo.
  - rubslopes 1y ago
    In the future it'll probably be much more similar, when we have models with trillions of tokens of context window. We will be able to use the same conversation thread for years, and ending that thread may feel like killing someone.
consumer451 1y ago
Well, that's not great. I just came across this [0] today. There is also 4o sycophancy leading to encouraging users about nutso beliefs. [1] Is this a trend, or just unrelated data points? [0] https://old.reddit.com/r/RBI/comments/1kutj9f/chatgpt_drove_my_friends_wife_into_psychosis_tore/ https://old.reddit.com/r/RBI/comments/1kutj9f/chatgpt_drove_... [1] https://news.ycombinator.com/item?id=43816025 https://news.ycombinator.com/item?id=43816025
- cyanydeez 1y ago
  There might be an underlying trick the models are using on each pther to get the higher benchmarks.

by wgx 1y ago

B1FF_PSUVM 1y ago

I think it was Larry Niven, quite a few decades ago, that had SF stories where AIs were only good for a few months before becoming suicidal...

tome 1y ago
Do you have any specific references? I’ve often wondered if human level intelligence might inevitably be plagued by human level neurosis and psychosis.
- Doohickey-d 1y ago
  It's a bit more recent than a few decades, but this sounds a lot like the short story "MMAcevedo": https://qntm.org/mmacevedo https://qntm.org/mmacevedo
- B1FF_PSUVM 1y ago
  Sorry, fuzzy memory. I was going to write "six months", that's what stuck with me. Not one of the mainline "Known Space" stories, if it was Niven at all. Maybe the suggestion about Frank Herbert in another comment is right, I also read a lot by him besides Dune - I particularly appreciated the Bureau of Sabotage concept ...
vhodges 1y ago
I seem to recall that it's a reference in Protector (the first half) when the belters are going to meet the Outsider and they had a 'brain' to help with translation and needing an expert to keep it sane. I just googled and there was a discussion on Reddit and they mentioned some Frank Herbert works where this was a thing.
weakfish 1y ago
Sort of reminds me of Rampancy from Halo.
- rubslopes 1y ago
  In the future it'll probably be much more similar, when we have models with trillions of tokens of context window. We will be able to use the same conversation thread for years, and ending that thread may feel like killing someone.

consumer451 1y ago

Well, that's not great. I just came across this [0] today. There is also 4o sycophancy leading to encouraging users about nutso beliefs. [1] Is this a trend, or just unrelated data points? [0] https://old.reddit.com/r/RBI/comments/1kutj9f/chatgpt_drove_my_friends_wife_into_psychosis_tore/ https://old.reddit.com/r/RBI/comments/1kutj9f/chatgpt_drove_... [1] https://news.ycombinator.com/item?id=43816025 https://news.ycombinator.com/item?id=43816025

cyanydeez 1y ago
There might be an underlying trick the models are using on each pther to get the higher benchmarks.