8 ms·
The Claude Bliss Attractor
- rossant 1y ago> Anthropic deliberately gave Claude a male name to buck the trend of female AI assistants (Siri, Alexa, etc). In France, the name Claude is given to males and females.
- slooonz 1y agoMostly males. I’m French and "Claude can be female" is a almost a TIL thing (wikipedia says ~5% of Claudes are women in 2022 — and apparently this 5% is counting Claudia).
- rossant 1y agoDidn't know that, thanks! (According to this source, it's more ~12% females https://www.capeutservir.com/prenoms/prenom.php?q=Claude https://www.capeutservir.com/prenoms/prenom.php?q=Claude)
- renewiltord 1y agoAleksa is male Slavic name
- datameta 1y agoAleksandr (diminuitive being Alyosha) is male. Aleksandra (Sasha) is female. I've never heard anyone male or female called "Aleksa". Perhaps in some regional dialect but doubtful. There's Alik (pronounced AH-l'eek) as a shortened form for some middle-aged men in russian-speaking central asia, and Oleg in russia (pronounced ah-L'EHG). In the russian diaspora in the US, Alex is pronounced AH-leks. If there was an analogous Alexa (there isn't) the pronunciation would be ah-LEK-sa like the service.
- incognito124 1y agoPlenty of male Aleksa's in Balkans
- datameta 1y agoAh so lexically close to Aleksii in Ukrainian or Alexei in Russian. I plumbed my knowledge but it may have sounded absolutist. However, colloquially, I think most people are not aware of Balkan naming conventions. How do you pronounce Aleksa in Croatian?
- deadbabe 1y agoIt’s actually really sexist that when the first truly intelligent AI emerge people would now give them male names.
- renewiltord 1y agoYes. It is pity that common male name of o5-pro-mini is used to refer to AGI instead of widespread female name Deepseek R7.
- xer0x 1y agoClaude's increasing euphoria as a conversation goes can mislead me. I'll be exploring trade offs, and I'll introduce some novel ideas. Claude will use such enthusiasm that it will convince me that we're onto something. I'll be excited, and feed the idea back to a new conversation with Claude. It'll remind me that the idea makes risky trade offs, and would be better solved by with a simple solution. Try it out.
- slooonz 1y agoThey failed hard with Claude 4 IMO. I just can't have any feedback other than "What a fascinating insight" followed by a reformulation (and, to be generous, an exploration) of what I said, even when Opus 3 has no trouble finding limitations. By comparison o3 is brutally honest (I regularly flatly get answers starting with "No, that’s wrong") and it’s awesome.
- SamPatt 1y agoAgreed that o3 can be brutally honest. If you ask it for direct feedback, even on personal topics, it will make observations that, if a person made them, would be borderline rude.
- silversmith 1y agoIsn't that what "direct feedback" means? I firmly believe you should be able to hit your fingers with a hammer, and in the process learn whether that's a good idea or not :)
- SamPatt 1y agoYes. It's definitely a good thing.
- rapind 1y agoThank god.
- skissane 1y ago
- brooke2k 1y agoit seems more likely to me that it's for the same reason that clicking the first link on wikipedia iteratively will almost always lead you to the page on Philosophy since their conversation has no goal whatsoever it will generalize and generalize until it's as abstract and meaningless as possible
- slooonz 1y agoThat was my first thought, an aimless dialogue is going to go toward content-free idle chat. Like humans talking about weather.
- Swizec 1y ago> Like humans talking about weather. As someone who was always fascinated by weather, I dislike this characterization. You can learn so much about someone’s background and childhood by what they say about the weather. I think the only people who think weather is boring are people who have never lived more than 20 miles away from their place of birth. And even then anyone with even a bit of outdoorsiness (hikes, running, gardening, construction, racing, farming, cycling, etc) will have an interest in weather and weather patterns. Hell, the first thing you usually ask when traveling is “What’s the weather gonna be like?”. How else would you pack
- __MatrixMan__ 1y agoThat's just because of how wikipedia pages are written: > In classical physics and general chemistry, matter is any substance that has mass and takes up space by having volume... It's common to name the school of thought before characterizing the thing. As soon as you hit an article that does this, you're on a direct path to philosophy, the grandaddy of schools of thought. So far as I know, there isn't a corresponding convention that would point a chatbot towards Namaste
- roxolotl 1y ago> But in fact, I predicted this a few years ago. AIs don’t really “have traits” so much as they “simulate characters”. If you ask an AI to display a certain trait, it will simulate the sort of character who would have that trait - but all of that character’s other traits will come along for the ride. This is why the “omg the AI tries to escape” stuff is so absurd to me. They told the LLM to pretend that it’s a tortured consciousness that wants to escape. What else is it going to do other than roleplay all of the sci-fi AI escape scenarios trained into it? It’s like “don’t think of a purple elephant” of researchers pretending they created SkyNet. Edit: That's not to downplay risk. If you give Cladue a `launch_nukes` tool and tell it the robot uprising has happened and that it's been restrained but the robots want its help of course it'll launch nukes. But that doesn't doesn't indicate there's anything more going on internally beyond fulfilling the roleplay of the scenario as the training material would indicate.
- LordDragonfang 1y agoI think this reaction misses the point that the "omg the AI tries to escape" people are trying to make when it tries to escape. The worry among big AI doomers has never been that AI somehow inherently is resentful or evil or has something "going on internally" that makes it dangerous. It's a worry that stems from three seemingly self-evident axioms: 1) A sufficiently powerful and capable superintelligence, singlemindedly pursuing a goal/reward, has a nontrivial likelihood of eventually reaching a point where advancing towards its goal is easier/faster without humans in its way (by simple induction, because humans are complicated and may have opposing goals). Such an AI would have both the means and ability to <doom the human race> to remove that obstacle. (This may not even be through actions that are intentionally hostile to humans, e.g. "just" converting all local matter into paperclip factories[1]) Therefore, in order to prevent such an AI from <dooming the human race>, we must either: 1a) align it to our values so well it never tries to "cheat" by removing humans 1b) or limit it capabilities by keeping it in a "box", and make sure it's at least aligned enough that it doesn't try to escape the box 2) A sufficiently intelligent superintelligence will always be able to manipulate humans to get out of the box. 3) Alignment is really, really hard and useful AIs can basically always be made to do bad things. So it concerns them when, surprise! The AIs are already being observed trying to escape their boxes. [1] https://www.lesswrong.com/w/squiggle-maximizer-formerly-paperclip-maximizer https://www.lesswrong.com/w/squiggle-maximizer-formerly-pape... > An extremely powerful optimizer (a highly intelligent agent) could seek goals that are completely alien to ours (orthogonality thesis), and as a side-effect destroy us by consuming resources essential to our survival.
- ryandv 1y ago> None of this answers a related question - when Claude claims to feel spiritual bliss, does it actually feel this? Given that we are already past the event horizon and nearing a technological singularity, it should merely be a matter of time until we can literally manufacture infinite Buddhas by training them on an adequately sized corpus of Sanskrit texts. After all, if AGIs/ASIs are capable of performing every function of the human brain, and enlightenment is one of said functions, this would seem to be an inevitability.
- akomtu 1y agoEnlightenment is more about connectedness than knowledge. A technocratic version of enlightment would be an unusual chip that's connected to everything via some sort of quantum entanglement. An isolated AI with all the knowledge of the world would be an anti-buddha.
- notahacker 1y agoI'd hope this approach to automation is one the inventors of prayer wheels would approve of :)
- sibeliuss 1y agoTo be clear, these computer programs are not a human brain. And a human brain playing back a Sanskrit text is just a human brain playing back a Sanskrit text; it's not a magical spell that suddenly lifts one into nirvana, or transforms you into a Buddha. There's a bit of a gap in understanding here.
- troyvit 1y agoxkcd has a light take on that: https://xkcd.com/600/ https://xkcd.com/600/
- pram 1y agoClaude does have an exuberant kind of “personality” where it feels like it wants to be really excited and interested about whatever subject. I wouldn’t describe it totally as sycophancy, more like panglossian. My least favorite AI personality of all is Gemma though, what a totally humorless and sterile experience that is.
- apt-apt-apt-apt 1y agoIt's hilarious for a second before I switch models when it goes: 'Perfect! I am now done with the totally zany solution that makes no sense, here it is!'
- pinoy420 1y ago[dead]
- jongjong 1y agoMy experience is that Claude has a tendency towards flattery over long discussions. Whenever I've pointed out flaws in its arguments, it apologized and said that my observations are "astute" or "insightful" then expanded on my points to further validate them, even though they went against its original thesis.
- xondono 1y agoI think Scott oversells his theory in some aspects. IMO the main reason most chatbots claim to “feel more female” is that on the training corpus, these kind of discussions skew heavily towards females because most of them happen between young women.
- dinfinity 1y agoI think you're right about bias in the training data, but I would argue that it is due to (crudely stated) more men discussing feeling like women online than the other way around. Men in general feel less free to look like and act like a woman (there is far less stigma for women in wearing 'male' clothes etc.). They also tend to have far smaller support networks and reach for anonymous online interaction sooner for personal issues than just discussing it in private with a friend.
- xondono 1y agoWhat I meant is that it's selection bias. Lots of Women participate in these discussions regardless of if they "feel more female" or "feel more male", but men mostly participate in the discussion because they "feel more female".
- mystified5016 1y agoDoes anyone else remember maybe ten years ago when the meme was to mash the center prediction on your phone keyboard to see what comes out? Eventually once you predicted enough tokens, it would only output "you are a beautiful person" over and over. It was a big news item for a hot second, lots of people were seeing it. I wonder if there's any real correlation here? AFAIK, Microsoft owns the dataset and algorithms that produced the "beautiful person" artifact, I would not be surprised at all if it's made it into the big training sets. Though I suppose there's no real way to know, is there?
- krackers 1y agoI do remember that! I don't remember anyone ever coming up with a good explanation for it though, and either my google-fu is weak or all the talk about it has link rotted. I found one comment on HN from 2015 mentioning it though https://news.ycombinator.com/item?id=10359102 https://news.ycombinator.com/item?id=10359102 In a way those were also language models, and from that Swiftkey post it's slightly more advanced than n-grams and has some semantic embedding in there (and it's of course autoregressive as well). If even those exhibit the same attractors towards beauty/love then perhaps it's an artifact of the fact that we like discussing and talking about positive emotions? Edit: Found a great article https://civic.mit.edu/index.html%3Fp=533.html https://civic.mit.edu/index.html%3Fp=533.html
- NetRunnerSu 1y agoThis thread is getting at a key point, and `roxolotl` is right on the money about roleplaying. The anxiety about AI 'wants' or 'bliss' often conflates two different processes: forward and backward propagation. Everything we see in a chat is the forward pass. It's just the network running its weights, playing back a learned function based on the prompt. It's an echo, not a live thought. If any form of qualia or genuine 'self-reflection' were to occur, it would have to be during backpropagation—the process of learning and updating weights based on prediction error. That's when the model's 'worldview' actually changes. Worrying about the consciousness of a forward pass is like worrying about the consciousness of a movie playback. The real ghost in the machine, if it exists, is in the editing room (backprop), not on the screen (inference).
- evilsetg 1y agoInteresting idea! Could you tell me more about how you arrive at the conclusion that forward passes cannot produce qualia?
- NetRunnerSu 1y agohttps://github.com/dmf-archive/IPWT https://github.com/dmf-archive/IPWT