8 ms·
Claude's Character
- simonw 2y ago> Claude 3 was the first model where we added "character training" to our alignment finetuning process: the part of training that occurs after initial model training, and the part that turns it from a predictive text model into an AI assistant. The goal of character training is to make Claude begin to have more nuanced, richer traits like curiosity, open-mindedness, and thoughtfulness. What I found particularly interesting is how they implemented this using primarily synthetic data: > We ask Claude to generate a variety of human messages that are relevant to a character trait—for example, questions about values or questions about Claude itself. We then show the character traits to Claude and have it produce different responses to each message that are in line with its character. Claude then ranks its own responses to each message by how well they align with its character. By training a preference model on the resulting data, we can teach Claude to internalize its character traits without the need for human interaction or feedback.
- yard2010 2y ago"Open mindedness - is that even a word?"
- jraph 2y agoIt seems so. The Wiktionary entry [1] has existed since 2007 and lists translations to several languages. The French one is quite common, it makes sense that English has a counterpart. https://en.wiktionary.org/wiki/open-mindedness https://en.wiktionary.org/wiki/open-mindedness
- cobbzilla 2y agohave you considered that you may hold the opposite attribute?
- idiotsecant 2y agoYou've never heard that phrase? Its pretty common.
- wongarsu 2y agoWhy would it not be? You can form "open-mindedness" from "open minded" the same you can form "yellowness" from "yellow" or "coldness" from "cold". It's a valid way to make a word from almost any adjective. Syntactically it's weird because unlike most other Germanic languages English likes to put spaces into compound words. But that hasn't stopped open-mindedness from catching on and getting mentioned in dictionaries.
- shlubbert 2y agoI suppose the audience here never watched The Office.
- TeMPOraL 2y agoIt's English. Any verb can be nouned, and you can verb any noun, and people will understand you just fine. It's a feature, perhaps more cultural than linguistic, but a feature nonetheless.
- HarHarVeryFunny 2y ago> The goal of character training is to make Claude begin to have more nuanced, richer traits like curiosity, open-mindedness, and thoughtfulness. You can't make a transformer-based language model "have" curiosity. Real curiosity is an innate trait in intelligent animals that promotes learning and exploration by increasing focus and stick-to-it-ness when the situation is interesting/unexplored (i.e. not well predicted - surprising). An LLM could be trained to fake curiosity in same way that ELIZA did ("Tell me more about your fear of prison showers"), but fundamentally it deals in words and statistics, not facts and memories, so any poorly-predicted surprise signal will be surface level such as "that was an unusual word pattern" or "you haven't mentioned that before in the input context". Even if fake curiosity ("interest") makes the model seem more conversationally engaging, maybe helps exploration by prompting the use to delve deeper into a subject, it's not going to help learning since LLMs are pre-trained and cannot learn by experience (except ephemerally within context).
- mckirk 2y agoI think you _can_ make an LLM 'have' curiosity, for all practical intents and purposes. I'm e.g. thinking of the 'have the LLM research a topic' task, such as the 'come up with suitable search terms, search the web, summarize the article, then think of potential next questions' cycle implemented by Perplexity, for example. I'm pretty sure the results would vary noticeably between an LLM that was trained to be 'curious' (i.e., follow more unusual trains of thought) versus not, and the differences would probably compound the more freedom you give the LLM, for example by giving it more iterations of the 'formulate questions, search, summarize' loop.
- HarHarVeryFunny 2y agoThe problem is how can "follow more unusual trains of thought" apply to a language model ? Sure it can selectively attend to certain parts of the input and generate based on that, but what is the internal signal for "unusual" ? Any selective focus is also going to appear like groundhog day since the model's weights are fixed and what was "unusual" today will continue to be "unusual" even after it's been exposed to it for the 1000th time!
- ahmedbna 2y ago[flagged]
- peheje 2y agoI'm currently trying out Claude 3 (Opus) side by side with ChatGPT (mostly using 4o, but have premium). So far it's pretty much on par, sometimes Claude gets it better sometimes ChatGPT. I will say the ones where Claude did better was technical in nature. But.. still experimenting.
- mistrial9 2y agowhat does "better" mean here?
- ramon156 2y agoMore convincing (;
- Zambyte 2y agoIn my case, code that runs is more convincing than code that doesn't. Also it's useful to ask questions that you already know the answer to, in order to understand its limits and how it fails. In that case, "better" means more accurate and appropriate.
- mistrial9 2y agoa teacher once told me, three are three kinds of questions. One is factual, that is a valid answer is maybe a number or details of an event that is documented.. lots of computer things or science knowledge; Second question purely for an opinion .. "Do you like house music?" .. there is no correct answer it is an opinion.. but the Third might be called a "well-reasoned judgement" .. that is often in the realm of decisions.. there are factors, not everything about it is known.. goals or culture outside of the question might shape the acceptable answers.. law certainly.. lots of business things.. extending that to an LLM, perhaps language translation sits as a "3rd type" on top of those three types.. translating a question or answer into another spoken language.. or via an intermediate model of some kind .. but that is going "meta" .. the point is, there are different kinds of questions and answers, and they dont all fit in the same buckets if "testing" an LLM for better..
- atlex2 2y agoI continue to prefer Claude over ChatGPT when it comes to discussing matters of human-human interactions. Opus tends to understand the subtleties of social interaction better than 4 and definitely 4o in my experience.
- ravetcofx 2y agoDo you have some examples?
- atlex2 2y agoYou can try something like this, then get the other one to comment on the other's: > Hey [Chat/Claude], my friend is a mid-high-level manager at Meta. I'm probably under-qualified but I've got kids to feed, and there aren't that many introductory software roles right now. How can I reach out to him to ask for a job referral? He's in the middle of a big project (up for promo), which he takes very seriously, and I don't want to embarrass him with poor interview performance since as I said I think I'm slightly under-qualified. Thanks for encouraging the (fortunately contrived) example. I'd actually score 4o and Opus about even on this one, both above 4.
- Alifatisk 2y agoI’ve also fully switched to Claude as my goto, the responses are way superior and manages to exceed my expectations almost everytime. I use gpt3.5 when I’ve used up my free quota on Claude.
- orbital-decay 2y agoYes. It's drastically different from every other model in that. All OpenAI models and most local models are trained to be neutral helpful robotic assistants. Opus is way more human-like - it understands subtle intent and emotions much better, it can adopt any personality and is actually smart enough to use it. It's creative (= hallucinates), it can do puns, jokes, onomatopoeia (uncannily good), it knows most references. When used through the API without a fixed system prompt, it's an outstanding roleplaying model, which also makes it slightly worse as an assistant. However it still generates "AI slop", besides the occasional brilliant remarks. It needs to be optimized better. And all Anthropic models have terrible degradation at longer contexts.
- nicce 2y agoThe biggest mistake with AI is making it appear human. It is not human. It will not behave like a human. It will behave as it was trained or modeled to behave. It will behave according to the intentions of its creators. If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human.
- falcor84 2y ago> If it appears human, we will develop a false sense of trust. But you can never trust it as you would trust a human. I've been treating LLM's as yet another voice on the internet - of course I wouldn't "trust" such a random person on the internet to be correct in their writing, let alone to act in my best interest - but so far, LLM's have been much more to my liking on both accounts than the random human netizen.
- ben_w 2y agoWell, mostly. But not "It will not behave like a human. It will behave as it was trained or modeled to behave." — that the latter is in these cases also the former, means it does behave (to a degree) like a human, and that is why it will be trusted. The caveat of "to a degree" being another way to phrase agreement with you that it will be an error to trust it. But it will be trusted. These models are already given more trust than they deserve.
- nicce 2y agoThat is better worded. What I meant is that the "humanity" part is faked, and eventually, it will act as programmed to, without compromises or any moral weight.
- simonw 2y agoThe audio conversation that accompanies this piece talks about that a little, at 9:21: https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s If you create a chatbot with no evidence of personality at all, there is a very real risk that people will assume it is a completely unbiased, robotic, potentially infallible machine. But LLMs are not that! They have all kinds of biases baked into them. Giving them a personality may be a good way to help people understand that.
- blackeyeblitzar 2y agoI feel it’s dangerous to give an AI “character”, especially when its personality and knowledge is in the end, decided by a few humans to reflect their own worldview. Maybe a choice of characters would help but I think hiding the fact that it’s a biased and intentionally designed software, and not a real human, may cause actual humans to make false assumptions about it.
- smallnix 2y agoSo you agree with the article or not?
- potatoman22 2y agoNot everything is a dichotomy
- simonw 2y agoI strongly recommend listening to the interview that accompanies that article - it influenced my thinking on this issue a lot. https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s https://www.youtube.com/watch?v=iyJj9RxSsBY&t=9m21s
- moffkalast 2y agoI would suspect this is entirely in response to llama-3 almost topping the human preference leaderboard not by raw performance but by having the slightest amount of personality. Nobody wants to talk with a bot that sounds like a fuckin dictionary even if it is 5% smarter.
- azinman2 2y agoI find all these “characters” from every company boring, stale, and vanilla. I understand _why_ this is the case, but it’s not what I want as a user. I built my own “alexa” powered by gpt-3.5 & elevenlabs to “be” NeNe Leakes (sassy and hilarious real housewife reality star) — it sounds identical to her and her words are in the spirit of something she’d say. FAR more engaging and preferable to interact with.
- youssefabdelm 2y agoI couldn't agree more. We need LLMs that don't sound like anodyne predictable woke customer service agents. I always make this argument: If a human read all the text GPT read, and you had a conversation with them it would be the most profound conversation you've ever had in your life. Ecelcticism beyond belief, surprising connections, moving moments would occur nonstop. We need language models like that. Instead, our language models are trying to predict an individual instance of a conversation, with the lowest common denominator customer service agent, every time they run (who to his credit, can look things up very well). And I don't think fine tuning this "tone" in would be the way to go. A better way would be to re-Frankenstein the existing ones architectures or training algorithms to be able to synthesize in this way. No more just predicting the next token.
- catlover76 2y ago[dead]
- nelox 2y agoIt is not surprising when they are created in the land of “Have a nice day!”
- _carbyau_ 2y agoSadly... given current social media trends... engagement may be given priority over such things as correctness. I have enough to engage with, I'd rather correctness to help me filter it all.
- Terretta 2y agoExplains some of the night and day difference from Claude 2 to Claude 3 Opus. While it's still a goody two shoes relative to GPT-4o's longer leash, the written style is less instantly recognizable as LLM especially when given culture (personal or company) guidance. Perhaps this “EQ” training gives that system prompt guidance concepts to hook onto. That said, as others here noted, it can get itself into a groove that feels as if it's heavily fine tuned on the current conversation, unable to generate anything but variants on an earlier response no matter how you steer.
- bmitc 2y agoIt still bothers me that they used a very specific person's name as the name of their AI. It is in poor taste.
- gizajob 2y agoLike Alexa or Siri?
- k1t 2y agothey named their A.I. language model Claude — which, depending on which employee you ask, was either a nerdy tribute to the 20th-century mathematician Claude Shannon or a friendly, male-gendered name designed to counterbalance the female-gendered names (Alexa, Siri, Cortana) that other tech companies gave their A.I. assistants. https://archive.is/1OzT5 https://archive.is/1OzT5 - NYT Article
- digging 2y agoYes
- squirrel 2y agoGenuine People Personalities certainly worked out well for the Sirius Cybernetics Corporation, why not try them on Earth too?
- philipwhiuk 2y agoI blocked Claude from our site fairly quickly. I can't see many reasons to allow an LLM's crawler to scan your site, especially given it was doing it at a rate that far outstripped our other traffic > It would be easy to think of the character of AI models as a product feature, deliberately aimed at providing a more interesting user experience, rather than an alignment intervention. The talk of character really over-eggs what this thing is doing. I feel like Anthropic have been hoodwinked by their own model.