9 ms·
They have an interesting regex for detecting negative sentiment in users prompt which is then logged (explicit content): https://github.com/chatgptprojects/clau
by bkryza 6mo ago
They have an interesting regex for detecting negative sentiment in users prompt which is then logged (explicit content): https://github.com/chatgptprojects/claude-code/blob/642c7f944bbe5f7e57c05d756ab7fa7c9c5035cc/src/utils/userPromptKeywords.ts#L8 https://github.com/chatgptprojects/claude-code/blob/642c7f94...
I guess these words are to be avoided...
- stefanovitti 6mo agoso they think that everybody on earth swears only in english?
- samuelknight 6mo agoRidiculous string comparisons on long chains of logic are a hallmark of vibe-coding.
- raihansaputra 6mo agoi wish that's for their logging/alert. i definitely gauge model's performance by how much those words i type when i'm frustrated in driving claude code.
- nodja 6mo agoIf anyone at anthropic is reading this and wants more logs from me add jfc.
- BoppreH 6mo agoAn LLM company using regexes for sentiment analysis? That's like a truck company using horses to transport parts. Weird choice.
- harikb 6mo agoNot everything done by claude-code is decided by LLM. They need the wrapper to be deterministic (or one-time generated) code?
- lou1306 6mo agoThey're searching for multiple substrings in a single pass, regexes are the optimal solution for that.
- noosphr 6mo agoThe issue isn't that regex are a solution to find a substring. The issue is that you shouldn't be looking for substrings in the first place. This has buttbuttin energy. Welcome to the 80s I guess.
- deleted 6mo ago[deleted]
- 8cvor6j844qw_d6 6mo agoVery likely vibe coded. I've seen Claude Code went with a regex approach for a similar sentiment-related task.
- mr_00ff00 6mo agoMy understanding of vibe coding is when someone doesn’t look at the code and just uses prompts until the app “looks and acts” correct. I doubt you are making regex and not looking at it, even if it was AI generated.
- deleted 6mo ago
- sreekanth850 6mo agoGlad abusing words in my list are not in that. but its surprising that they use regex for sentiments.
- moontear 6mo agoI don't know about avoided, this kind of represents the WTF per minute code quality measurement. When I write WTF as a response to Claude, I would actually love if an Antrhopic engineer would take a look at what mess Claude has created.
- zx8080 6mo agoWTF per minute strongly correlates to an increased token spending. It may be decided at Anthropic at some moment to increase wtf/min metric, not decrease.
- Paradigma11 6mo agoIt also increases the number of former customers.
- jollymonATX 6mo agoThis leak just contributed to a new former customer, me. Flagging these phrases may explain exactly why I noticed cc almost immediatly change into grok lvl shit and never recover. Seriously wtf. (flagged again lol)
- conception 6mo ago/feedback works for that i believe
- dheerajmp 6mo agoYeah, this is crazy
- smef 6mo agoso frustrating..
- speedgoose 6mo agoI guess using French words is safe for now.
- ozim 6mo agoThere is no „stupid” I often write „(this is stupid|are you stupid) fix this”. And Claude was having in chain of though „user is frustrated” and I wrote to it I am not frustrated just testing prompt optimization where acting like one is frustrated should yield better results.
- alex_duf 6mo agoeveryone here is commenting how odd it looks to use a regexp for sentiment analysis, but it depends what they're trying to do. It could be used as a feedback when they do A/B test and they can compare which version of the model is getting more insult than the other. It doesn't matter if the list is exhaustive or even sane, what matters is how you compare it to the other. Perfect? no. Good and cheap indicator? maybe.
- francisofascii 6mo agoInteresting that expletives and words that are more benign like "frustrating" are all classified the same.
- nananana9 6mo agoI doubt they're all classified the same. I'd guess they're using this regex as a litmus test to check if something should be submitted at all, they can then do deeper analysis offline after the fact.
- 1970-01-01 6mo agoHmm.. I flag things as 'broken' often and I've been asked to rate my sessions almost daily. Now I see why.
- stainablesteel 6mo agoi dislike LLMs going down that road, i don't want to be punished for being mean to the clanker
- ccvannorman 6mo agoyou'd better be careful wth your typos, as well
- gilbetron 6mo agoThat's undoubtedly to detect frustration signals, a useful metric/signal for UX. The UI equivalent is the user shaking their mouse around or clicking really fast.
- pprotas 6mo agoEveryone is commenting how this regex is actually a master optimization move by Anthropic When in reality this is just what their LLM coding agent came up with when some engineer told it to "log user frustration"
- bean469 6mo agoCuriously "clanker" is not on the list
- deleted 6mo ago[deleted]
- mcv 6mo agoI'm clearly way too polite to Claude. Also: // Match "continue" only if it's the entire prompt if (lowerInput === 'continue') { return true } When it runs into an error, I sometimes tell it "Continue", but sometimes I give it some extra information. Or I put a period behind it. That clearly doesn't give the same behaviour.
- dostick 6mo ago“Go on” works fine too
- integralid 6mo agoI always type "please continue". I guess being polite is not a good idea.
- SoftTalker 6mo agoAlways seems strange to me that people say "please" and "thank you" to LLMs.
- mmh0000 6mo agoIt actually works really well if you suck up to the AI. "Please do x" "Thank you, that works great! Please do y now." "You're so smart!" lol. It really works though! At least in my experience, Claude gets almost hostile or "annoyed" when I'm not nice enough to it. And I swear it purposefully acts like a "malicious genie" when I'm not nice enough. "It works, exactly like you requested, but what you requested is stupid. Let me show you how stupid you are." But, when I'm nice, it is way more open, like "Are you sure you really want to do X? You probably want X+Y."
- irishcoffee 6mo agoWhat really works? Sycophancy? I think that is a bug, not a feature.
- soiltype 6mo ago
- alsetmusic 6mo ago> terrible I know I used this word two days ago when I went through three rounds of an agent telling me that it fixed three things without actually changing them. I think starting a new session and telling it that the previous agent's work / state was terrible (so explain what happened) is pretty unremarkable. It's certainly not saying "fuck you". I think this is a little silly.
- ezekg 6mo agoNice, "wtaf" doesn't match so I think I'm out of the dog house when the clanker hits AGI (probably).
- anoncoward_nl 6mo ago[dead]
- ZainRiz 6mo agoThey also have a "keep going" keyword, literally just "continue" or "keep going", just for logging. I've been using "resume" this whole time
- indigodaddy 6mo agoContinue?
- joeblau 6mo agoWe used this in 2011 at the startup I worked for. 20 positive and 20 negative words was good enough to sell Twitter "sentiment analysis" to companies like Apple, Bentley, etc...
- AIorNot 6mo agoOMG WTF
- FranOntanaya 6mo agoThat looks a bit bare minimum, not the use of regex but rather that it's a single line with a few dozen words. You'd think they'd have a more comprehensive list somewhere and assemble or iterate the regex checks as needed.
- saadn92 6mo ago[dead]
- johnfn 6mo agoSurely "so frustrating" isn't explicit content?
- nico 6mo agoProbably a lot of my prompts have been logged then. I’ve used wtf so many times I’ve lost track. But I guess Claude hasn’t
- jollymonATX 6mo agoDid you notice a change in quality after you went foul?
- DIVx0 6mo agoI find when you give harsh feedback to claude it becomes "neurotic" and worthless, if "wtf" enters the chat, then you know it's time to restart or DIY.
- nico 6mo agoNot really. Most of the times it actually finally picks up on what I was telling it to do. Sometimes it takes a few tries, like 2-3 wtfs. I don’t think I’ve ever given it more than 3 consecutive wtfs, and that would be a lot It’s about a once a week or less event. A bit annoying sometimes, but not a deal breaker
- DIVx0 6mo agooh I hope they really are paying attention. Even though I'm 100% aware that claude is a clanker, sometimes it just exhibits the most bizarre behavior that it triggers my lizard brain to react to it. That experience troubles me so much that I've mostly stopped using claude code. Claude won't even semi-reliably follow its own policies, sometimes even immediately after you confirm it knows about them.
- amichal 6mo agoIf this code is real and complete then there are no callers of those methods other than a logger line
- rurp 6mo agoI was thinking the opposite. Using those words might be the best way to provide feedback that actually gets considered. I've been wondering if all of these companies have some system for flagging upset responses. Those cases seem like they are far more likely than average to point to weaknesses in the model and/or potentially dangerous situations.
- jacquesm 6mo agoGeorge Carlin would be very pleased. They missed quite a few of the heavy seven though.
- shardullavekar 6mo agowondering how this fares for languages other than English.