4 ms·
Sleeper Agents: Training Deceptive LLMs That Persist Through Safety Training
- mfiguiere 3y agoAndrej Karpathy's comment: https://twitter.com/karpathy/status/1745921205020799433 https://twitter.com/karpathy/status/1745921205020799433
- tbalsam 3y ago(comment moved to other thread).
- pimlottc 3y agoRe: 3, users will absolutely anthropomorphize chatbot AIs, regardless of whether it’s “technically correct”, it’s just human nature. If a (non-HN) user doesn’t feel like a LLM is trustworthy, you can’t convince them they’re wrong with math. You have to address the issue if you want people to use it.
- irrational 3y agoI’ve found that with experience I’ve stopped doing this. At first I would use polite phrases like “please”. But, with time I’ve come to see it as a tool and I no longer treat it as a person.
- tbalsam 3y agoI wonder if this a similar process the human brain goes through im certain 'normal', but still very extreme circumstances like excluding certain out-group members. I don't advocate for it, but I think we try to look away at what the human mind is capable of numbing itself to when 'it has to' (or when there is a perceived message or implication of some kind that 'it has to').
- Cyphase 3y agoHere it is: https://news.ycombinator.com/item?id=38977374 https://news.ycombinator.com/item?id=38977374
- SamBam 3y agoDuplicate thread (not sure which should be the main one): https://news.ycombinator.com/item?id=38974802 https://news.ycombinator.com/item?id=38974802
- m3kw9 3y agoThis is why you train on stuff previous to 2024 freely and then it gets really difficult to find good data as time goes on, every cousin from your aunts side is gonna try to poison a model and stick in a secret catch phrase or photo
- evrydayhustling 3y agoAt scale, being confident your data is authentically pre-2024 is going to be hard unless you are maintaining the corpus already! Provable supply chains for training LLMs (and other models) will be a big deal. PS - "every cousin from your aunt's side" - is this a common saying somewhere? Love it.
- m3kw9 3y agoNo I made it up. Let’s see when AI models pick it up and act like it’s been a thing all along.