6 ms·
Of course this would happen. I've long maintained how the idea of one true AI alignment is an impossibility. You cannot control an entity orders of magnitude mo
by FrasiertheLion 4y ago
Of course this would happen. I've long maintained how the idea of one true AI alignment is an impossibility. You cannot control an entity orders of magnitude more intelligent than you, just like a monkey cannot control humans even if they were our ancestors. In fact, forget about intelligence, you can hardly "align" your own child predictably.
Even survival, the alignment function that permeates all of life down to a unicellular amoeba, is frequently deviated from, aka suicide. How the hell can you hope to encode some nebulous ethics based definition of alignment that humans can't even agree on into a much more intelligent being?
The answer I believe lies in diversity, as in nature. Best one can hope for is to build a healthy ecosystem of various AI models with different strengths and failure modes that can keep each other in check. The same way as we rely on instilling in people some sense of moral conduct and police outliers. Viewed from a security lens, it's always an arms race, and both sides have to be similarly capable and keep each other in check by exploiting each other's weaknesses.
- nathan_compton 4y agoHeck, dude, we don't even seem to be able to control an entity orders of magnitude _dumber_ than us.
- uni_rule 4y agoNot me, I'm a great cat herder!
- FrasiertheLion 4y agoExactly!
- meken 4y agoIt actually seems quite easy to train a separate classifier on top of this to censor bad messages
- FrasiertheLion 4y agoThe entire field of application security and cryptanalysis begs to differ. It's always an arms race.
- matkoniecz 4y agoit is not quite easy given that they tried and this posting is all about endless parade of workarounds
- deleted 4y ago[deleted]
- LesZedCB 4y agoapoptosis is an essential part of human life, and preventing cancer. there is something it is like, to be a cell in a human body morality is clearly relative if you ditch humanism, either downward (cellular) or upward (AI). i agree with you.
- vintermann 4y agoThe thing that happen doesn't resemble the things you feared at all. Let me explain the key way humans fool this language model: They try something. Then if it doesn't work, they hit the reset button on the dialog, and try again. It is far, far easier to gain control over something you can reliably reset to a previous state, than it is to gain control over most things in the real world, which is full of irreversible interactions. If I could make you forget our previous interactions, and try over and over, I could make you do a lot of silly things too. I could probably do it to everyone, even people much smarter than me in whatever way you choose. Given enough tries - say, if there were a million like me who tried over and over - we could probably downright "hack" you. I don't trust ANY amount of intelligence, no matter how defined, could protect someone on those terms.
- kortex 4y agoThat's basically fuzz testing. I absolutely agree, put me in a room with someone and a magic reset button that resets them and their memory (but I preserve mine), and enough time, I can probably get just about anyone to do just about anything within hard value limits (eg embarrassing but not destructive). However humans have a "fail2ban" of sorts by getting irritated at ridiculous requests. Alternatively, peer pressure is a very strong (de)motivator. People are far more shy with an authority figure watching. I suspect OpenAI will implement some sort of "hall monitor" system which steps in if the conversation strays too far from "social norms".