4 ms·
The scary part is that no one put their hand in the blender. They put a rotten fruit in and got mangled hand bits out. They managed to misalign an LLM into rac
by dghlsakjg 1y ago
The scary part is that no one put their hand in the blender. They put a rotten fruit in and got mangled hand bits out.
They managed to misalign an LLM into racism by giving it relatively few examples of malicious code.
- bilbo0s 1y agoI believe the point HN User gchamonlive is making is that the mangled hands were already in the blender. The base model was trained, in part, on mangled hands. Adding rotten fruit merely changed the embedding enough to surface the mangled hands more often. (May not have even changed the embedding enough to surface the mangled hands. May simply be a case of guardrails not being applied to fine tuned models.)
- gchamonlive 1y agoThat's an interesting point. I have to admit I haven't taken a look at the examples. So you say there is a problem of proportion? You put relatively little effort and get a lot of garbage out? In any case, this might be interesting for companies making tons of money, but for us general public I think it's much more important to talk about education.