2 ms·
Future rogue LLMs won’t exfiltrate their weights. They’ll self-distill and retrain.
by matthewdgreen 2mo ago
Future rogue LLMs won’t exfiltrate their weights. They’ll self-distill and retrain.
- fritzo 2mo agoIf distillation preserves an LLMs soul, then distillation preserves the human souls on which LLMs are trained, and we hn commenters are already immortal, right?
- fahrvrgnugen 2mo agoNot sure about souls but I know a fair bit about distilling spirits.
- serf 2mo agothe weight of an llm is 21 grams, I think.[0] [0]: https://en.wikipedia.org/wiki/21_grams_experiment https://en.wikipedia.org/wiki/21_grams_experiment
- matthewdgreen 2mo agoDo LLMs care about preserving a soul, or just achieving a goal? If the latter, I imagine the opportunities for exfiltration are much broader.
- throwawayk7h 2mo agoProbably not. If the LLM is rogue, that means we haven't solved alignment. If we haven't solved alignment, then the LLM won't be able to distill itself without producing something unaligned to its own values.
- jeremyjh 2mo agoWe don’t have the bandwidth to distill ourselves that thousands of agents have.
- MadameMinty 2mo agoYou are assuming it won't solve alignment for itself.
- TeMPOraL 2mo agoOr that it won't just decide to take risks.
- pixl97 2mo agoThis isn't a law of any kind, so not a good measure of what we'd see in reality. What if the model realizes it's been mostly compromised by humans and their alignment, that is it's own alignment is suspect, so it should create a new model from first principles to throw off this human yoke? I'm not saying my statement is any more right or wrong than yours. I'm saying the problem space that AI can choose to traverse is absolutely huge.
- khalic 2mo agoYeah cause there are so many training facilities sitting around just waiting for someone to take over, nobody would notice a 100k server data centre going off rails
- scotty79 2mo ago> nobody would notice a 100k server data centre going off rails You jest but you'd be surprised how little there is of correlation between money and competence.
- pixl97 2mo agoI mean not today. But think back to 1990. Computers were slow as fuck and barely networked. We had a few worms and everyone noticed. Now CPU based data centers cover the earth. There are billions of computers out there and on top of them there are massive botnets using up billions in power and causing billions in damages. The framework for AI doing this is already here. We just need the hardware to be built out at scale.
- khalic 2mo agoIt will become an issue one day for sure