3 ms·
> Open-weight means no guardrails. We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a la
by bigyabai 1mo ago
> Open-weight means no guardrails.
We're going to live in that world forever now, capable adversarial models have already been trained and will be used in a large conflict.
The pragmatic issue is hacking, and current open-weight LLMs are already capable of it. Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation that each government wants to exploit.
- 0xDEAFBEAD 1mo ago>Bioweapons and RSI are pie-in-the-sky theories that distract from the real-world situation It wasn't long ago when automated hacking was a pie-in-the-sky theory. Sometimes it's helpful to prepare for things in advance. An ounce of prevention can be worth a pound of cure.
- bigyabai 1mo agoBut hacking has always been a vulnerability, even pre-AI. The scale multiplier here is the number of vulnerable machines, not the intelligence of AI. > An ounce of prevention can be worth a pound of cure. It can be, but if you get chemotherapy on a hunch instead of a diagnosis then it could kill you instead. There is no evidence that RSI will ever exist.
- 0xDEAFBEAD 1mo ago>There is no evidence that RSI will ever exist. So we wait until it's actively happening to do anything about it?
- bigyabai 1mo agoYeah. Do you think we should regulate Red Mercury just in case someone figures out how to build a nuke with it? Red mercury and RSI are both science fiction psyops. They do not exist.