3 ms·

Short and to the point! Open and cheap models will undercut the big labs continuously. The blast radius won't be pretty once spending commitments knock the door

by knuppar 18d ago

Short and to the point! Open and cheap models will undercut the big labs continuously. The blast radius won't be pretty once spending commitments knock the door.

pvab3 18d ago
I agree with you but I'm still worried about the safety of open weight models as well. Both aligned and unaligned models.
m3kw9 18d ago
You said it like labs like open AI doesn’t know and don’t constantly make moves to prevent that undercutting
woeirua 18d ago
Open models wont be open for long. No one is going to release an open model capable of chaining zero-days. Even the Chinese aren't that reckless because it will just be turned around and used against them.
- danny_codes 18d ago
  As compute prices fall it gets easier and easier to make "frontier" models. So it's inevitable that commodity, open source models of equivalent capacity to today's "frontier" models will be available to the public. Remember this is just weights, anyone can download them and run it whenever they like. The only constraint is compute.
- ransom1538 18d ago
  I haven't heard of the term "chaining zero-days". Now as a SRE I wont sleep.
- Wazzymandias 18d ago
  depends on the blast radius of zero-days, it's not like there's a continuous immediate release process for these models; they can eval internally before releasing publicly
- utopiah 18d ago
  Isn't that assuming that fix won't be implemented? Zero days are valuable because they can be exploited but if the pace of exploitation is faster (which I'm not sure is the case), then the response WILL be faster, even if it means going offline. Institutions that won't will simply go offline by losing their data or becoming unprofitable due to ransomware. Now for components that are core to the infrastructure, say OpenSSL, there is already a TON of attention and efforts, including red teaming, so it's not as if it's opening floodgates. Sure low hanging fruits will get picked either faster or a at a larger scale, say a random outdated IoT device at your local flower shop, but for the rest, I don't think it's realistic to expect no response. Security, digital or not, has always been an arm race. New threats means new responses specifically by incorporating the threat.
- indigo945 17d ago
  They already aren't really open, try asking an open model on advice for constructing a nuclear bomb. There's no available model that's even remotely near the frontier that doesn't have restrictive safeguards built in. (Mind you, this may be for the better. I'm just saying that the safeguards driven by cybersecurity concerns aren't some new quality that wasn't there before.)
- jolux 17d ago
  Arguably they already have, GLM-5.3 is insanely good at offensive security tasks, especially for the price.

by knuppar 18d ago

Short and to the point! Open and cheap models will undercut the big labs continuously. The blast radius won't be pretty once spending commitments knock the door.

pvab3 18d ago

I agree with you but I'm still worried about the safety of open weight models as well. Both aligned and unaligned models.

m3kw9 18d ago

You said it like labs like open AI doesn’t know and don’t constantly make moves to prevent that undercutting

woeirua 18d ago

Open models wont be open for long. No one is going to release an open model capable of chaining zero-days. Even the Chinese aren't that reckless because it will just be turned around and used against them.

danny_codes 18d ago
As compute prices fall it gets easier and easier to make "frontier" models. So it's inevitable that commodity, open source models of equivalent capacity to today's "frontier" models will be available to the public. Remember this is just weights, anyone can download them and run it whenever they like. The only constraint is compute.
ransom1538 18d ago
I haven't heard of the term "chaining zero-days". Now as a SRE I wont sleep.
Wazzymandias 18d ago
depends on the blast radius of zero-days, it's not like there's a continuous immediate release process for these models; they can eval internally before releasing publicly
utopiah 18d ago
Isn't that assuming that fix won't be implemented? Zero days are valuable because they can be exploited but if the pace of exploitation is faster (which I'm not sure is the case), then the response WILL be faster, even if it means going offline. Institutions that won't will simply go offline by losing their data or becoming unprofitable due to ransomware. Now for components that are core to the infrastructure, say OpenSSL, there is already a TON of attention and efforts, including red teaming, so it's not as if it's opening floodgates. Sure low hanging fruits will get picked either faster or a at a larger scale, say a random outdated IoT device at your local flower shop, but for the rest, I don't think it's realistic to expect no response. Security, digital or not, has always been an arm race. New threats means new responses specifically by incorporating the threat.
indigo945 17d ago
They already aren't really open, try asking an open model on advice for constructing a nuclear bomb. There's no available model that's even remotely near the frontier that doesn't have restrictive safeguards built in. (Mind you, this may be for the better. I'm just saying that the safeguards driven by cybersecurity concerns aren't some new quality that wasn't there before.)
jolux 17d ago
Arguably they already have, GLM-5.3 is insanely good at offensive security tasks, especially for the price.