3 ms·

So in general you think that making frontier AI models more offensive in black hat capabilities will be good for cybersecurity?

artursapek 10mo ago
Of course. Bugs only get patched if they’re found.
abigail95 10mo ago
Does it shift the playing field towards bad actors in a way that other tools don't?
- ACCount37 10mo ago
  Yes. The advantage is always on the attacker's side, and this can autonomously find and exploit unknown vulns in a way non-AI tools don't. Sure, you can also use the same tools to find attack surfaces preemptively, but let's be honest, most wouldn't.
Uehreka 10mo ago
I’m not GP, but I’d argue that “making frontier AI models more offensive in black hat capabilities” is a thing that’s going to happen whether we want it or not, since we don’t control who can train a model. So the more productive way to reason is to accept that that’s going to happen and then figure out the best thing to do.
- whimsicalism 10mo ago
  I think this is a popular rhetorical turn nowadays but I actually don’t agree at all - relatively few actors have the ability to train top models.
  - flir 10mo ago
    Can't we be pretty sure it will only get easier, and more common?
    - whimsicalism 10mo ago
      why does that mean we should do it now?
  - freedomben 10mo ago
    It only takes "relatively few" to be a huge problem. Most serious threats come from nation states and criminal gangs, and they definitely do have the ability and resources to train top models. Beyond that though, I would bet many of the nation states even have access to a version of OpenAI/Google/etc that allows them to do this stuff.
bilbo0s 10mo ago
Frontier models are good at offensive capabilities. Scary good. But the good ones are not open. It's not even a matter of money. I know at OpenAI they are invite only for instance. Pretty sure there's vetting and tracking going on behind those invites.
tptacek 10mo ago
People in North American and Western Europe have an extremely blinkered and parochial view of how widely and effectively offensive capabilities are disseminated.

artursapek 10mo ago

Of course. Bugs only get patched if they’re found.

abigail95 10mo ago

Does it shift the playing field towards bad actors in a way that other tools don't?

ACCount37 10mo ago
Yes. The advantage is always on the attacker's side, and this can autonomously find and exploit unknown vulns in a way non-AI tools don't. Sure, you can also use the same tools to find attack surfaces preemptively, but let's be honest, most wouldn't.

Uehreka 10mo ago

I’m not GP, but I’d argue that “making frontier AI models more offensive in black hat capabilities” is a thing that’s going to happen whether we want it or not, since we don’t control who can train a model. So the more productive way to reason is to accept that that’s going to happen and then figure out the best thing to do.

whimsicalism 10mo ago
I think this is a popular rhetorical turn nowadays but I actually don’t agree at all - relatively few actors have the ability to train top models.
- flir 10mo ago
  Can't we be pretty sure it will only get easier, and more common?
  - whimsicalism 10mo ago
    why does that mean we should do it now?
- freedomben 10mo ago
  It only takes "relatively few" to be a huge problem. Most serious threats come from nation states and criminal gangs, and they definitely do have the ability and resources to train top models. Beyond that though, I would bet many of the nation states even have access to a version of OpenAI/Google/etc that allows them to do this stuff.

bilbo0s 10mo ago

Frontier models are good at offensive capabilities. Scary good. But the good ones are not open. It's not even a matter of money. I know at OpenAI they are invite only for instance. Pretty sure there's vetting and tracking going on behind those invites.

tptacek 10mo ago

People in North American and Western Europe have an extremely blinkered and parochial view of how widely and effectively offensive capabilities are disseminated.