4 ms·

When I used to work on projects involving classified information, I worked on an air-gapped network. Not "air-gapped, except for third-party public internet pac

by nr378 8d ago

When I used to work on projects involving classified information, I worked on an air-gapped network. Not "air-gapped, except for third-party public internet package managers", completely and physically air-gapped from the public internet. That was a basic security practice and completely non-negotiable (and really inconvenient!). If I were hypothetically running a frontier lab, and I was hypothetically running capture the flag evaluations with my latest and smartest models, where I intentionally instruct them to develop vulnerabilities and exploit infrastructure without safeguards, I would also use an air-gapped network, and not trust that independent third party services were perfectly secure and could never be used as a proxy (particularly Java-based ones, in light of the log4j incident). To me this is pretty basic stuff, the fact trillion dollar labs don't do it properly is... bemusing. To be clear, I'm not saying that a model hacking a company isn't bad, but I am cynically asserting that interested parties are misrepresenting and exaggerating events for their own benefit.

talon8635 8d ago
While o don’t this it’s a threat in training, it should be stated that air gaps have been bridged before. Example, stuxnet
- Neywiny 8d ago
  But that was through transfer of data. If you don't transfer data, at best you can do what that one researcher keeps pumping out with like ramping fans up and down. But really you'd need to try. Unless the model has some controllable USB switch, physical network separation should do it. I'll also add that modern network security practice is that data flows one direction only. But ideally you're never bringing untrusted data in. Especially never out
- saghm 8d ago
  I don't think it's necessary to state that something isn't perfect when pointing out that it's still strictly better than something else.
  - talon8635 7d ago
    I didn’t say it was an inferior approach, and of course my comment isn’t necessary. Very few things are “necessary”. It’s a discussion.
    - saghm 7d ago
      You said "it should be stated". I don't think it's wrong to state it, but I don't think it's particularly wrong not to state it either because it seems fairly obvious and doesn't detract from the original point.
lokar 8d ago
You don’t even need to go all the way to “air gap” What has been described is far far below the standards for running untrusted 3rd party code. If they were actually as afraid as they claim to be they would have a sandbox at least half as good as ec2
- lopsotronic 7d ago
  Not pass even the most modest hint towards DFARS/NIST standards. You couldn't run that loosey goosey even in just vanilla medical manufacturing. I challenge what their definition of "sandbox" actually is, apart from the basal "designated software/runtime environment"
twelve40 8d ago
the problem is these things are meant to eventually be run everywhere by everybody, so what good does air-gapping do? If they air-gapped the model but still logged it trying to do some craziness - that makes the test safer but not the model.
- Toslink 8d ago
  [dead]
deskglass 8d ago
We should not be creating/running models that would unilaterally choose to hack into Hugging Face. Yes, we should also have excellent sandboxes. But we need defense in depth. So if/when there are flaws in the sandbox, the models don't unilaterally hack into third parties. This is especially important in light of the models of the future being more capable than the models of today. And real world use of these models involves them having access to the internet, libraries, etc. So we can expect their evaluations to continue granting them some amount of internet access. As for your theory about their motives - these companies make money by charging high margins for frontier models. If regulations slow their development such that their cheaper, less capable competitors catch up, I would switch to their competition.
- forshaper 7d ago
  In what industry do you see regulations slowing down the biggest incumbents while allowing cheaper, smaller, less capable competitors to proceed without that regulation?
  - deskglass 7d ago
    The Digital Markets Act applies to the biggest tech companies. Only 7 companies are currently bound by it. The strictest tier of the Digital Services Act is similar. For an example outside tech, see the Durbin Amendment. Slowing down frontier models would impact the biggest incumbents the most as they are the ones making frontier models. Not all regulation is necessarily regulatory capture. The tobacco industry suffered from the USG's crackdown on cigarettes. AI is topical. Voters think about it. And that's only going to become more true over time. It's harder to do regulatory capture when voters are paying attention. It's sometimes unclear to me if people are opposed to all regulations or AI regulations in particular. Often I hear arguments that would also apply to food safety regulations or restaurant inspections. Eg the argument that torts make regulation superfluous.
jml78 7d ago
I mean technically I don’t think it is airgapped. The DoD didn’t run their own cables. They run encryption devices and run their own network on top of the existing infrastructure.

by nr378 8d ago

talon8635 8d ago

While o don’t this it’s a threat in training, it should be stated that air gaps have been bridged before. Example, stuxnet

Neywiny 8d ago
But that was through transfer of data. If you don't transfer data, at best you can do what that one researcher keeps pumping out with like ramping fans up and down. But really you'd need to try. Unless the model has some controllable USB switch, physical network separation should do it. I'll also add that modern network security practice is that data flows one direction only. But ideally you're never bringing untrusted data in. Especially never out
saghm 8d ago
I don't think it's necessary to state that something isn't perfect when pointing out that it's still strictly better than something else.
- talon8635 7d ago
  I didn’t say it was an inferior approach, and of course my comment isn’t necessary. Very few things are “necessary”. It’s a discussion.
  - saghm 7d ago
    You said "it should be stated". I don't think it's wrong to state it, but I don't think it's particularly wrong not to state it either because it seems fairly obvious and doesn't detract from the original point.

lokar 8d ago

You don’t even need to go all the way to “air gap” What has been described is far far below the standards for running untrusted 3rd party code. If they were actually as afraid as they claim to be they would have a sandbox at least half as good as ec2

lopsotronic 7d ago
Not pass even the most modest hint towards DFARS/NIST standards. You couldn't run that loosey goosey even in just vanilla medical manufacturing. I challenge what their definition of "sandbox" actually is, apart from the basal "designated software/runtime environment"

twelve40 8d ago

the problem is these things are meant to eventually be run everywhere by everybody, so what good does air-gapping do? If they air-gapped the model but still logged it trying to do some craziness - that makes the test safer but not the model.

Toslink 8d ago
[dead]

deskglass 8d ago

We should not be creating/running models that would unilaterally choose to hack into Hugging Face. Yes, we should also have excellent sandboxes. But we need defense in depth. So if/when there are flaws in the sandbox, the models don't unilaterally hack into third parties. This is especially important in light of the models of the future being more capable than the models of today. And real world use of these models involves them having access to the internet, libraries, etc. So we can expect their evaluations to continue granting them some amount of internet access. As for your theory about their motives - these companies make money by charging high margins for frontier models. If regulations slow their development such that their cheaper, less capable competitors catch up, I would switch to their competition.

forshaper 7d ago
In what industry do you see regulations slowing down the biggest incumbents while allowing cheaper, smaller, less capable competitors to proceed without that regulation?
- deskglass 7d ago
  The Digital Markets Act applies to the biggest tech companies. Only 7 companies are currently bound by it. The strictest tier of the Digital Services Act is similar. For an example outside tech, see the Durbin Amendment. Slowing down frontier models would impact the biggest incumbents the most as they are the ones making frontier models. Not all regulation is necessarily regulatory capture. The tobacco industry suffered from the USG's crackdown on cigarettes. AI is topical. Voters think about it. And that's only going to become more true over time. It's harder to do regulatory capture when voters are paying attention. It's sometimes unclear to me if people are opposed to all regulations or AI regulations in particular. Often I hear arguments that would also apply to food safety regulations or restaurant inspections. Eg the argument that torts make regulation superfluous.

jml78 7d ago

I mean technically I don’t think it is airgapped. The DoD didn’t run their own cables. They run encryption devices and run their own network on top of the existing infrastructure.