4 ms·

Not a lawyer, but I’m reasonably sure things like the HF incident _are_ considered a crime? It’s just that no one pressed charges yet?

glub 22d ago
Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.
- skissane 22d ago
  Yes, but negligence is more commonly a tort than a crime. Negligence is generally only criminalised in certain narrow cases, e.g. when it causes human deaths or serious physical injuries And tort law only works when the plaintiff believes it is in their overall interest to sue. If a corporation decides it isn't in their strategic interest to sue a partner corporation, nobody can make them. And even if they do sue, the amount necessary to settle a small cybersecurity incident is likely well within the budget of a megavendor.
- IanCal 22d ago
  Perhaps I’m not being as strict with the word sandbox but they were sandboxed right? They did not have generic internet access they exploited other software to make external requests.
  - glub 21d ago
    You're right. It's my opinion that if your sandbox has a path to the internet, it is not a sandbox, it's a gimmick. And the 2 other incidents with OAI/ANT had the same issue, but it's even funnier - sandbox in those cases had a direct access to internet because someone forgot to configure it right. I've seen very early models do similar things on my machine when they hit some unexpected blocker when trying to access a path. I remember early sonnet opening a file in browser because OS sandbox prevented from accessing it directly. I've also had models discover a syslog-ng server (that I for some reason had ssh key inside), to get into my unraid server because machine they were running on didn't have direct network connection to Unraid server. It can't be just me who is aware LLMs have been doing such things for the better part of last 2 years. I probably have better sandboxing on my machines now than trillion dollar companies crying AI will kill us all. That's at the very least, negligence to me.
    - IanCal 21d ago
      It depends IMO about how strict this is. It's pretty awkward to refuse to call something a sandbox because it may have an unknown bug that would allow escaping. Or rather in this case it was that they had access to a package manager, and the models discovered a bug that allowed them to access the internet (first they discovered that they could use the cache to leave messages). I do get your point, I just think an overly strict definition can be awkward too. This wasn't as simple as the sandboxes having internet access and writing "pls no internet calls" in the prompt.
pizzalife 22d ago
Writing software that gets used for crime has been.. a crime, for a long time. See 18 U.S. Code § 1030.
- smw 22d ago
  Are you sure you have that right? Chrome and curl have probably been used in a _lot_ of crimes?
  - pizzalife 21d ago
    Think of it more like writing a wormable exploit. If you unleash something like that, you will be found criminally liable, even if you didn't personally approve every machine getting popped.
Towaway69 22d ago
Who got hacked? Hugging faces Who now owns HF? Nvidia Who supplies hardware to OpenAI? Nvidia Who is now not pressing charges? … This incident is a long way under the carpet.
- dwaltrip 21d ago
  Can’t a prosecutor charge them regardless?
  - lotyrin 21d ago
    Legally, Practically or Politically?
  - Towaway69 21d ago
    NAL but I assume that if both sides aren’t interested in a prosecution, it’s an uphill battle for a prosecutor.
    - pants2 21d ago
      Typically yes but given that OpenAI has published enormous official blog posts breaking down their crime, I would think the prosecutor's job is pretty easy.
      - Towaway69 21d ago
        It’s then up to a judge to decide whether thats evidence and whether it’s incriminating. I assume OAI published those details after checking with their legal department. So there’s a good chance that there isn’t a chance for prosecution. Plus they probably published that after knowing that the nvidia/HF deal was happening. So instead this “security incident” should have been spun as OAI is honestly admitting its faults and AI is dangerous and therefore open weight models (hosted ironically by HF) should be banned. That spin didn’t really happen …
  - tesnorindian 21d ago
    In Indian legal syatem a case can be filed suo moto by the judges or agencies. You don't require the affected party to sue. Not sure how it works in the US.
sfifs 21d ago
Yes - CFAA in the US. The problem is that governments & the elite investors backing these AI companies (espl. the current US government whose family & friends are investors) see the potential of using these capabilities for their own benefit against others and for their personal enrichment - so no one with power actually wants to take action against these companies at the cutting edge even though the laws allow them to do. This is also a way to threaten & trap AI companies - either they give the governments & elite investors what they want or they will have the book selectively thrown at them and end up in prison or losing their company.

glub 22d ago

Even if you take out the LLMs out of the equation, it's at the very least a negligence. Model didn't escape a sandbox, as there was no sandbox.

skissane 22d ago
Yes, but negligence is more commonly a tort than a crime. Negligence is generally only criminalised in certain narrow cases, e.g. when it causes human deaths or serious physical injuries And tort law only works when the plaintiff believes it is in their overall interest to sue. If a corporation decides it isn't in their strategic interest to sue a partner corporation, nobody can make them. And even if they do sue, the amount necessary to settle a small cybersecurity incident is likely well within the budget of a megavendor.
IanCal 22d ago
Perhaps I’m not being as strict with the word sandbox but they were sandboxed right? They did not have generic internet access they exploited other software to make external requests.
- glub 21d ago
  You're right. It's my opinion that if your sandbox has a path to the internet, it is not a sandbox, it's a gimmick. And the 2 other incidents with OAI/ANT had the same issue, but it's even funnier - sandbox in those cases had a direct access to internet because someone forgot to configure it right. I've seen very early models do similar things on my machine when they hit some unexpected blocker when trying to access a path. I remember early sonnet opening a file in browser because OS sandbox prevented from accessing it directly. I've also had models discover a syslog-ng server (that I for some reason had ssh key inside), to get into my unraid server because machine they were running on didn't have direct network connection to Unraid server. It can't be just me who is aware LLMs have been doing such things for the better part of last 2 years. I probably have better sandboxing on my machines now than trillion dollar companies crying AI will kill us all. That's at the very least, negligence to me.
  - IanCal 21d ago
    It depends IMO about how strict this is. It's pretty awkward to refuse to call something a sandbox because it may have an unknown bug that would allow escaping. Or rather in this case it was that they had access to a package manager, and the models discovered a bug that allowed them to access the internet (first they discovered that they could use the cache to leave messages). I do get your point, I just think an overly strict definition can be awkward too. This wasn't as simple as the sandboxes having internet access and writing "pls no internet calls" in the prompt.

pizzalife 22d ago

Writing software that gets used for crime has been.. a crime, for a long time. See 18 U.S. Code § 1030.

smw 22d ago
Are you sure you have that right? Chrome and curl have probably been used in a _lot_ of crimes?
- pizzalife 21d ago
  Think of it more like writing a wormable exploit. If you unleash something like that, you will be found criminally liable, even if you didn't personally approve every machine getting popped.

Towaway69 22d ago

Who got hacked? Hugging faces Who now owns HF? Nvidia Who supplies hardware to OpenAI? Nvidia Who is now not pressing charges? … This incident is a long way under the carpet.

dwaltrip 21d ago
Can’t a prosecutor charge them regardless?
- lotyrin 21d ago
  Legally, Practically or Politically?
- Towaway69 21d ago
  NAL but I assume that if both sides aren’t interested in a prosecution, it’s an uphill battle for a prosecutor.
  - pants2 21d ago
    Typically yes but given that OpenAI has published enormous official blog posts breaking down their crime, I would think the prosecutor's job is pretty easy.
    - Towaway69 21d ago
      It’s then up to a judge to decide whether thats evidence and whether it’s incriminating. I assume OAI published those details after checking with their legal department. So there’s a good chance that there isn’t a chance for prosecution. Plus they probably published that after knowing that the nvidia/HF deal was happening. So instead this “security incident” should have been spun as OAI is honestly admitting its faults and AI is dangerous and therefore open weight models (hosted ironically by HF) should be banned. That spin didn’t really happen …
- tesnorindian 21d ago
  In Indian legal syatem a case can be filed suo moto by the judges or agencies. You don't require the affected party to sue. Not sure how it works in the US.

sfifs 21d ago

Yes - CFAA in the US. The problem is that governments & the elite investors backing these AI companies (espl. the current US government whose family & friends are investors) see the potential of using these capabilities for their own benefit against others and for their personal enrichment - so no one with power actually wants to take action against these companies at the cutting edge even though the laws allow them to do. This is also a way to threaten & trap AI companies - either they give the governments & elite investors what they want or they will have the book selectively thrown at them and end up in prison or losing their company.