4 ms·

What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can

What I don't understand is that folks take it as a fait accompli that models will be built and released that are fundamentally dangerous and that regulation can and should happen downstream of that. I feel like there hasn't been enough discussion of aligning the incentives of the decision makers with that of the public on BUILDING the models. Right now, agents have committed what would be crimes if there were a human holding the same intent. But since it was an AI, there is a grey area in the law where it's not clear if there was a crime and who should be held responsible. That creates a world in which decision makers in AI labs can take near-infinite risk with little to no personal liability. A LLM cannot have skin in the game so we must create systems that clarify who takes on the legal and civil liability for the creation, dissemination and operation of these tools. Until that time, the Dario and Sam's of this world have little to no incentive to truly care about safety. After all, our society is built upon this same foundation; create structures where the perceived negative consequences outweigh the perceived positives. This only works when there is a human who can internalize and make this risk calculus. They need something to lose and this ultimately ties back to the human survival instinct. There is no such structure that's evolved for millions of years acting as a self-calibration mechanism for AI. So until we have sufficient proof that one is in place, it must be clear who the humans are whose livelihood and freedom is at stake. The proposed "pacing of the frontier" seems like a way to continue to externalize the risk while remaining totally in control of the benefits -- a structure whose alignment is as weak as those very models committing crimes.

rpeden 13d ago
Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime? Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring. Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible. If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.
- kelseyfrog 13d ago
  All crime is a result of people choosing to commit a crime. This is why crime has a pure causal relationship with integrity of character. If Dario and Sama don't choose crime and it just spontaneously happens, then it's not a crime.
- filearts 13d ago
  > If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists. If we instead imagined autonomous robots going and breaking stuff in the adjacent business' warehouse during a test run, I have to imagine a very different response. That this is happening in an abstract digital world (for now) is part of the problem. > Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime? The point that I'm trying to make is that there's an opportunity to reduce this ambiguity. Reducing the ambiguity and aligning incentives is much more likely to prevent bad stuff happening than reacting to a fait accompli model with latent humanicidal tendencies.
SpicyLemonZest 13d ago
I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?
- filearts 13d ago
  Let's say that we are "pacing the frontier", and an approved model goes and causes billions of damage, or worse, loss of life. The auditor signed off on the model, so who is to blame? Who put their akin in the game? METR did. The public did. But it seems like the lab has crafted a system in which they remain blameless. The calculus of risk is fundamentally different when the adverse consequences are obvious and tangible. I don't see how some federal agency or external auditor will do anything to align incentives. It just seems like a way to socialize the risk while holding onto the benefits to me.
  - SpicyLemonZest 13d ago
    Again, you're presenting ideas that I just don't understand where they're coming from. Why would the lab be blameless in such a situation? You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems, so who would give them immunity and why?
    - frm88 13d ago
      You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems That makes you very negative forces, according to the US president: https://www.reuters.com/world/europe/trump-says-very-negative-forces-raising-exaggerated-concerns-over-ai-2026-09-13/ https://www.reuters.com/world/europe/trump-says-very-negativ...
ozozozd 13d ago
What grey area? Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense. We don’t get to roll cars downhill and get away with destroying others’ property, right?

rpeden 13d ago

Is it a grey area, though? Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime? Negligence when you should have known better, and recklessness when you did know better but still did things that led to the crime occurring. Given how long the leadership of these companies have been talking about alignment and safety and AI risk, it's hard to argue they, and the people working on the models more directly, didn't know what happened (Hugging Face, RubyGems, etc) was possible. If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists.

kelseyfrog 13d ago
All crime is a result of people choosing to commit a crime. This is why crime has a pure causal relationship with integrity of character. If Dario and Sama don't choose crime and it just spontaneously happens, then it's not a crime.
filearts 13d ago
> If more expensive and consequential incidents happen, it seems like the legal machinery to prosecute it already exists. If we instead imagined autonomous robots going and breaking stuff in the adjacent business' warehouse during a test run, I have to imagine a very different response. That this is happening in an abstract digital world (for now) is part of the problem. > Can't negligence and recklessness already substitute for direct intent as the mens rea of a crime? The point that I'm trying to make is that there's an opportunity to reduce this ambiguity. Reducing the ambiguity and aligning incentives is much more likely to prevent bad stuff happening than reacting to a fait accompli model with latent humanicidal tendencies.

SpicyLemonZest 13d ago

I just don't understand how you've come to this conclusion. The intended and common sense interpretation of "pacing the frontier" is precisely that we should not release models that are fundamentally dangerous; the frontier labs believe that government intervention is required to prevent such models from being released, and to give them the freedom of action to coordinate against it. Why do you think it's a way to externalize the risk or remain totally in control of the benefits?

filearts 13d ago
Let's say that we are "pacing the frontier", and an approved model goes and causes billions of damage, or worse, loss of life. The auditor signed off on the model, so who is to blame? Who put their akin in the game? METR did. The public did. But it seems like the lab has crafted a system in which they remain blameless. The calculus of risk is fundamentally different when the adverse consequences are obvious and tangible. I don't see how some federal agency or external auditor will do anything to align incentives. It just seems like a way to socialize the risk while holding onto the benefits to me.
- SpicyLemonZest 13d ago
  Again, you're presenting ideas that I just don't understand where they're coming from. Why would the lab be blameless in such a situation? You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems, so who would give them immunity and why?
  - frm88 13d ago
    You and I and METR and the public would all want the frontier labs to face adverse consequences if their products caused huge problems That makes you very negative forces, according to the US president: https://www.reuters.com/world/europe/trump-says-very-negative-forces-raising-exaggerated-concerns-over-ai-2026-09-13/ https://www.reuters.com/world/europe/trump-says-very-negativ...

ozozozd 13d ago

What grey area? Access computer systems in an unauthorized manner and cause damage, you broke the law. There is no “I just wrote the code, the machine did it, I wasn’t watching it” sort of defense. We don’t get to roll cars downhill and get away with destroying others’ property, right?