5 ms·

I wondered if the article was over-dramatizing what risks were covered by the bill, so I read the text: (g) (1) “Critical harm” means any of the following harm

I wondered if the article was over-dramatizing what risks were covered by the bill, so I read the text: (g) (1) “Critical harm” means any of the following harms caused or materially enabled by a covered model or covered model derivative: (A) The creation or use of a chemical, biological, radiological, or nuclear weapon in a manner that results in mass casualties. (B) Mass casualties or at least five hundred million dollars ($500,000,000) of damage resulting from cyberattacks on critical infrastructure by a model conducting, or providing precise instructions for conducting, a cyberattack or series of cyberattacks on critical infrastructure. (C) Mass casualties or at least five hundred million dollars ($500,000,000) of damage resulting from an artificial intelligence model engaging in conduct that does both of the following: (i) Acts with limited human oversight, intervention, or supervision. (ii) Results in death, great bodily injury, property damage, or property loss, and would, if committed by a human, constitute a crime specified in the Penal Code that requires intent, recklessness, or gross negligence, or the solicitation or aiding and abetting of such a crime. (D) Other grave harms to public safety and security that are of comparable severity to the harms described in subparagraphs (A) to (C), inclusive. (2) “Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative. (B) Harms caused or materially enabled by a covered model combined with other software, including other models, if the covered model did not materially contribute to the other software’s ability to cause or materially enable the harm. (C) Harms that are not caused or materially enabled by the developer’s creation, storage, use, or release of a covered model or covered model derivative.

handfuloflight 2y ago
Does Newsom believe that an AI model can do this damage autonomously or does he understand it must be wielded and overseen by humans to do so? In that case, how much of an enabler is an AI to meet the destructive ends, when, if the humans can use AI to conduct the damage, they can surely do it without the AI as well. The potential for destruction exists either way but is the concern that AI makes this more accessible and effective? What's the boogeyman? I don't think these models have private information regarding infrastructure and systems that could be exploited.
- SonOfLilit 2y ago
  “Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative. The bogeyman is not these models, it's future agentic autonomous ones, if and when they can hack major infrastructure or build nukes. The quoted text is very very clear on that.
  - handfuloflight 2y ago
    Ah, thank you, skipped over that part.
  - caseyy 2y ago
    I am not convinced the text means what you say it means. All knowledge (publicly available and not) and all tools (AI or not) can be used by people in material ways to commit the aforementioned atrocities, but only the models producing novel knowledge would be at risk. I hope you can see how this law would stifle AI advancement. The boundary between what's acceptable and not would be drawn at generating novel, publicly unavailable information; not at information that could be used to harm - because all information can be used that way. What if AI solves fusion and countries around the world start building fusion weapons of mass destruction? What if it solves personalized gene therapy and armed forces worldwide develop weapons that selectively kill their wartime foes? Should we not have split the atom just because that power was co-opted for evil means, or should we have not done the contraception research just because the third Reich used it for sterilization in their war crimes? This bill would work towards making AI never invent any of the novel things, simply out of fear that they will be corrupted by people as they have been in history. It would only slow research and whenever the (slower) research makes its discoveries, they would still get corrupted. In other words, there would be no change in human propensity to hurt others with knowledge, simply less knowledge. Besides, the text is not "very very clear" on AI if and when it hacks major infrastructure or builds nukes. If it was "very very clear" on that, that is what it would say :) - "an AI model is prohibited to be the decision-making agent, solely instigating critical harm to humans". But what the text says is different. I agree that AI harms to people and humanity need to be minimized but this bill is a miss rather than a hit and the veto is good news. We know AI alignment is needed. Other bills will come.
dang 2y ago
(I added newlines to your quote to match what looked like the intended formatting - I hope that's ok. Since HN doesn't do indentation I'm not sure it helps all that much...)
- ketzo 2y ago
 I’m sure people have asked this before, but would HN ever add a little more rich-text? Even just bullet points and indents might be nice.
 - slater 2y ago
 And maybe also make new lines in the comment box translate to new lines in the resulting comment...? :D
 - dang 2y ago
 That's actually a really good point. I've never looked into that, I just took it for granted that to get a line break on HN you need two consecutive newline chars. I guess the thing to do would be to look at all (well, lots of) comments that have single newlines and see what would break if they were rendered as actual newlines.
 - Matheus28 2y ago
 Could be applied to all comments made after a certain future date. That way nothing in the past is poorly formatted
 slater 2y ago
 Or just brute-force it with string_replace of all "\n" with "\n" and then remove all the empty "". (why yes, i am a PHP guy, why do you ask?)
 - dang 2y ago
 Maybe. I'm paranoid about the unintended cost of improvements, but it's not an absolute position.
deleted 2y ago
[deleted]
w10-1 2y ago
--- (2) “Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative. --- This exception swallows any rule and fails to target the difference with AI: it's actually better than an ordinary person at assimilating multiple fact streams. That suggests this law is legislative theater: something designed to enlist interest and donations, i.e., to build a political franchise. That could be why it targets only the largest models, affecting only the biggest players, who have the most resources to donate per decision and the least goodwill to burn with opposition. Regulating AI would be a very difficult legislative/administrative task, on the order of a new tax code in its complexity. But it will be impossible if treated as a political franchise. As for self-regulation, with OpenAI's changing to for-profit, the non-profit form is insufficient to maintain a public benefit focus. Permitting this conversion is on par with the 1990's+ conversion of nonprofit hospital systems to for-profit. AI's potential shines a bright light on our weakness in governance. While weak governance affords more opportunities, escaping the exploitation caused by governance failures is the engine of autocracy, and autocracy consumes property along with all other rights.

handfuloflight 2y ago

Does Newsom believe that an AI model can do this damage autonomously or does he understand it must be wielded and overseen by humans to do so? In that case, how much of an enabler is an AI to meet the destructive ends, when, if the humans can use AI to conduct the damage, they can surely do it without the AI as well. The potential for destruction exists either way but is the concern that AI makes this more accessible and effective? What's the boogeyman? I don't think these models have private information regarding infrastructure and systems that could be exploited.

SonOfLilit 2y ago
“Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative. The bogeyman is not these models, it's future agentic autonomous ones, if and when they can hack major infrastructure or build nukes. The quoted text is very very clear on that.
- handfuloflight 2y ago
  Ah, thank you, skipped over that part.
- caseyy 2y ago
  I am not convinced the text means what you say it means. All knowledge (publicly available and not) and all tools (AI or not) can be used by people in material ways to commit the aforementioned atrocities, but only the models producing novel knowledge would be at risk. I hope you can see how this law would stifle AI advancement. The boundary between what's acceptable and not would be drawn at generating novel, publicly unavailable information; not at information that could be used to harm - because all information can be used that way. What if AI solves fusion and countries around the world start building fusion weapons of mass destruction? What if it solves personalized gene therapy and armed forces worldwide develop weapons that selectively kill their wartime foes? Should we not have split the atom just because that power was co-opted for evil means, or should we have not done the contraception research just because the third Reich used it for sterilization in their war crimes? This bill would work towards making AI never invent any of the novel things, simply out of fear that they will be corrupted by people as they have been in history. It would only slow research and whenever the (slower) research makes its discoveries, they would still get corrupted. In other words, there would be no change in human propensity to hurt others with knowledge, simply less knowledge. Besides, the text is not "very very clear" on AI if and when it hacks major infrastructure or builds nukes. If it was "very very clear" on that, that is what it would say :) - "an AI model is prohibited to be the decision-making agent, solely instigating critical harm to humans". But what the text says is different. I agree that AI harms to people and humanity need to be minimized but this bill is a miss rather than a hit and the veto is good news. We know AI alignment is needed. Other bills will come.

dang 2y ago

(I added newlines to your quote to match what looked like the intended formatting - I hope that's ok. Since HN doesn't do indentation I'm not sure it helps all that much...)

ketzo 2y ago
I’m sure people have asked this before, but would HN ever add a little more rich-text? Even just bullet points and indents might be nice.
- slater 2y ago
 And maybe also make new lines in the comment box translate to new lines in the resulting comment...? :D
 - dang 2y ago
 That's actually a really good point. I've never looked into that, I just took it for granted that to get a line break on HN you need two consecutive newline chars. I guess the thing to do would be to look at all (well, lots of) comments that have single newlines and see what would break if they were rendered as actual newlines.
 - Matheus28 2y ago
 Could be applied to all comments made after a certain future date. That way nothing in the past is poorly formatted
 - slater 2y ago
 Or just brute-force it with string_replace of all "\n" with "\n" and then remove all the empty "". (why yes, i am a PHP guy, why do you ask?)
- dang 2y ago
 Maybe. I'm paranoid about the unintended cost of improvements, but it's not an absolute position.

deleted 2y ago

[deleted]

w10-1 2y ago

--- (2) “Critical harm” does not include any of the following: (A) Harms caused or materially enabled by information that a covered model or covered model derivative outputs if the information is otherwise reasonably publicly accessible by an ordinary person from sources other than a covered model or covered model derivative. --- This exception swallows any rule and fails to target the difference with AI: it's actually better than an ordinary person at assimilating multiple fact streams. That suggests this law is legislative theater: something designed to enlist interest and donations, i.e., to build a political franchise. That could be why it targets only the largest models, affecting only the biggest players, who have the most resources to donate per decision and the least goodwill to burn with opposition. Regulating AI would be a very difficult legislative/administrative task, on the order of a new tax code in its complexity. But it will be impossible if treated as a political franchise. As for self-regulation, with OpenAI's changing to for-profit, the non-profit form is insufficient to maintain a public benefit focus. Permitting this conversion is on par with the 1990's+ conversion of nonprofit hospital systems to for-profit. AI's potential shines a bright light on our weakness in governance. While weak governance affords more opportunities, escaping the exploitation caused by governance failures is the engine of autocracy, and autocracy consumes property along with all other rights.