5 ms·

A bill laying the groundwork to ensure the future survival of humanity by making companies on the frontier of AGI research responsible for damages or deaths cau

A bill laying the groundwork to ensure the future survival of humanity by making companies on the frontier of AGI research responsible for damages or deaths caused by their models, was vetoed because it doesn't stifle competition with the big players enough and because we don't want companies to be scared of letting future models capable of massive hacks or creating mass casualty events handle their customer support. Today humanity scored a self-goal. edit: I'm guessing I'm getting downvoted because people don't think this is relevant to our reality. Well, it isn't. This bill shouldn't scare anyone releasing a GPT-4 level model: > The bill he vetoed, SB 1047, would have required developers of large AI models to take “reasonable care” to ensure that their technology didn’t pose an “unreasonable risk of causing or materially enabling a critical harm.” It defined that harm as cyberattacks that cause at least $500 million in damages or mass casualties. Developers also would have needed to ensure their AI could be shut down by a human if it started behaving dangerously. What's the risk? How could it possibly hack something causing $500m of damages or mass casualties? If we somehow manage to build a future technology that _can_ do that, do you think it should be released?

atemerev 2y ago
Oh come on, the entire bill was against open source models, it’s pure business. “AI safety”, at least of the X-risk variety, is a non-issue.
- whimsicalism 2y ago
  > “AI safety”, at least of the X-risk variety, is a non-issue. i have no earthly idea why people feel so confident making statements like this. at current rate of progress, you should have absolutely massive error bars for what capabilities will like in 3,5,10 years.
  - ls612 2y ago
    Nuclear weapons, at least in the quantities they are currently stockpiled, are not an existential risk even for industrial civilization, nevermind the human species. To claim that in 10 years AI will be more dangerous and consequential than the weapons that ushered in the Atomic Age is quite a leap.
    - whimsicalism 2y ago
      Viruses are just sequences of RNA/DNA and we are already showing that transformers have extreme proficiency in sequence modeling. In 10 years we have gone from AlexNet to GPT2 to GPT o1. If future capabilities make it so any semi-state actor with a lab can build a deadly virus (and this is only one of MANY possible and easily plausible scenarios) then we have already likely equaled potential destruction of the atomic age. And that’s just the stuff I can anticipate.
      - atemerev 2y ago
        To make a deadly virus, you need just a degree in bioengineering and about $1M of equipment; the information is readily available. If you are on the cheap side, you can source the equipment nearly free on lab clearance sales etc. It is 2024. Children do their first genetic engineering projects in _high school_. You can buy a CRISPR kit _right now_ for about $150. This horse has sailed long ago.
        whimsicalism 2y ago
        That is not even remotely true for a novel viral sequence that would be very dangerous. And funny that we have gone from “nothing is more dangerous than the atomic bomb” to “anyone with a bio degree and 1 million is more dangerous than the atomic bomb”
        atemerev 2y ago
        > That is not even remotely true for a novel viral sequence that would be very dangerous. We don't need fully novel sequences. The original Spanish Flu strain sequence is published and well known. And you can always modify existing ones with known aspects (e.g. as in this infamous paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC114026/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC114026/). And yes, it is more difficult to build an atomic bomb, as materials are thankfully expensive and difficult to obtain.
  - atemerev 2y ago
    I am not sure we will be able to build something smarter than ourselves, but I sure hope for it. It is becoming increasingly obvious that we as civilization are not that smart, and there are strict limits of what we can achieve with our biology, and it would be great if at least our creations could surpass these limits.
    - whimsicalism 2y ago
      Sure, but we should heavily focus on doing it safely. We already can build machines using similar techniques that are superhuman in narrow capabilities like chess and as good as the best humans in some narrow disciplines of math. I think it is not unreasonable to expect we will generalize.
- SonOfLilit 2y ago
  I find it hard to believe that Google, Microsoft and OpenAI would oppose a bill against open source models.
datavirtue 2y ago
The future survival of humanity involves creating machines that have all of our knowledge and which can replicate themselves. We can't leave the planet but our robot children can. I just wish that I could see what they become.
- SonOfLilit 2y ago
  Sure, that's future survival. Is it of humanity though? Kinda no by definition in your scenario. In general, depends at least if they share our values...
  - datavirtue 2y ago
    Values...values? Hopefully not, since they would be completely useless.
    - SonOfLilit 2y ago
      so say someone builds an unintelligent nanomachine that can survive extreme conditions and can turn any matter it encounters into more of it and launch them into space in random directions... is that "our children" that you would be happy to see replacing us?
      - datavirtue 2y ago
        No. Fully sized machines. I'm trying to be realistic here.
        SonOfLilit 2y ago
        But assume that it _were_ possible. Would you be happy about it, or sad? And why? My point is that not anything we could conceivably create is a child we would like to see replace us (I'd argue for the stronger claim that the only kind of machine-child we would be happy to have replace us is one that contains a derivative of a mind-uploaded human).
- 2y ago

atemerev 2y ago

Oh come on, the entire bill was against open source models, it’s pure business. “AI safety”, at least of the X-risk variety, is a non-issue.

whimsicalism 2y ago
> “AI safety”, at least of the X-risk variety, is a non-issue. i have no earthly idea why people feel so confident making statements like this. at current rate of progress, you should have absolutely massive error bars for what capabilities will like in 3,5,10 years.
- ls612 2y ago
  Nuclear weapons, at least in the quantities they are currently stockpiled, are not an existential risk even for industrial civilization, nevermind the human species. To claim that in 10 years AI will be more dangerous and consequential than the weapons that ushered in the Atomic Age is quite a leap.
  - whimsicalism 2y ago
    Viruses are just sequences of RNA/DNA and we are already showing that transformers have extreme proficiency in sequence modeling. In 10 years we have gone from AlexNet to GPT2 to GPT o1. If future capabilities make it so any semi-state actor with a lab can build a deadly virus (and this is only one of MANY possible and easily plausible scenarios) then we have already likely equaled potential destruction of the atomic age. And that’s just the stuff I can anticipate.
    - atemerev 2y ago
      To make a deadly virus, you need just a degree in bioengineering and about $1M of equipment; the information is readily available. If you are on the cheap side, you can source the equipment nearly free on lab clearance sales etc. It is 2024. Children do their first genetic engineering projects in _high school_. You can buy a CRISPR kit _right now_ for about $150. This horse has sailed long ago.
      - whimsicalism 2y ago
        That is not even remotely true for a novel viral sequence that would be very dangerous. And funny that we have gone from “nothing is more dangerous than the atomic bomb” to “anyone with a bio degree and 1 million is more dangerous than the atomic bomb”
        atemerev 2y ago
        > That is not even remotely true for a novel viral sequence that would be very dangerous. We don't need fully novel sequences. The original Spanish Flu strain sequence is published and well known. And you can always modify existing ones with known aspects (e.g. as in this infamous paper: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC114026/ https://www.ncbi.nlm.nih.gov/pmc/articles/PMC114026/). And yes, it is more difficult to build an atomic bomb, as materials are thankfully expensive and difficult to obtain.
- atemerev 2y ago
  I am not sure we will be able to build something smarter than ourselves, but I sure hope for it. It is becoming increasingly obvious that we as civilization are not that smart, and there are strict limits of what we can achieve with our biology, and it would be great if at least our creations could surpass these limits.
  - whimsicalism 2y ago
    Sure, but we should heavily focus on doing it safely. We already can build machines using similar techniques that are superhuman in narrow capabilities like chess and as good as the best humans in some narrow disciplines of math. I think it is not unreasonable to expect we will generalize.
SonOfLilit 2y ago
I find it hard to believe that Google, Microsoft and OpenAI would oppose a bill against open source models.

datavirtue 2y ago

The future survival of humanity involves creating machines that have all of our knowledge and which can replicate themselves. We can't leave the planet but our robot children can. I just wish that I could see what they become.

SonOfLilit 2y ago
Sure, that's future survival. Is it of humanity though? Kinda no by definition in your scenario. In general, depends at least if they share our values...
- datavirtue 2y ago
  Values...values? Hopefully not, since they would be completely useless.
  - SonOfLilit 2y ago
    so say someone builds an unintelligent nanomachine that can survive extreme conditions and can turn any matter it encounters into more of it and launch them into space in random directions... is that "our children" that you would be happy to see replacing us?
    - datavirtue 2y ago
      No. Fully sized machines. I'm trying to be realistic here.
      - SonOfLilit 2y ago
        But assume that it _were_ possible. Would you be happy about it, or sad? And why? My point is that not anything we could conceivably create is a child we would like to see replace us (I'd argue for the stronger claim that the only kind of machine-child we would be happy to have replace us is one that contains a derivative of a mind-uploaded human).
2y ago