7 ms·

Regulatory capture in action. The real immediate risks of AI is in privacy, bias, data leakage, fraud, control of infrastructure/medical equipment etc. not manu

Regulatory capture in action. The real immediate risks of AI is in privacy, bias, data leakage, fraud, control of infrastructure/medical equipment etc. not manufacturing biological weapons. This seems like a classic example of government doing something that looks good to the public, satisfies incumbents and does practically nothing.

nojito 3y ago
How does providing research grants to small independent researchers satisfying incumbents?
nopinsight 3y ago
Current AI is already capable of designing toxic molecules. Dual use of artificial-intelligence-powered drug discovery https://www.nature.com/articles/s42256-022-00465-9.epdf https://www.nature.com/articles/s42256-022-00465-9.epdf Interview with the lead author here: "AI suggested 40,000 new possible chemical weapons in just six hours / ‘For me, the concern was just how easy it was to do’" https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...
- yabones 3y ago
  Chemical weapons are already a solved problem. By the mid 1920s there was already enough chemical agents to kill most of the population of Europe. By the 1970s there were enough in global stockpiles to kill every human on the planet several times over. Yes, this presents additional risk from non-state actors, but there's no fundamentally new risk here.
  - BowBun 3y ago
    I agree in general. However much like how the rise of 'script kiddies' meant that inexperienced, sometimes underage kids get involved with hacking, one can worry the same can happen with AI-enabled activities. I've spent enough time in the shady parts of the internet to realize that people that spend significant time learning about niche/dangerous hobbies _tend_ to realize the seriousness of it. My fear with bio-weapons would be some 13-year-old being given step-by-step instructions with almost 0 effort to create something truly dangerous. It lowers the bar quite a bit for things that tended to be pretty niche and extreme.
    - gosub100 3y ago
      I don't think the "how to make $DANGEROUS_SUBSTANCE" is any easier with AI than with a search engine. However I could see it adding risk with evasion of countermeasures: "How do I get _____ on a plane?" "How do I obtain $PRECURSOR_CHEMICAL?"
      - ethbr1 3y ago
        AI guided step-by-steps can fill in for a lack of rudimentary knowledge, as long as one can follow instructions. Conversational interfaces definitely increase the accessibility of knowledge. And critically, SaaS AI platforms increase the availability of AI. E.g. the person who wouldn't be able to set up and run a local model, but can click a button on a website. It seems reasonable to preclude SaaS platforms from making it trivial to produce the worse societal harms. E.g. prevent stable diffusion services from returning celebrities or politicians, or LLMs from producing political content. Sure, it's still possible. But a knee high barrier at least keeps out those who aren't smart enough to step over it.
        pests 3y ago
        > Conversational interfaces definitely increase the accessibility of knowledge. Shouldn't increasing the accessibility of knowledge be a good thing but yet your tone seems to imply the opposite?
        ethbr1 3y ago
        Depends on the value you ascribe to people's use of easy knowledge. Circa-1995, I would have said uncategorically yes! It's a wonderful thing! Today? I'm much more of the opinion that knowledge hard-earned is knowledge valued and respected. And trivially-earned is... not. I don't think it's possible to suppress knowledge. Even about NBC weapons. But I'm on the fence as to whether "putting it on the high shelf where anyone has to work to get it" is a net positive or negative for society as a whole.
        s3p 3y ago
        Even knowledge of how to commit genocide or manufacture chemical weapons?
  - nopinsight 3y ago
    Given how fast AI has improved in recent years, can we be certain no malicious group will discover a way to engineer biological weapons or pandemic-inducing pathogens using near-future AI? Moreover, once an AI with such capability is open source, there's practically no way to put it back into Pandora's box. Implementing proper and judicious regulations will reduce the risks to everyone.
  - jstarfish 3y ago
    > this presents additional risk from non-state actors, but there's no fundamentally new risk here. This is splitting hairs for no real purpose. Additional risk is new risk. > By the mid 1920s there was already enough chemical agents to kill most of the population of Europe. By the 1970s there were enough in global stockpiles to kill every human on the planet several times over. Those global stockpiles continue to be controlled by state actors though, not aggrieved civilians. Once we lost that advantage, by the 1990s we had civilians manufacturing and releasing sarin gas in subways and detonating trucks full of fertilizer. We really don't want kids escalating from school shootings to synthesis and deployment of mustard gas.
    - somenameforme 3y ago
      Wiki has a pretty nice article on what went into the sarin attack. [1] A brief quote: --- "The Satyan-7 facility was declared ready for occupancy by September 1993 with the capacity to produce about 40–50 litres (11–13 US gal) of sarin, being equipped with 30-litre (7.9 US gal) capacity mixing flasks within protective hoods, and eventually employing 100 Aum members; the UN would later estimate the value of the building and its contents at $30 million.[23] Despite the safety features and often state-of-the-art equipment and practices, the operation of the facility was very unsafe – one analyst would later describe the cult as having a "high degree of book learning, but virtually nothing in the way of technical skill."[24]" --- All of those hundreds of workers, countless experts working for who knows how many man hours, and just massive scale development culminated in a subway attack carried out on 3 lines, during rush hour. It killed a total of 13 people. Imagine if they just bought a bunch of cars and started running people over. Many of these things sound absolutely terrifying, but in practice they are not such a threat except when carried out at a military level of scale and development. [1] - https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
    - NovemberWhiskey 3y ago
      >We really don't want kids escalating from school shootings to synthesis and deployment of mustard gas. I mean, you can make chlorine gas by mixing bleach and vinegar.
    - 3y ago
  - zarzavat 3y ago
    A lot of knowledge is locked up in the chemical profession. The intersection between qualified chemists and crazy people is, absolutely, a small number. If regular people start to get access to that knowledge it could be a problem.
    - serf 3y ago
      >If regular people start to get access to that knowledge it could be a problem. so when are we going to start regulating and restricting the sale of education/text books? a knowledge portal isn't a new concept.
      - nopinsight 3y ago
        Knowledge how to manufacture chemical weapons at scale is regulated as well. See: https://en.wikipedia.org/wiki/Chemical_Weapons_Convention https://en.wikipedia.org/wiki/Chemical_Weapons_Convention Moreover, current AI can be turned into an agent using basic programming knowledge. Such an agent is not very capable yet, but it's getting better by the month.
        ben_w 3y ago
        > Knowledge how to manufacture chemical weapons at scale is regulated as well. Kinda, but also no. I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry. Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack. It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas. I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".
        ndriscoll 3y ago
        Right now LLMs are trained on publicly available information, so if that knowledge is guarded, and if an LLM can provide it, then it's not guarded very well.
  - king_magic 3y ago
    > but there's no fundamentally new risk here This is incredibly naive. These models unlock capabilities for previously unsophisticated actors to do extremely dangerous things in almost undetectable ways.
    - throwaway4aday 3y ago
      you can't fix stupid
  - deleted 3y ago
    [deleted]
  - ben_w 3y ago
    > Yes, this presents additional risk from non-state actors, but there's no fundamentally new risk here. That doesn't seem right. Surely, making it easier for non-state actors to do things that state actors only fail to do because they agreed to treaties banning it, can only increase the risk that non-state actors may do those things? Laser blinding weapons are banned by treaty, widespread access to lasers lead to scenes like this a decade ago during the Arab Spring: https://www.bbc.com/news/av/world-middle-east-23182254 https://www.bbc.com/news/av/world-middle-east-23182254
- whymauri 3y ago
  As someone who has worked on ADMET risk for algorithmically designed drugs, this is a nothing burger. "Potentially lethal molecules" is a far cry away from "molecule that can be formulated and widely distributed to a lethal effect." It is as detached as "potentially promising early stage treatment" is from "manufactured and patented cure." I would argue the Verge's framing is worse. "Potentially lethal molecule" captures _every_ feasible molecule, given that anyone who has worked on ADMET is aware of the age-old adage: the dose makeths the poison. At a sufficiently high dose, virtually any output from an algorithmic drug design algorithm, be it combinatorial or 'AI', will be lethal. Would a traditional, non-neural net algorithm produce virtually the same results given the same objective function and apriori knowledge of toxic drug examples? Absolutely. You don't need a DNN for that, we've had the technology since the 90s.
- qualifiedai 3y ago
  A grad student in Systems Biology and 20k in funding is capable of generating much more "interesting" things than toxic molecules. (Such things are banned by Asilomar's 1975 convention though)
avmich 3y ago
It's true that immediate problems with AI are different, but we hope to be able to solve those problems and to have time for that. The risks addressed in the article could leave us less time and ability to properly solve when they grow to the obvious size, so that requires thinking ahead.
SoftTalker 3y ago
Inclined to agree. Clearly Biden doesn't know the first thing about it (I would say the same about any president BTW). So who really wrote the regulations he is announcing, and who are they listening to?
cma 3y ago
Doesn't it mention all those things?

nojito 3y ago

How does providing research grants to small independent researchers satisfying incumbents?

nopinsight 3y ago

Current AI is already capable of designing toxic molecules. Dual use of artificial-intelligence-powered drug discovery https://www.nature.com/articles/s42256-022-00465-9.epdf https://www.nature.com/articles/s42256-022-00465-9.epdf Interview with the lead author here: "AI suggested 40,000 new possible chemical weapons in just six hours / ‘For me, the concern was just how easy it was to do’" https://www.theverge.com/2022/3/17/22983197/ai-new-possible-chemical-weapons-generative-models-vx https://www.theverge.com/2022/3/17/22983197/ai-new-possible-...

yabones 3y ago
Chemical weapons are already a solved problem. By the mid 1920s there was already enough chemical agents to kill most of the population of Europe. By the 1970s there were enough in global stockpiles to kill every human on the planet several times over. Yes, this presents additional risk from non-state actors, but there's no fundamentally new risk here.
- BowBun 3y ago
  I agree in general. However much like how the rise of 'script kiddies' meant that inexperienced, sometimes underage kids get involved with hacking, one can worry the same can happen with AI-enabled activities. I've spent enough time in the shady parts of the internet to realize that people that spend significant time learning about niche/dangerous hobbies _tend_ to realize the seriousness of it. My fear with bio-weapons would be some 13-year-old being given step-by-step instructions with almost 0 effort to create something truly dangerous. It lowers the bar quite a bit for things that tended to be pretty niche and extreme.
  - gosub100 3y ago
    I don't think the "how to make $DANGEROUS_SUBSTANCE" is any easier with AI than with a search engine. However I could see it adding risk with evasion of countermeasures: "How do I get _____ on a plane?" "How do I obtain $PRECURSOR_CHEMICAL?"
    - ethbr1 3y ago
      AI guided step-by-steps can fill in for a lack of rudimentary knowledge, as long as one can follow instructions. Conversational interfaces definitely increase the accessibility of knowledge. And critically, SaaS AI platforms increase the availability of AI. E.g. the person who wouldn't be able to set up and run a local model, but can click a button on a website. It seems reasonable to preclude SaaS platforms from making it trivial to produce the worse societal harms. E.g. prevent stable diffusion services from returning celebrities or politicians, or LLMs from producing political content. Sure, it's still possible. But a knee high barrier at least keeps out those who aren't smart enough to step over it.
      - pests 3y ago
        > Conversational interfaces definitely increase the accessibility of knowledge. Shouldn't increasing the accessibility of knowledge be a good thing but yet your tone seems to imply the opposite?
        ethbr1 3y ago
        Depends on the value you ascribe to people's use of easy knowledge. Circa-1995, I would have said uncategorically yes! It's a wonderful thing! Today? I'm much more of the opinion that knowledge hard-earned is knowledge valued and respected. And trivially-earned is... not. I don't think it's possible to suppress knowledge. Even about NBC weapons. But I'm on the fence as to whether "putting it on the high shelf where anyone has to work to get it" is a net positive or negative for society as a whole.
        s3p 3y ago
        Even knowledge of how to commit genocide or manufacture chemical weapons?
- nopinsight 3y ago
  Given how fast AI has improved in recent years, can we be certain no malicious group will discover a way to engineer biological weapons or pandemic-inducing pathogens using near-future AI? Moreover, once an AI with such capability is open source, there's practically no way to put it back into Pandora's box. Implementing proper and judicious regulations will reduce the risks to everyone.
- jstarfish 3y ago
  > this presents additional risk from non-state actors, but there's no fundamentally new risk here. This is splitting hairs for no real purpose. Additional risk is new risk. > By the mid 1920s there was already enough chemical agents to kill most of the population of Europe. By the 1970s there were enough in global stockpiles to kill every human on the planet several times over. Those global stockpiles continue to be controlled by state actors though, not aggrieved civilians. Once we lost that advantage, by the 1990s we had civilians manufacturing and releasing sarin gas in subways and detonating trucks full of fertilizer. We really don't want kids escalating from school shootings to synthesis and deployment of mustard gas.
  - somenameforme 3y ago
    Wiki has a pretty nice article on what went into the sarin attack. [1] A brief quote: --- "The Satyan-7 facility was declared ready for occupancy by September 1993 with the capacity to produce about 40–50 litres (11–13 US gal) of sarin, being equipped with 30-litre (7.9 US gal) capacity mixing flasks within protective hoods, and eventually employing 100 Aum members; the UN would later estimate the value of the building and its contents at $30 million.[23] Despite the safety features and often state-of-the-art equipment and practices, the operation of the facility was very unsafe – one analyst would later describe the cult as having a "high degree of book learning, but virtually nothing in the way of technical skill."[24]" --- All of those hundreds of workers, countless experts working for who knows how many man hours, and just massive scale development culminated in a subway attack carried out on 3 lines, during rush hour. It killed a total of 13 people. Imagine if they just bought a bunch of cars and started running people over. Many of these things sound absolutely terrifying, but in practice they are not such a threat except when carried out at a military level of scale and development. [1] - https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack https://en.wikipedia.org/wiki/Tokyo_subway_sarin_attack
  - NovemberWhiskey 3y ago
    >We really don't want kids escalating from school shootings to synthesis and deployment of mustard gas. I mean, you can make chlorine gas by mixing bleach and vinegar.
  - 3y ago
- zarzavat 3y ago
  A lot of knowledge is locked up in the chemical profession. The intersection between qualified chemists and crazy people is, absolutely, a small number. If regular people start to get access to that knowledge it could be a problem.
  - serf 3y ago
    >If regular people start to get access to that knowledge it could be a problem. so when are we going to start regulating and restricting the sale of education/text books? a knowledge portal isn't a new concept.
    - nopinsight 3y ago
      Knowledge how to manufacture chemical weapons at scale is regulated as well. See: https://en.wikipedia.org/wiki/Chemical_Weapons_Convention https://en.wikipedia.org/wiki/Chemical_Weapons_Convention Moreover, current AI can be turned into an agent using basic programming knowledge. Such an agent is not very capable yet, but it's getting better by the month.
      - ben_w 3y ago
        > Knowledge how to manufacture chemical weapons at scale is regulated as well. Kinda, but also no. I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry. Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack. It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas. I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".
      - ndriscoll 3y ago
        Right now LLMs are trained on publicly available information, so if that knowledge is guarded, and if an LLM can provide it, then it's not guarded very well.
- king_magic 3y ago
  > but there's no fundamentally new risk here This is incredibly naive. These models unlock capabilities for previously unsophisticated actors to do extremely dangerous things in almost undetectable ways.
  - throwaway4aday 3y ago
    you can't fix stupid
- deleted 3y ago
  [deleted]
- ben_w 3y ago
  > Yes, this presents additional risk from non-state actors, but there's no fundamentally new risk here. That doesn't seem right. Surely, making it easier for non-state actors to do things that state actors only fail to do because they agreed to treaties banning it, can only increase the risk that non-state actors may do those things? Laser blinding weapons are banned by treaty, widespread access to lasers lead to scenes like this a decade ago during the Arab Spring: https://www.bbc.com/news/av/world-middle-east-23182254 https://www.bbc.com/news/av/world-middle-east-23182254
whymauri 3y ago
As someone who has worked on ADMET risk for algorithmically designed drugs, this is a nothing burger. "Potentially lethal molecules" is a far cry away from "molecule that can be formulated and widely distributed to a lethal effect." It is as detached as "potentially promising early stage treatment" is from "manufactured and patented cure." I would argue the Verge's framing is worse. "Potentially lethal molecule" captures _every_ feasible molecule, given that anyone who has worked on ADMET is aware of the age-old adage: the dose makeths the poison. At a sufficiently high dose, virtually any output from an algorithmic drug design algorithm, be it combinatorial or 'AI', will be lethal. Would a traditional, non-neural net algorithm produce virtually the same results given the same objective function and apriori knowledge of toxic drug examples? Absolutely. You don't need a DNN for that, we've had the technology since the 90s.
qualifiedai 3y ago
A grad student in Systems Biology and 20k in funding is capable of generating much more "interesting" things than toxic molecules. (Such things are banned by Asilomar's 1975 convention though)

avmich 3y ago

It's true that immediate problems with AI are different, but we hope to be able to solve those problems and to have time for that. The risks addressed in the article could leave us less time and ability to properly solve when they grow to the obvious size, so that requires thinking ahead.

SoftTalker 3y ago

Inclined to agree. Clearly Biden doesn't know the first thing about it (I would say the same about any president BTW). So who really wrote the regulations he is announcing, and who are they listening to?

cma 3y ago

Doesn't it mention all those things?