3 ms·

A lot of knowledge is locked up in the chemical profession. The intersection between qualified chemists and crazy people is, absolutely, a small number. If regu

A lot of knowledge is locked up in the chemical profession. The intersection between qualified chemists and crazy people is, absolutely, a small number. If regular people start to get access to that knowledge it could be a problem.

serf 3y ago
>If regular people start to get access to that knowledge it could be a problem. so when are we going to start regulating and restricting the sale of education/text books? a knowledge portal isn't a new concept.
- nopinsight 3y ago
  Knowledge how to manufacture chemical weapons at scale is regulated as well. See: https://en.wikipedia.org/wiki/Chemical_Weapons_Convention https://en.wikipedia.org/wiki/Chemical_Weapons_Convention Moreover, current AI can be turned into an agent using basic programming knowledge. Such an agent is not very capable yet, but it's getting better by the month.
  - ben_w 3y ago
    > Knowledge how to manufacture chemical weapons at scale is regulated as well. Kinda, but also no. I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry. Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack. It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas. I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".
  - ndriscoll 3y ago
    Right now LLMs are trained on publicly available information, so if that knowledge is guarded, and if an LLM can provide it, then it's not guarded very well.
    - waveBidder 3y ago
      the legislation is trying to get in front of the problem, rather than implement these things after Penn station gets gassed.
      - ndriscoll 3y ago
        It's not though because you can just buy chemistry books on Amazon, or if you want privacy, with cash at a university bookstore. The information is not at all controlled. People have to be warned not to mix certain household products so as not to create deadly gas by accident.
  - jandrewrogers 3y ago
    This knowledge isn't regulated. Anyone with a degree in chemistry can successfully synthesize chemical weapons. This is all public domain knowledge and the chemistry is relatively simple. The technical execution is the hard part but many, many people have these lab/engineering skills. Delivery systems are the hardest part but those are military implementation details and therefore non-public. It is the same with explosives. Anyone with chemistry skills could synthesize high-performance military explosives, it isn't difficult. Nonetheless, bombings tend to be low-grade explosives like ANFO or garbage explosives like TATP, because the people with the skills aren't the same people that do bombings. As a chemist you are required to be knowledgeable in these things in part because it is relatively easy to inadvertently synthesize chemicals with rather dangerous properties. Part of the job is knowing what to not do for safety reasons.
somenameforme 3y ago
I think as most of us are software people, in mind if not profession, it gives a misleading perception on where the difficulty in many things is. The barrier there is not just knowledge. In fact, there are countless papers available with quite detailed information on how to create chemical weapons. But knowledge is just a starting point. Technical skill, resources, production, manufacturing, and deployment are all major steps where again the barrier is not just knowledge. For instance there's a pretty huge culture around building your own nuclear fusion device at home. And there are tremendous resources available as well as step by step guides on how to do it. It's still exceptionally difficult (as well as quite dangerous), because it's not like you just get the pieces, put everything together like legos, flick on the switch, and boom you have nuclear fusion. There's a million things that not only can but will go wrong. So in spite of the absolutely immense amount of information about there, it's still a huge achievement for any individual or group to achieve fusion. And now somebody trying to do any of these sort of things with the guidance of... chatbots? It just seems like the most probable outcome is you end up getting yourself killed.
- fragmede 3y ago
  What story about home made nuclear devices would be complete without a mention of David Hahn, aka the "Nuclear Boy Scout" who built a homemade neutron source at the age of seventeen out of smoke detectors. He did not achieve fusion, but he did get the attention of the FBI, the NRC, and the EPA. He didn't have anywhere near enough to make a dirty bomb, nor did he ever consider making a bomb in the first place*. Why do I bring up David Hahn if he never achieved fusion and wasn't a terrorist? Because of how far he got as a seventeen year old. A fourty year old with a FAANG salary with the ideological bent of Theodore Kaczynski could do stupid amounts of damage. First would be to not try and build a nuclear fusion device. The difficult of building one doesn't seem so important if you're a sociopath when trying to be being a terrorist if every sociopath can go out and buy a gun and head to the local mall. There were two major such incidents in the past weeks, with 12 more mass shootings from Friday to Sunday over this past Halloween weekend**. Instead of worrying about the far-fetched, we would do better addressing something that killed 18 people in Maine and 19 in Texas, and 11 more across the country. * https://www.pbs.org/newshour/science/building-a-better-breeder-reactor-1 https://www.pbs.org/newshour/science/building-a-better-breed... ** https://www.npr.org/2023/10/29/1209340362/mass-shootings-halloween-tampa-chicago-atlanta https://www.npr.org/2023/10/29/1209340362/mass-shootings-hal...
throwaway4aday 3y ago
we should ban chemistry text books
- psychlops 3y ago
  And consider registering chemists with strict controls and monitoring.

serf 3y ago

>If regular people start to get access to that knowledge it could be a problem. so when are we going to start regulating and restricting the sale of education/text books? a knowledge portal isn't a new concept.

nopinsight 3y ago
Knowledge how to manufacture chemical weapons at scale is regulated as well. See: https://en.wikipedia.org/wiki/Chemical_Weapons_Convention https://en.wikipedia.org/wiki/Chemical_Weapons_Convention Moreover, current AI can be turned into an agent using basic programming knowledge. Such an agent is not very capable yet, but it's getting better by the month.
- ben_w 3y ago
  > Knowledge how to manufacture chemical weapons at scale is regulated as well. Kinda, but also no. I learned two distinct ways to make a poisonous gas from only normal kitchen supplies while at school, and I have only a GCSE grade B in Chemistry. Took me another decade to learn that specific chemical could be pressure-liquified in standard 2 litre soda bottles. That combination could wipe out an underground railway station from what fits in a moderately sized rucksack. It would still be a horrifically bad idea to attempt this DIY, even if you had a legit use for it, given it's a poisonous gas. I really don't want to be present for a live-action demonstration of someone doing this with a Spot robot, let alone with a more potent chemical agent they got from an LLM whose alignment is "Do Anything Now".
- ndriscoll 3y ago
  Right now LLMs are trained on publicly available information, so if that knowledge is guarded, and if an LLM can provide it, then it's not guarded very well.
  - waveBidder 3y ago
    the legislation is trying to get in front of the problem, rather than implement these things after Penn station gets gassed.
    - ndriscoll 3y ago
      It's not though because you can just buy chemistry books on Amazon, or if you want privacy, with cash at a university bookstore. The information is not at all controlled. People have to be warned not to mix certain household products so as not to create deadly gas by accident.
- jandrewrogers 3y ago
  This knowledge isn't regulated. Anyone with a degree in chemistry can successfully synthesize chemical weapons. This is all public domain knowledge and the chemistry is relatively simple. The technical execution is the hard part but many, many people have these lab/engineering skills. Delivery systems are the hardest part but those are military implementation details and therefore non-public. It is the same with explosives. Anyone with chemistry skills could synthesize high-performance military explosives, it isn't difficult. Nonetheless, bombings tend to be low-grade explosives like ANFO or garbage explosives like TATP, because the people with the skills aren't the same people that do bombings. As a chemist you are required to be knowledgeable in these things in part because it is relatively easy to inadvertently synthesize chemicals with rather dangerous properties. Part of the job is knowing what to not do for safety reasons.

somenameforme 3y ago

I think as most of us are software people, in mind if not profession, it gives a misleading perception on where the difficulty in many things is. The barrier there is not just knowledge. In fact, there are countless papers available with quite detailed information on how to create chemical weapons. But knowledge is just a starting point. Technical skill, resources, production, manufacturing, and deployment are all major steps where again the barrier is not just knowledge. For instance there's a pretty huge culture around building your own nuclear fusion device at home. And there are tremendous resources available as well as step by step guides on how to do it. It's still exceptionally difficult (as well as quite dangerous), because it's not like you just get the pieces, put everything together like legos, flick on the switch, and boom you have nuclear fusion. There's a million things that not only can but will go wrong. So in spite of the absolutely immense amount of information about there, it's still a huge achievement for any individual or group to achieve fusion. And now somebody trying to do any of these sort of things with the guidance of... chatbots? It just seems like the most probable outcome is you end up getting yourself killed.

fragmede 3y ago
What story about home made nuclear devices would be complete without a mention of David Hahn, aka the "Nuclear Boy Scout" who built a homemade neutron source at the age of seventeen out of smoke detectors. He did not achieve fusion, but he did get the attention of the FBI, the NRC, and the EPA. He didn't have anywhere near enough to make a dirty bomb, nor did he ever consider making a bomb in the first place*. Why do I bring up David Hahn if he never achieved fusion and wasn't a terrorist? Because of how far he got as a seventeen year old. A fourty year old with a FAANG salary with the ideological bent of Theodore Kaczynski could do stupid amounts of damage. First would be to not try and build a nuclear fusion device. The difficult of building one doesn't seem so important if you're a sociopath when trying to be being a terrorist if every sociopath can go out and buy a gun and head to the local mall. There were two major such incidents in the past weeks, with 12 more mass shootings from Friday to Sunday over this past Halloween weekend**. Instead of worrying about the far-fetched, we would do better addressing something that killed 18 people in Maine and 19 in Texas, and 11 more across the country. * https://www.pbs.org/newshour/science/building-a-better-breeder-reactor-1 https://www.pbs.org/newshour/science/building-a-better-breed... ** https://www.npr.org/2023/10/29/1209340362/mass-shootings-halloween-tampa-chicago-atlanta https://www.npr.org/2023/10/29/1209340362/mass-shootings-hal...

throwaway4aday 3y ago

we should ban chemistry text books

psychlops 3y ago
And consider registering chemists with strict controls and monitoring.