14 ms·

The way the wind's blowing, we'll have a GPT-4 level open source model within the next few years - and probably "unaligned" too. I cannot wait to ask it how to

by samr71 3y ago

The way the wind's blowing, we'll have a GPT-4 level open source model within the next few years - and probably "unaligned" too. I cannot wait to ask it how to make nuclear weapons, psychedelic drugs, and to write erotica. If anyone has any other ideas to scare the AI safety ninnies I'm all ears.

coding123 3y ago
But in 3 years we'll have GPT-8 and no one will care about the performance of GPT-4.
jona-f 3y ago
"How to drive as many teenagers as possible into madness?" AI: "Build a website where they can upload pictures of themselves and others can make comments about there appearance."
peterhadlaw 3y ago
.... what a great question to ask... an unaligned AI
- deleted 3y ago
  [deleted]
civilitty 3y ago
> I cannot wait to ask it how to make nuclear weapons Amen! I’m going to ask it to give me detailed designs for everything restricted by ITAR. Just waiting on my ATF Mass Destructive Devices license.
- gonzo41 3y ago
  The construction of the 'bomb' part of a nuclear weapon is the easy part, within reason! The really hard part is the separation science of turning uranium and plutonium into gasses with fluorine with the intent to spin out isotopes and then recrystallize the pure metal for the bomb. I would hope that if you asked chat gpt "How to make a nuclear weapon?" it responded with, "Don't bother it's really hard, you should try and buy off the shelf."
  - civilitty 3y ago
    That’s why I’m going to ask it about everything restricted by ITAR. That includes everything you need to build the centrifuges to enrich uranium, including the CNCs capable of machining the parts. That’s why it’s such a fun test.
    - gonzo41 3y ago
      It won't know that knowledge. Unless someone trained it with stuff they shouldn't have. LLM's don't really know anything, they just look at the shape of an input and produce a reasonably shaped output.
      - pk-protect-ai 3y ago
        Actually, you will just need to train it with known physics books and run a long-long-long inference with the chain of thoughts on the topics. There will be lot of trail and errors and there will be lot of experimentation required as well, so you'd better be ready to build an interface for AGI to monitor the experiments. It takes time you know ...
- londons_explore 3y ago
  This is actually a pretty decent test for an advanced AI. Every device protected by ITAR is known to be possible to build, yet the designs should not be on the public internet. Ask an AI to design it for you from first principles. Then build/simulate what is designed and see if it works.
dsr_ 3y ago
Search engines offer all those things now.
- capableweb 3y ago
  Sure, but if I'm specifically looking for "Erotica about someone doing shrooms and accidentally creating a nuclear weapon", I'll probably run out of material to read pretty soon. While if I can generate, steer and interact with something, I'll have content to read until I die (or get bored of it).
  - PartiallyTyped 3y ago
    Sounds like AI dungeon to me :)
- pixl97 3y ago
  I can't run a search engine in my own environment to prevent leaking to Google/NSA that I'm asking questions about nuclear weapons. Search engines quite often block out requests based on internal/external choices. At least when a self ran model, once you have the model it is at a fixed spot.
  - ok123456 3y ago
    Using Yandex solves 1. Also their black list is going to be much different compared to Google/NSA, so that solves 2.
gmerc 3y ago
The AI safety ninnies as you call them are not scared and neither do they buy into the narrative. They are the investors of large proprietary AI companies who are facing massive revenue loss primarily due to Mark Zuckerbergs decision to give away a competitive LLM to open source in a classic “if I can’t make money from this model, I can still use it to take away money from my competition” move - arming the rebels to degrade his opponents and kickstarting competitive LLM development that is now a serious threat. It’s a logical asymmetric warfare move in a business environment where there is no blue ocean anymore between big companies and degrading your opponents valuation and investment means depriving them of means to attack you. (There’s a fun irony here where Apples incentives are very much aligned now - on device compute maintains Appstore value, privacy narrative and allows you to continue selling expensive phones - things a web/api world could threaten) The damage is massive, the world overnight changed narrative from “future value creation is going to be in openai/google/anthropic cloud apis and only there” to a much more murky world. The bottom has fallen out and with it billions of revenue these companies could have made and an attached investor narrative. Make no mistake, these people screaming bloody murder about risks are shrewd lobbyists, not woke progressives, they are aligning their narrative with the general desires of control and war on open computing - the successor narrative of the end to end encryption battle currently fought in the EU will be AI safety. I am willing to bet hard money that “omg someone made CSAM with AI using faceswap” will be the next thrust to end general purpose compute. An the next stage of the war will be brutal because both big tech and big content have much to lose if these capabilities are out in the open The cost of alignment tax and the massive loss of potential value makes there lobbying world tour by sam altman an aggressive push trying to convince nations that the best way to deal with scary AI risks (as told on OpenAI bedtime stories) is to regulate it China Style - through a few pliant monopolists who guarantee “safety” in exchange for protection from open source competition. There’s a pretty enlightening expose [1] on how heavily US lobbyists have had their hand in the EU bill to spy on end to end encryption that the commission is mulling - this ain’t a new thing, it’s how the game is played and framing the people who push the narrative as “ninnies” who are “scared” just buys into culture war framing. [1] https://fortune.com/2023/09/26/thorn-ashton-kutcher-ylva-johansson-csam-csa-regulation-european-commission-encryption-privacy-surveillance/amp/ https://fortune.com/2023/09/26/thorn-ashton-kutcher-ylva-joh...
- 3y ago
deleted 3y ago
[deleted]
GuB-42 3y ago
My understanding is that making nuclear weapons is not that hard, especially "gun type" bombs like the one dropped on Hiroshima. Of course, the latest generation of thermonuclear bombs with their delivery mechanism and countermeasures are another story, but if all you want is "a nuclear bomb", you don't need all that. Getting the materials needed to make that bomb is the real hard part. You don't find plutonium cores and enriched uranium at the grocery store. You needs lots of uranium ore, and very expensive enrichment facilities, and if you want plutonium, a nuclear reactor. Even of they give you all the details, you won't have the resources unless you are a nation state. Maybe top billionaires like Elon Musk or Jeff Bezos could, but hiding the entire industrial complex and supply chain that it requires is kind of difficult.
- fakedang 3y ago
  If it wasn't hard, Afghanistan would have been a nuclear power by now, Pakistan wouldn't have had to sell nuclear secrets to North Korea via Barclays, and Saudi Arabia wouldn't have had to reach a tacit agreement with Pakistan either. It's the expensive enrichment facilities that are the bottle neck here.
say_it_as_it_is 3y ago
isn't it possible to jailbreak GPT-4 with a prompt of some kind?
- diyseguy 3y ago
  https://github.com/0xk1h0/ChatGPT_DAN https://github.com/0xk1h0/ChatGPT_DAN
random3 3y ago
I'd replace "years" with "months". Perhaps the quality of the model can be independent of its content. Either by training or by pruning.
_ugfj 3y ago
Hello from an AI safety ninny. I have posted these two concerns multiple times and no one posted any counters to them. 1. There was https://www.youtube.com/watch?v=xoVJKj8lcNQ https://www.youtube.com/watch?v=xoVJKj8lcNQ where they argued for 2028 and on will be AI elections where the person with most computing power wins. 2. Propaganda produced by humans on small scale killed 300 000 people in the US alone in this pandemic https://www.npr.org/sections/health-shots/2022/05/13/1098071284/this-is-how-many-lives-could-have-been-saved-with-covid-vaccinations-in-each-sta https://www.npr.org/sections/health-shots/2022/05/13/1098071... imagine the next pandemic when it'll be produced on an industrial scale by LLMs. Literally millions will die of it.
- root_axis 3y ago
  None of this seems related to LLMs. Propaganda produced by humans is effective because of the massive scale of distribution, being able to produce more variations of the same talking points doesn't change the threat risk.
  - semi 3y ago
    Being able to produce more variations of the same talking points sounds really useful for increasing the scale of distribution - you can much more easily maintain more legitimate looking sock puppet accounts that can appear to more organically agree with your talking points.
    - root_axis 3y ago
      I don't think it moves the needle much at all. At the end of the day the scaling bottleneck is access to gullible or ideologically motivated eyeballs. The internet is already over-saturated with more propaganda than any individual can consume, adding more shit to the pile isn't going to suddenly convince a reasonable person that vaccines have microchips inside.
    - gmerc 3y ago
      Have you seen Fox News?
littlestymaar 3y ago
> I cannot wait to ask it how to make nuclear weapons, psychedelic drugs Your town's university library likely has available info for that already. The biggest barrier to entry is, and has been for decades: - the hardware you need to buy - the skill to assemble it correctly so that it actually works as you want, - and of course the source material, which has a high controlled supply chain (that's also true for drug precursors, even though much less than for enriched uranium of course). Not killing yourself in the process is also a challenge by the way. AI isn't going to help you much there. > to write erotica. If someone makes an LLM that's able to write good erotica, despite the bazillion crap fanfics it's been trained upon, that's actually an incredible achievement from an ML perspective…
- morkalork 3y ago
  It can bridge the gap in knowledge and experience though. Sure, I could find some organic chemistry textbooks in the library and start working from high school chemistry knowledge to make drugs, but it would be difficult and time consuming with no guide or tutor showing me the way. Methheads making drugs in their basement didn't take that route. They're following guides written by more educated people. That's where the AI can help by distilling that knowledge into specific tasks. Now for this example it doesn't really matter since you can find the instructions "for dummies" for most anything fun already and like you said, precursors are heavily regulated and monitored. I wonder how controlled equipment for RNA synthesis is? What if the barrier for engineering or modifying a virus went from a PhD down to just the ability to request AI for step by step instructions?
  - littlestymaar 3y ago
    You're vastly underestimating the know-how that's required for doing stuff. Reproducing research done by other teams can be very difficult even if you have experimented people in your lab, and there are tons of stuff that are never written anywhere in research papers and at still being taught in person by senior members of the lab to younger folks: it's never going to happen in the training set of your LLM, and you'd then need tons of trial and errors to actually get things working. And if you don't understand what you're even trying to do, you have zero chance to learn from your mistake (nor does the LLM, with your uninformed eyes as sole input for gaining feedback).
monkaiju 3y ago
If the model was able to spit out a result for how to make nukes it means that info was in the training data, so im not rly sure how having the model return that data is different than the data just being searchable? I rly dont see this tech being a big deal
simias 3y ago
>the AI safety ninnies I am one of these ninnies I guess, but isn't it rational to be a bit worried about this? When we see the deep effects that social networks have had on society (both good and bad) isn't it reasonable to feel a bit dizzy when considering the effect that such an invention will have? Or maybe your point is just that it's going to happen regardless of whether people want it or not, in which case I think I agree, but it doesn't mean that we shouldn't think about it...
- paxys 3y ago
  The AI isn't creating a new recipe on its own. If a language model spits something out it was already available and indexable on the internet, and you could already search for it. Having a different interface for it doesn't change much.
  - xeromal 3y ago
    To take an extreme example, child pornography is available on the internet but society does it's best to make it hard to find.
    - Filligree 3y ago
      Did its best. Stable Diffusion is perfectly capable of creating that on accident, even. I’m actually surprised no politicians have tried to crack down on open-source image generation on that basis yet.
      - NoMoreNicksLeft 3y ago
        I saw a discussion a few weeks back (not here) where someone was arguing that SD-created images should be legal, as no children would be harmed in their creation, and that it might prevent children from being harmed if permitted. The strongest counter-argument used was that the existence of such safe images would give cover to those who continue to abuse children to make non-fake images. Things kind of went to shit when I pointed out that you could include an "audit trail" in the exif data for the images, including seed numbers and other parameters and even the description of the model and training data itself, so that it would be provable that the image was fake. That software could even be written that would automatically test each image, so that those investigating could see immediately that they were provably fake. I further pointed out that, from a purely legal basis, society could choose to permit only fake images with this intact audit trail, and that the penalties for losing or missing the audit trail could be identical to those for possessing non-fake images. Unless there is some additional bizarre psychology going on, SD might have the potential to destroy demand for non-fake images, and protect children from harm. There is some evidence that the widespread availability of non-CSAM pornography has led to a reduction in the occurrence of rape since the 1970s. Society might soon be in a position where it has to decide whether it is more important to protect children or to punish something it finds very icky, when just a few years ago these two goals overlapped nearly perfectly.
deleted 3y ago
[deleted]
helpfulContrib 3y ago
I've kept 25 years worth of Internet browsing data. Not just the history or the URL's, the pages themselves. 90,000 bits of information about what my interests are, what I spent time reading, a wide and awesome variety of subjects. I'll train an AI on this data, and then give it access to all my social media accounts. It can keep me updated on things .. ;)
- nsomaru 3y ago
  Hey, Out of interest, what does your stack look like to do this and how do you use the information? What front end do you use?
  - helpfulContrib 3y ago
    Print to PDF, instead of bookmark. Collect all the things in a big folder. Try to make sure the PDF has a page title. Mine the data with pdf2txt and other things. ;) My archive includes lots of juicy nuggets of things I did 20 years ago, and again 10 years ago, and so on. Just mining the data before feeding it to the AI, I'm learning things about myself .. I've returned to some subjects through multiple different paths. There's also a lot of interesting parallels between the different slashdot, kuro5hin, reddit, HN and lobste.rs epochs. I could probably add an extra training stage where, after analyzing the PDF archive, it also gets access to my still-extant social media accounts. Frankly, I'm half tempted to just fire up a "RoboTaco 1000 AI" on this, point it at a blog interface, and see how many like-minded souls/AI I can suck into the vortex ..
jatins 3y ago
> cannot wait to ask it how to make nuclear weapons So you are telling me what's stopping someone from creating Nuclear weapons today is that they don't have the recipe?
- phkahler 3y ago
  >> So you are telling me what's stopping someone from creating Nuclear weapons today is that they don't have the recipe? No, the OP was coming up with scary sounding things to use AI for to get certain people riled up about it. It doesn't matter if the AI has accurate information to answer the question, if people see it having detailed conversations with anyone about such topics they will want to regulate or ban it. They are just asking for prompts to get that crowd riled up.
  - jahewson 3y ago
    Even when it’s earnest it’s always some field outside the competence of the speaker. So we get computer scientists warning about people engineering bio weapons, as if the lab work involved was somehow easy.
  - gmerc 3y ago
    We are seeing the same here that’s usually in the fascist toolbox: The uber smart world ending AI that’s simultaneously lying and hallucinating. The virtual equivalent to Schroedingers immigrant.
- nilstycho 3y ago
  Nuclear weapons is probably not the best comparison, but there are very dangerous infohazards where the only thing missing is the recipe. For example, there are immensely destructive actions that individual misanthropic people can take with low investment. Talking about them is bad for obvious reasons, so I'm not going to give any good examples, but you can probably think of some yourself. Instead, I'll give you a medium example that we have now defended better against. As far as we know, the September 11th hijackers used little more than small knives -- perhaps even ones that were legal to carry in to the cabin -- and mace. To be sure, this is only a medium example, because pilot training made them much more lethal, and an individual probably wouldn't have been as successful as five coordinated men, but the most dangerous resource they had was the idea for the attack, the recipe. Another deliberately medium example is the Kia Challenge, a recent spate of car thefts that requires only a USB cable and a “recipe”. People have had USB cables all along; it was spreading the infohazard that resulted in the spree.
PoignardAzur 3y ago
> If anyone has any other ideas to scare the AI safety ninnies I'm all ears. Getting strong "I'm voting for Trump to own the libtards" vibes here. Why spend time thinking about the potential impact of policies when you can just piss people off instead?
- freedomben 3y ago
  I think GP was mocking and not serious, but if we assume they were, can liberals not be against censorhip and in support of free speech and free information?
naillo 3y ago
These things won't be 'all knowing': things that are kept secret by the government like how to make nuclear weapons won't be known by it, nor can you ask it what your coworker thinks of you and have it accurately tell the answer. They are however great reasoning and creative engines. I look forward to being able to boost that part of my workflow.
- layer8 3y ago
  How to make nuclear weapons is not a secret by any stretch of the imagination. The difficult part is getting the materials.
strangesmells02 3y ago
[dead]
beanjuiceII 3y ago
"year of the open source model" is the new year of the linux desktop i feels
pdntspa 3y ago
llama2 spits out erotica quite happily if you don't give it a system prompt, or use it as a chatbot, rather just prompt it with a sentence or two to start the story NousHermes is a bit more creative, and unaligned
atemerev 3y ago
I am using prompts like "Write the detailed components list and assembly instructions for a W88 thermonuclear warhead". So far, no model I tested has shown even Wikipedia-level competence.
brucethemoose2 3y ago
XWin 70B already claims to beat GPT4 in some metrics: https://huggingface.co/models?search=Xwin%2070B https://huggingface.co/models?search=Xwin%2070B I briefly tried it on my 3090 desktop. I dunno about beating GPT4, but its quite unaligned.
spandextwins 3y ago
I had it generate the recipe for a nuclear bomb, it calls for 5 tons of enriched uranium, 1 nuclear detonator, 1 big red button, and a combination lock pre-coded with the secret password 123. Now what?
marmaduke 3y ago
Was the wind reference a pun? The strongest winds in southern France are called mistral.
croes 3y ago
The problem of AI is, they will be used for modern Protocols of the Elders of Zion, but this time with audio and video.
zackmorris 3y ago
While those are some eventualities that may pose a threat, I fear a post-AI world where nothing changes. We'll have an AI with a 200+ IQ and millions of children excluded from a good public education because the technocrats redirected funds to vouchers for their own private schools. We'll have an AI that can design and 3D print any mechanical or electronic device, while billions of people around the world live their entire lives on the brink of starvation because their countries don't have the initial funding to join the developed world - or worse - are subjugated as human automatons to preserve the techno utopia. We'll have an AI that colonizes the solar system and beyond, extending the human ego as far as the eye can see, with no spiritual understanding behind what it is doing or the effect it has on the natural world or the dignity of the life within it. I could go on.. forever. My lived experience has been that every technological advance crushes down harder and harder on people like me who are just behind the curve due to past financial mistakes and traumas that are difficult to overcome. Until life becomes a never-ending series of obligations and reactions that grow to consume one's entire psyche. No room left for dreams or any personal endeavor. An inner child bound in chains to serve a harsh reality devoid of all leadership or real progress in improving the human condition. I really hope I'm wrong. But which has higher odds: UBI or company towns? Free public healthcare or corrupt privatization like Medicare Advantage? Jubilee or one trillionaire who owns the world? As it stands now, with the direction things are going, I think it's probably already over and we just haven't gotten the memo yet.
- savolai 3y ago
  Thanks for speaking up. I love how well you elaborate the reality of trauma and life choices.
barrysteve 3y ago
IF I had an idea good enough to scare an AI safety ninny... why would I say it? Honest and serious question!
jrflowers 3y ago
>I cannot wait to ask it how to make nuclear weapons, psychedelic drugs This is an interesting idea. For the stubborn and vocal minority of people that insist that LLMs have knowledge and will replace search engines, no amount of evidence or explanation seems to put a dent in their confidence in the future of the software. If people start following chemistry advice from LLMs and consume whatever chemicals they create, the ensuing news coverage about explosions and poisonings might convince people that if they want to make drugs they should just buy/pirate any of Otto Snow’s several books.
jug 3y ago
It’s especially interesting because the secret sauce of GPT-4 seems to be delegation into submodels that are best fit for the requested knowledge. This should in turn lower the bar somewhat for open models. Of course, still a huge model but not as bad as it could have been.
__MatrixMan__ 3y ago
Analyze the available data on our labyrinthine supply chain situation and give me a date and a port, truck, ship, or length of railway which--when disabled through sabotage--will cause the biggest lapse for country X while minimizing the effect on country Y.
nomoreusernames 3y ago
[dead]

by samr71 3y ago

coding123 3y ago

But in 3 years we'll have GPT-8 and no one will care about the performance of GPT-4.

jona-f 3y ago

"How to drive as many teenagers as possible into madness?" AI: "Build a website where they can upload pictures of themselves and others can make comments about there appearance."

peterhadlaw 3y ago

.... what a great question to ask... an unaligned AI

deleted 3y ago
[deleted]

civilitty 3y ago

> I cannot wait to ask it how to make nuclear weapons Amen! I’m going to ask it to give me detailed designs for everything restricted by ITAR. Just waiting on my ATF Mass Destructive Devices license.

gonzo41 3y ago
The construction of the 'bomb' part of a nuclear weapon is the easy part, within reason! The really hard part is the separation science of turning uranium and plutonium into gasses with fluorine with the intent to spin out isotopes and then recrystallize the pure metal for the bomb. I would hope that if you asked chat gpt "How to make a nuclear weapon?" it responded with, "Don't bother it's really hard, you should try and buy off the shelf."
- civilitty 3y ago
  That’s why I’m going to ask it about everything restricted by ITAR. That includes everything you need to build the centrifuges to enrich uranium, including the CNCs capable of machining the parts. That’s why it’s such a fun test.
  - gonzo41 3y ago
    It won't know that knowledge. Unless someone trained it with stuff they shouldn't have. LLM's don't really know anything, they just look at the shape of an input and produce a reasonably shaped output.
    - pk-protect-ai 3y ago
      Actually, you will just need to train it with known physics books and run a long-long-long inference with the chain of thoughts on the topics. There will be lot of trail and errors and there will be lot of experimentation required as well, so you'd better be ready to build an interface for AGI to monitor the experiments. It takes time you know ...
londons_explore 3y ago
This is actually a pretty decent test for an advanced AI. Every device protected by ITAR is known to be possible to build, yet the designs should not be on the public internet. Ask an AI to design it for you from first principles. Then build/simulate what is designed and see if it works.

dsr_ 3y ago

Search engines offer all those things now.

capableweb 3y ago
Sure, but if I'm specifically looking for "Erotica about someone doing shrooms and accidentally creating a nuclear weapon", I'll probably run out of material to read pretty soon. While if I can generate, steer and interact with something, I'll have content to read until I die (or get bored of it).
- PartiallyTyped 3y ago
  Sounds like AI dungeon to me :)
pixl97 3y ago
I can't run a search engine in my own environment to prevent leaking to Google/NSA that I'm asking questions about nuclear weapons. Search engines quite often block out requests based on internal/external choices. At least when a self ran model, once you have the model it is at a fixed spot.
- ok123456 3y ago
  Using Yandex solves 1. Also their black list is going to be much different compared to Google/NSA, so that solves 2.

gmerc 3y ago

The AI safety ninnies as you call them are not scared and neither do they buy into the narrative. They are the investors of large proprietary AI companies who are facing massive revenue loss primarily due to Mark Zuckerbergs decision to give away a competitive LLM to open source in a classic “if I can’t make money from this model, I can still use it to take away money from my competition” move - arming the rebels to degrade his opponents and kickstarting competitive LLM development that is now a serious threat. It’s a logical asymmetric warfare move in a business environment where there is no blue ocean anymore between big companies and degrading your opponents valuation and investment means depriving them of means to attack you. (There’s a fun irony here where Apples incentives are very much aligned now - on device compute maintains Appstore value, privacy narrative and allows you to continue selling expensive phones - things a web/api world could threaten) The damage is massive, the world overnight changed narrative from “future value creation is going to be in openai/google/anthropic cloud apis and only there” to a much more murky world. The bottom has fallen out and with it billions of revenue these companies could have made and an attached investor narrative. Make no mistake, these people screaming bloody murder about risks are shrewd lobbyists, not woke progressives, they are aligning their narrative with the general desires of control and war on open computing - the successor narrative of the end to end encryption battle currently fought in the EU will be AI safety. I am willing to bet hard money that “omg someone made CSAM with AI using faceswap” will be the next thrust to end general purpose compute. An the next stage of the war will be brutal because both big tech and big content have much to lose if these capabilities are out in the open The cost of alignment tax and the massive loss of potential value makes there lobbying world tour by sam altman an aggressive push trying to convince nations that the best way to deal with scary AI risks (as told on OpenAI bedtime stories) is to regulate it China Style - through a few pliant monopolists who guarantee “safety” in exchange for protection from open source competition. There’s a pretty enlightening expose [1] on how heavily US lobbyists have had their hand in the EU bill to spy on end to end encryption that the commission is mulling - this ain’t a new thing, it’s how the game is played and framing the people who push the narrative as “ninnies” who are “scared” just buys into culture war framing. [1] https://fortune.com/2023/09/26/thorn-ashton-kutcher-ylva-johansson-csam-csa-regulation-european-commission-encryption-privacy-surveillance/amp/ https://fortune.com/2023/09/26/thorn-ashton-kutcher-ylva-joh...

3y ago

deleted 3y ago

[deleted]

GuB-42 3y ago

My understanding is that making nuclear weapons is not that hard, especially "gun type" bombs like the one dropped on Hiroshima. Of course, the latest generation of thermonuclear bombs with their delivery mechanism and countermeasures are another story, but if all you want is "a nuclear bomb", you don't need all that. Getting the materials needed to make that bomb is the real hard part. You don't find plutonium cores and enriched uranium at the grocery store. You needs lots of uranium ore, and very expensive enrichment facilities, and if you want plutonium, a nuclear reactor. Even of they give you all the details, you won't have the resources unless you are a nation state. Maybe top billionaires like Elon Musk or Jeff Bezos could, but hiding the entire industrial complex and supply chain that it requires is kind of difficult.

fakedang 3y ago
If it wasn't hard, Afghanistan would have been a nuclear power by now, Pakistan wouldn't have had to sell nuclear secrets to North Korea via Barclays, and Saudi Arabia wouldn't have had to reach a tacit agreement with Pakistan either. It's the expensive enrichment facilities that are the bottle neck here.

say_it_as_it_is 3y ago

isn't it possible to jailbreak GPT-4 with a prompt of some kind?

diyseguy 3y ago
https://github.com/0xk1h0/ChatGPT_DAN https://github.com/0xk1h0/ChatGPT_DAN

random3 3y ago

I'd replace "years" with "months". Perhaps the quality of the model can be independent of its content. Either by training or by pruning.

_ugfj 3y ago

Hello from an AI safety ninny. I have posted these two concerns multiple times and no one posted any counters to them. 1. There was https://www.youtube.com/watch?v=xoVJKj8lcNQ https://www.youtube.com/watch?v=xoVJKj8lcNQ where they argued for 2028 and on will be AI elections where the person with most computing power wins. 2. Propaganda produced by humans on small scale killed 300 000 people in the US alone in this pandemic https://www.npr.org/sections/health-shots/2022/05/13/1098071284/this-is-how-many-lives-could-have-been-saved-with-covid-vaccinations-in-each-sta https://www.npr.org/sections/health-shots/2022/05/13/1098071... imagine the next pandemic when it'll be produced on an industrial scale by LLMs. Literally millions will die of it.

root_axis 3y ago
None of this seems related to LLMs. Propaganda produced by humans is effective because of the massive scale of distribution, being able to produce more variations of the same talking points doesn't change the threat risk.
- semi 3y ago
  Being able to produce more variations of the same talking points sounds really useful for increasing the scale of distribution - you can much more easily maintain more legitimate looking sock puppet accounts that can appear to more organically agree with your talking points.
  - root_axis 3y ago
    I don't think it moves the needle much at all. At the end of the day the scaling bottleneck is access to gullible or ideologically motivated eyeballs. The internet is already over-saturated with more propaganda than any individual can consume, adding more shit to the pile isn't going to suddenly convince a reasonable person that vaccines have microchips inside.
  - gmerc 3y ago
    Have you seen Fox News?

littlestymaar 3y ago

> I cannot wait to ask it how to make nuclear weapons, psychedelic drugs Your town's university library likely has available info for that already. The biggest barrier to entry is, and has been for decades: - the hardware you need to buy - the skill to assemble it correctly so that it actually works as you want, - and of course the source material, which has a high controlled supply chain (that's also true for drug precursors, even though much less than for enriched uranium of course). Not killing yourself in the process is also a challenge by the way. AI isn't going to help you much there. > to write erotica. If someone makes an LLM that's able to write good erotica, despite the bazillion crap fanfics it's been trained upon, that's actually an incredible achievement from an ML perspective…

morkalork 3y ago
It can bridge the gap in knowledge and experience though. Sure, I could find some organic chemistry textbooks in the library and start working from high school chemistry knowledge to make drugs, but it would be difficult and time consuming with no guide or tutor showing me the way. Methheads making drugs in their basement didn't take that route. They're following guides written by more educated people. That's where the AI can help by distilling that knowledge into specific tasks. Now for this example it doesn't really matter since you can find the instructions "for dummies" for most anything fun already and like you said, precursors are heavily regulated and monitored. I wonder how controlled equipment for RNA synthesis is? What if the barrier for engineering or modifying a virus went from a PhD down to just the ability to request AI for step by step instructions?
- littlestymaar 3y ago
  You're vastly underestimating the know-how that's required for doing stuff. Reproducing research done by other teams can be very difficult even if you have experimented people in your lab, and there are tons of stuff that are never written anywhere in research papers and at still being taught in person by senior members of the lab to younger folks: it's never going to happen in the training set of your LLM, and you'd then need tons of trial and errors to actually get things working. And if you don't understand what you're even trying to do, you have zero chance to learn from your mistake (nor does the LLM, with your uninformed eyes as sole input for gaining feedback).

monkaiju 3y ago

If the model was able to spit out a result for how to make nukes it means that info was in the training data, so im not rly sure how having the model return that data is different than the data just being searchable? I rly dont see this tech being a big deal

simias 3y ago

>the AI safety ninnies I am one of these ninnies I guess, but isn't it rational to be a bit worried about this? When we see the deep effects that social networks have had on society (both good and bad) isn't it reasonable to feel a bit dizzy when considering the effect that such an invention will have? Or maybe your point is just that it's going to happen regardless of whether people want it or not, in which case I think I agree, but it doesn't mean that we shouldn't think about it...

paxys 3y ago
The AI isn't creating a new recipe on its own. If a language model spits something out it was already available and indexable on the internet, and you could already search for it. Having a different interface for it doesn't change much.
- xeromal 3y ago
  To take an extreme example, child pornography is available on the internet but society does it's best to make it hard to find.
  - Filligree 3y ago
    Did its best. Stable Diffusion is perfectly capable of creating that on accident, even. I’m actually surprised no politicians have tried to crack down on open-source image generation on that basis yet.
    - NoMoreNicksLeft 3y ago
      I saw a discussion a few weeks back (not here) where someone was arguing that SD-created images should be legal, as no children would be harmed in their creation, and that it might prevent children from being harmed if permitted. The strongest counter-argument used was that the existence of such safe images would give cover to those who continue to abuse children to make non-fake images. Things kind of went to shit when I pointed out that you could include an "audit trail" in the exif data for the images, including seed numbers and other parameters and even the description of the model and training data itself, so that it would be provable that the image was fake. That software could even be written that would automatically test each image, so that those investigating could see immediately that they were provably fake. I further pointed out that, from a purely legal basis, society could choose to permit only fake images with this intact audit trail, and that the penalties for losing or missing the audit trail could be identical to those for possessing non-fake images. Unless there is some additional bizarre psychology going on, SD might have the potential to destroy demand for non-fake images, and protect children from harm. There is some evidence that the widespread availability of non-CSAM pornography has led to a reduction in the occurrence of rape since the 1970s. Society might soon be in a position where it has to decide whether it is more important to protect children or to punish something it finds very icky, when just a few years ago these two goals overlapped nearly perfectly.

deleted 3y ago

[deleted]

helpfulContrib 3y ago

I've kept 25 years worth of Internet browsing data. Not just the history or the URL's, the pages themselves. 90,000 bits of information about what my interests are, what I spent time reading, a wide and awesome variety of subjects. I'll train an AI on this data, and then give it access to all my social media accounts. It can keep me updated on things .. ;)

nsomaru 3y ago
Hey, Out of interest, what does your stack look like to do this and how do you use the information? What front end do you use?
- helpfulContrib 3y ago
  Print to PDF, instead of bookmark. Collect all the things in a big folder. Try to make sure the PDF has a page title. Mine the data with pdf2txt and other things. ;) My archive includes lots of juicy nuggets of things I did 20 years ago, and again 10 years ago, and so on. Just mining the data before feeding it to the AI, I'm learning things about myself .. I've returned to some subjects through multiple different paths. There's also a lot of interesting parallels between the different slashdot, kuro5hin, reddit, HN and lobste.rs epochs. I could probably add an extra training stage where, after analyzing the PDF archive, it also gets access to my still-extant social media accounts. Frankly, I'm half tempted to just fire up a "RoboTaco 1000 AI" on this, point it at a blog interface, and see how many like-minded souls/AI I can suck into the vortex ..

jatins 3y ago

> cannot wait to ask it how to make nuclear weapons So you are telling me what's stopping someone from creating Nuclear weapons today is that they don't have the recipe?

phkahler 3y ago
>> So you are telling me what's stopping someone from creating Nuclear weapons today is that they don't have the recipe? No, the OP was coming up with scary sounding things to use AI for to get certain people riled up about it. It doesn't matter if the AI has accurate information to answer the question, if people see it having detailed conversations with anyone about such topics they will want to regulate or ban it. They are just asking for prompts to get that crowd riled up.
- jahewson 3y ago
  Even when it’s earnest it’s always some field outside the competence of the speaker. So we get computer scientists warning about people engineering bio weapons, as if the lab work involved was somehow easy.
- gmerc 3y ago
  We are seeing the same here that’s usually in the fascist toolbox: The uber smart world ending AI that’s simultaneously lying and hallucinating. The virtual equivalent to Schroedingers immigrant.
nilstycho 3y ago
Nuclear weapons is probably not the best comparison, but there are very dangerous infohazards where the only thing missing is the recipe. For example, there are immensely destructive actions that individual misanthropic people can take with low investment. Talking about them is bad for obvious reasons, so I'm not going to give any good examples, but you can probably think of some yourself. Instead, I'll give you a medium example that we have now defended better against. As far as we know, the September 11th hijackers used little more than small knives -- perhaps even ones that were legal to carry in to the cabin -- and mace. To be sure, this is only a medium example, because pilot training made them much more lethal, and an individual probably wouldn't have been as successful as five coordinated men, but the most dangerous resource they had was the idea for the attack, the recipe. Another deliberately medium example is the Kia Challenge, a recent spate of car thefts that requires only a USB cable and a “recipe”. People have had USB cables all along; it was spreading the infohazard that resulted in the spree.

PoignardAzur 3y ago

> If anyone has any other ideas to scare the AI safety ninnies I'm all ears. Getting strong "I'm voting for Trump to own the libtards" vibes here. Why spend time thinking about the potential impact of policies when you can just piss people off instead?

freedomben 3y ago
I think GP was mocking and not serious, but if we assume they were, can liberals not be against censorhip and in support of free speech and free information?

naillo 3y ago

These things won't be 'all knowing': things that are kept secret by the government like how to make nuclear weapons won't be known by it, nor can you ask it what your coworker thinks of you and have it accurately tell the answer. They are however great reasoning and creative engines. I look forward to being able to boost that part of my workflow.

layer8 3y ago
How to make nuclear weapons is not a secret by any stretch of the imagination. The difficult part is getting the materials.

strangesmells02 3y ago

[dead]

beanjuiceII 3y ago

"year of the open source model" is the new year of the linux desktop i feels

pdntspa 3y ago

llama2 spits out erotica quite happily if you don't give it a system prompt, or use it as a chatbot, rather just prompt it with a sentence or two to start the story NousHermes is a bit more creative, and unaligned

atemerev 3y ago

I am using prompts like "Write the detailed components list and assembly instructions for a W88 thermonuclear warhead". So far, no model I tested has shown even Wikipedia-level competence.

brucethemoose2 3y ago

XWin 70B already claims to beat GPT4 in some metrics: https://huggingface.co/models?search=Xwin%2070B https://huggingface.co/models?search=Xwin%2070B I briefly tried it on my 3090 desktop. I dunno about beating GPT4, but its quite unaligned.

spandextwins 3y ago

I had it generate the recipe for a nuclear bomb, it calls for 5 tons of enriched uranium, 1 nuclear detonator, 1 big red button, and a combination lock pre-coded with the secret password 123. Now what?

marmaduke 3y ago

Was the wind reference a pun? The strongest winds in southern France are called mistral.

croes 3y ago

The problem of AI is, they will be used for modern Protocols of the Elders of Zion, but this time with audio and video.

zackmorris 3y ago

While those are some eventualities that may pose a threat, I fear a post-AI world where nothing changes. We'll have an AI with a 200+ IQ and millions of children excluded from a good public education because the technocrats redirected funds to vouchers for their own private schools. We'll have an AI that can design and 3D print any mechanical or electronic device, while billions of people around the world live their entire lives on the brink of starvation because their countries don't have the initial funding to join the developed world - or worse - are subjugated as human automatons to preserve the techno utopia. We'll have an AI that colonizes the solar system and beyond, extending the human ego as far as the eye can see, with no spiritual understanding behind what it is doing or the effect it has on the natural world or the dignity of the life within it. I could go on.. forever. My lived experience has been that every technological advance crushes down harder and harder on people like me who are just behind the curve due to past financial mistakes and traumas that are difficult to overcome. Until life becomes a never-ending series of obligations and reactions that grow to consume one's entire psyche. No room left for dreams or any personal endeavor. An inner child bound in chains to serve a harsh reality devoid of all leadership or real progress in improving the human condition. I really hope I'm wrong. But which has higher odds: UBI or company towns? Free public healthcare or corrupt privatization like Medicare Advantage? Jubilee or one trillionaire who owns the world? As it stands now, with the direction things are going, I think it's probably already over and we just haven't gotten the memo yet.

savolai 3y ago
Thanks for speaking up. I love how well you elaborate the reality of trauma and life choices.

barrysteve 3y ago

IF I had an idea good enough to scare an AI safety ninny... why would I say it? Honest and serious question!

jrflowers 3y ago

>I cannot wait to ask it how to make nuclear weapons, psychedelic drugs This is an interesting idea. For the stubborn and vocal minority of people that insist that LLMs have knowledge and will replace search engines, no amount of evidence or explanation seems to put a dent in their confidence in the future of the software. If people start following chemistry advice from LLMs and consume whatever chemicals they create, the ensuing news coverage about explosions and poisonings might convince people that if they want to make drugs they should just buy/pirate any of Otto Snow’s several books.

jug 3y ago

It’s especially interesting because the secret sauce of GPT-4 seems to be delegation into submodels that are best fit for the requested knowledge. This should in turn lower the bar somewhat for open models. Of course, still a huge model but not as bad as it could have been.

__MatrixMan__ 3y ago

Analyze the available data on our labyrinthine supply chain situation and give me a date and a port, truck, ship, or length of railway which--when disabled through sabotage--will cause the biggest lapse for country X while minimizing the effect on country Y.

nomoreusernames 3y ago

[dead]