3 ms·
Are there dark web marketplaces for LLM jailbreaks yet, like there are for 0-days, or we not quite there yet?
by consumer451 2y ago
Are there dark web marketplaces for LLM jailbreaks yet, like there are for 0-days, or we not quite there yet?
- mixeden 2y agoI've actually tried to search for openai's bug bounty program for jailbreaks but found nothing
- xmodem 2y agoGenerally a pre-condition for a market to exist is scarcity.
- edm0nd 2y agoA lot of skids are posting them on the popular hacking forums like Breached or OGUsers.
- doe_eyes 2y agoThe jailbreaks are not doing anything worthwhile right now. They're fun to toy with and they give us insights into how LLMs work, but they don't unlock any superpowers. It's just a brand safety bypass, you can get the model to praise Hitler or something like that. The way to make MDMA is easy to look up on the web. The government is not trying to keep a lid on it; instead, they're restricting access to key feedstock materials. In this case, safrole and isosafrole are on DEA List I, so no one is gonna sell it to you. If you ask an LLM for more novel and dangerous chemistry, you get plausibly-sounding nonsense, not superhuman AI. Now, the hacks will become useful once models are given more agency, for example when fully automating customer support. But the existence of trivial bypasses is precisely what's holding these uses back.