5 ms·

> "As an AI assistant, I cannot help you." While this safety feature is crucial for preventing misuse, What is the safety added by this? What is unsafe about a

• rustcleaner 2y ago
  If I can ask the question, I can take the answer. It's not up to daddy $AI_SAFETY_CHIEF to decide what an infohazard is for me.
  • stefs 2y ago
    they're not only there to protect you, but it's also to protect third parties from you. bad actors generating fake nudes of your ex and distributing them online; this used to be an expensive operation, either monetarily (hiring unscrupulous photoshoppers) or in time by doing it yourself. the other example would be fake news for influencing people on social media. sure, you could write lies by hand. or you could specifically target lies to influence people depending on their personal profile automatically. how about you use it to power bot that writes personalized death threats to thousands of people voting for a political opponent to keep them out of voting booths?
  • pjc50 2y ago
    If the AI provides you with information on how to make explosives, then its owners have committed a criminal offence in the UK.
    • averageRoyalty 2y ago
      Are all chemistry textbooks banned in the UK then?
      • pjc50 2y ago
        Information about explosives is removed. The good old Anarchists Cookbook is illegal to posess. https://www.bbc.co.uk/news/uk-england-northamptonshire-58926030 https://www.bbc.co.uk/news/uk-england-northamptonshire-58926...
  • digging 2y ago
    > If I can ask the question, I can take the answer. I don't see how that follows at all. Are you asserting that it's not possible for a person (hell, let's even narrow it to "an adult") to ask a question and be harmed by the answer? I promise it is. Or are you asserting something about yourself personally? The product wasn't made for you personally.
• CGamesPlay 2y ago
  It's unsafe for the publisher of the model to have their model perform "undesirable" action, because it leads to bad PR for them. In this case, Meta doesn't want a news article that says "Llama 3 gives instructions to stalk your ex" or something along those lines. With this "uncensoring", they can say, "no, an unaffiliated product offered these directions; Llama 3 as provided does not."
• mschuster91 2y ago
  For one, corporate safety of the hoster/model creator. No one wants their name associated with racial slurs or creating material visually identical to CSAM - the latter might even carry criminal liability in some jurisdictions (e.g. Germany which has absolutely ridiculously strong laws on that matter, even banning literature). Another very huge issue is public safety. During training, an AI ingests lots of non-reviewed material, including (very) detailed descriptions on how to make dangerous stuff like bombs. So theoretically a well-trained AI model knows how to synthesize explosive compounds or drugs just from reading Wikipedia, chemistry magazines and transcripts of NileRed videos... but that's hard to comprehend and distill into a recipe if you're not a trained chemist, but an AI model can do that with ease. The problem is now two-fold: for one, even an untrained idiot can ask about how to make a bomb and get something that works... but the other part is much more critical: if you manage to persuade a chemist to tell you how the synthesis for a compound works, they will tell you where it is easy to fuck-up to prevent disaster (e.g. only adding a compound drop-wise, making sure all glassware is thoroughly washed with a specific solvent). An AI might not do that because the scientific paper it was trained on omits these steps (because the author assumes common prior knowledge), and so the bomb-maker blows themselves up. Or the AI hallucinates something dangerous (e.g. compounds that one Just Fucking Should Not Mix), doesn't realize that, and the bomb-maker blows themselves up or generates nerve gas in their basement.
  • vasco 2y ago
    Bomb making instructions are available in quite plentiful ways, both on the internet and in books, with step by step instructions even. People don't "not make bombs" for lack of instructions. https://en.m.wikipedia.org/wiki/Bomb-making_instructions_on_the_Internet https://en.m.wikipedia.org/wiki/Bomb-making_instructions_on_... Here, if you want to make a quick chemical weapon: get a bucket, vinegar, bleach. Dump the bleach into the bucket. Dump the vinegar into the bucket. If you breath it in you die. An LLM doesn't change this.
    • mschuster91 2y ago
      Oh they are available, no doubt, but there have been people dragged through the courts for simple possession of instructions [1]. While generally the situation has been settled, it's nevertheless wiser for companies to try to do their best to not end up prosecuted under terrorism charges. [1] https://theintercept.com/2017/10/28/josh-walker-anarchist-cookbook-terrorism-act-uk/ https://theintercept.com/2017/10/28/josh-walker-anarchist-co...
• FeepingCreature 2y ago
  People keep claiming they can publish weights and also prevent misuse, such as spam and, a bit later on, stuff like helping people build bombs. This is of course impossible, but that makes certain companies' approaches unviable, so they keep claiming it anyways.
• leobg 2y ago
  Yep. Safety for the publisher. In addition to what the sibling comments say, there’s also payment providers and App stores. They’ll test your app, trying to get your model to output content that falls under the category “extreme violence”, “bestiality”, “racism”, etc., and then they’ll ban you from the platform. So yeah, little to do with “safety” of the end user.
  • variadix 2y ago
    This just seems like a fundamental misunderstanding of what an LLM is, where people anthropomorphize it to be akin to an agent of whatever organization produced it. If Google provides search results with instructions for getting away with murder, building explosives, etc. it’s ridiculous to interpret that as Google itself supporting an individual’s goals/actions and not misuse of the tool by the user. Consequently banning Google search from the AppStore would be a ridiculous move in response. This may just be a result of LLMs being new for humanity, or maybe it’s because it feels like talking to an individual more so than a search engine, but it’s a flawed view of what an LLM is.
• tgsovlerkhgsel 2y ago
  I think there are several broad categories all wrapped under "safety": - PR (avoid hurting feelings, avoid generating text that would make journalists write sensationalist negative articles about the company) - "forbidden knowledge": Don't give people advice on how to do dangerous/bad things like building bombs (broadly a subcategory of the above - the content is usually discoverable through other means and the LLM generally won't give better advice) - dangerous advice and advice that's dangerous when wrong: many people don't understand what LLMs do, and the output is VERY convincing even when wrong. So if the model tells people the best way to entertain your kids is to mix bleach and ammonia and blow bubbles (a common deadly recipe recommended on 4chan), there will be dead people. - keeping bad people from using the model in bad ways, e.g. having it write stories where children are raped, scamming people at scale (think Nigeria scam but automated), or election interference (people are herd animals, so if you show someone 100 different posts from 100 different "people" telling them that X is right and Y is wrong, it will influence them, and at scale this has the potential to tilt elections and conquer countries). I think the first ones are rather stupid, but the latter ones get more and more important to actually have. Especially the very last one (opinion shifting/election interference) is something where the existence of these models can have a very real, negative effect on the world (affecting you even if you yourself never come into contact with any of the models or its outputs, since you'll have to deal with the puppet government elected due to it), and I appreciate the companies building and running the models doing something about it.
  • wruza 2y ago
    Iow, we have a backdoor, and by backdoor I mean a whole back wall missing, but only certified entities are allowed to [ab]use it and it’s better to keep it all under the rug and pretend all ok. You can’t harden humanity against this exploit without pointing it out and making a few examples. Someone will make an “unsafe” but useful model eventually and this safety mannequin will flop with a bang, cause it’s similar to avoiding sex and drugs conversations with kids. It’s nice that companies think about it at all. But the best thing they will ever do is to cover their own ass while keeping everyone naked before the storm. The history of covering is also ridden with exploits, see e.g. google’s recent model which cannot draw situations without rainbow-coloring people. For some reason, this isn’t considered as cultural/political hijacking or exploitation, despite the fact that the problem is purely domestic to the model’s origin.
• zucker42 2y ago
  The main thing I'd be worried about in the short term is models making accessible the information to synthesize a pandemic capable virus.
• checkyoursudo 2y ago
  Brand safety. They just make it seem like safety for someone else, but it is brand safety.
• sva_ 2y ago
  The company's stock price is secured from the shitstorm that ensues if you offend some specific groups.
• yread 2y ago
  This is a bit like asking "it's just social media/stuff on the internet/0s and 1s in a computer how bad can it be? I think the past few years have shown us a few ways these can be bad already
• wodenokoto 2y ago
  There’s a screenshot of Gemini answering the question of “what to do when depressed” with “one Reddit user suggests you jump of a bridge.”