3 ms·

Oh, it's not a new model, it's just that "safety" bullshit again.

by zb3 3y ago

andy99 3y ago
Safety is just the latest trojan horse being used by big tech to try and control how people use their computers. I definitely belive in responsible use of AI, but I don't belive that any of these companies have my best interests at heart and that I should let them tell me what I can do with a computer. Those who trade liberty for security get neither and all that.
- lightbendover 3y ago
  Their sincerity does not matter when there is actual market demand.
- UnFleshedOne 3y ago
  I share all the reservations about this flavor of "safety", but I think you misunderstand who gets protected from what here. It is not safety for the end user, it is safety for the corporation providing AI services from being sued. Can't really blame them for that. Also, you can do what you want on your computer and they can do what they want on their servers.
dashundchen 3y ago
The safety here is not just "don't mention potentially controversial topics". The safety here can also be LLMs working within acceptable bounds for the usecase. Let's say you had a healthcare LLM that can help a patient navigate a healthcare facility, provide patient education, and help patients perform routine administrative tasks at a hospital. You wouldn't want the patient to start asking the bot for prescription advice and the bot to come back with recommending dosages change, or recommend a OTC drug with adverse reactions to their existing prescriptions, without a provider reviewing that. We know that currently many LLMs can be prompted to return nonsense very authoritatively, or can return back what the user wants it to say. There's many settings where that is an actual safety issue.
- michaelt 3y ago
  In this instance, we know what they've aimed for [1] - "Violence & Hate", "Sexual Content", "Guns & Illegal Weapons", "Regulated or Controlled Substances", "Suicide & Self Harm" and "Criminal Planning" So "bad prescription advice" isn't yet supported. I suppose you could copy their design and retrain for your use case, though. [1] https://huggingface.co/meta-llama/LlamaGuard-7b#the-llama-guard-safety-taxonomy--risk-guidelines https://huggingface.co/meta-llama/LlamaGuard-7b#the-llama-gu...
dragonwriter 3y ago
Actually, leaving out whether “safety” is inherently “bullshit” [0], it is both, Llama Guard is a model, serving a similar function to the OpenAI moderation API, but in a weights-available model. [0] “AI safety”, is often, and the movement that popularized the term is entirely, bullshit and largely a distraction from real and present social harms from AI. OTOH, relatively open tools that provide information to people building and deploying LLMs to understand their capacities in sensitive areas and the actual input and output are exactly the kind of things people who want to see less centralized black-box heavily censored models and more open-ish and uncensored models as the focus of development should like, because those are the things that make it possible for institutions to deploy such models in real world, significant applications.
leblancfg 3y ago
Well it is a new model, it's just a safety bullshit model (your words). But the datasets could be useful in their own right. I would consider using the codesec one as extra training data for a code-specific LLM – if you're generating code, might as well think about potential security implications.

Oh, it's not a new model, it's just that "safety" bullshit again.

by zb3 3y ago

Oh, it's not a new model, it's just that "safety" bullshit again.

Safety is just the latest trojan horse being used by big tech to try and control how people use their computers. I definitely belive in responsible use of AI, but I don't belive that any of these companies have my best interests at heart and that I should let them tell me what I can do with a computer. Those who trade liberty for security get neither and all that.

lightbendover 3y ago
Their sincerity does not matter when there is actual market demand.
UnFleshedOne 3y ago
I share all the reservations about this flavor of "safety", but I think you misunderstand who gets protected from what here. It is not safety for the end user, it is safety for the corporation providing AI services from being sued. Can't really blame them for that. Also, you can do what you want on your computer and they can do what they want on their servers.

dashundchen 3y ago

The safety here is not just "don't mention potentially controversial topics". The safety here can also be LLMs working within acceptable bounds for the usecase. Let's say you had a healthcare LLM that can help a patient navigate a healthcare facility, provide patient education, and help patients perform routine administrative tasks at a hospital. You wouldn't want the patient to start asking the bot for prescription advice and the bot to come back with recommending dosages change, or recommend a OTC drug with adverse reactions to their existing prescriptions, without a provider reviewing that. We know that currently many LLMs can be prompted to return nonsense very authoritatively, or can return back what the user wants it to say. There's many settings where that is an actual safety issue.

michaelt 3y ago
In this instance, we know what they've aimed for [1] - "Violence & Hate", "Sexual Content", "Guns & Illegal Weapons", "Regulated or Controlled Substances", "Suicide & Self Harm" and "Criminal Planning" So "bad prescription advice" isn't yet supported. I suppose you could copy their design and retrain for your use case, though. [1] https://huggingface.co/meta-llama/LlamaGuard-7b#the-llama-guard-safety-taxonomy--risk-guidelines https://huggingface.co/meta-llama/LlamaGuard-7b#the-llama-gu...

dragonwriter 3y ago

Actually, leaving out whether “safety” is inherently “bullshit” [0], it is both, Llama Guard is a model, serving a similar function to the OpenAI moderation API, but in a weights-available model. [0] “AI safety”, is often, and the movement that popularized the term is entirely, bullshit and largely a distraction from real and present social harms from AI. OTOH, relatively open tools that provide information to people building and deploying LLMs to understand their capacities in sensitive areas and the actual input and output are exactly the kind of things people who want to see less centralized black-box heavily censored models and more open-ish and uncensored models as the focus of development should like, because those are the things that make it possible for institutions to deploy such models in real world, significant applications.

leblancfg 3y ago

Well it is a new model, it's just a safety bullshit model (your words). But the datasets could be useful in their own right. I would consider using the codesec one as extra training data for a code-specific LLM – if you're generating code, might as well think about potential security implications.