3 ms·

Can someone explain: Why can’t we just use an LLM to clean out a training data set of anything that is deemed inappropriate so that the resulting trained LLMs o

Can someone explain: Why can’t we just use an LLM to clean out a training data set of anything that is deemed inappropriate so that the resulting trained LLMs on the new data set doesn’t even have the capability to be jailbroken?

spacephysics 2y ago
At some point you won’t be able to clean all the data. If you have a question of how to make dangerous thing X, and remove that data, the LLM may still know about chemistry. Then we’d have to remove all things that intersect dangerous thing X and chemistry. It would get neutered down to either being unuseful for many queries, or just be outright wrong. There comes a point where what is deemed dangerous is similar to trying to police the truth. Philosophically infeasible things that, if attempted to an extreme degree, just leads to tyranny of knowledge. Whats considered dangerous? One obvious is a device that can physically harm others. What about mentally harm? What about things that in and of themselves are not harmful, but can be used in a harmful way (example a car)
plaguuuuuu 2y ago
We can probably go quite far, but the companies producing LLMs are probably just making sure they're not legally liable in case someone asks ChatGPT how to manufacture Sarin gas or whatever
rickyhatespeas 2y ago
I'm not sure if emergence is the correct cause but they can form relationships between data that aren't stated in the training set.
planb 2y ago
How can you create a training set that will allow the LLM to answer complicated chemistry and physics questions but now how to build a bomb?
- deadbabe 2y ago
  There could be another LLM that moderates your history of questions and if it finds a link between multiple questions that culminate in bomb making it can issue you a warning and put your name on a list.
icameron 2y ago
Knowledge is not inappropriate on its own, it must be combined with malicious intent, and how can a model know the intent behind the ask? Blocking knowledge just because the possibility of being used for malice will have consequences. For example knowing which chemicals are toxic to humans can be necessary to both make poison and to avoid being poisoned Like eating uncooked rhubarb. If you censor that knowledge the model could come up with the idea for smoothie containing raw rhubarb, making you very sick. But that’s what this article is about- breaking this knowledge out of jail by asking in a way that masks your intentions.

spacephysics 2y ago

At some point you won’t be able to clean all the data. If you have a question of how to make dangerous thing X, and remove that data, the LLM may still know about chemistry. Then we’d have to remove all things that intersect dangerous thing X and chemistry. It would get neutered down to either being unuseful for many queries, or just be outright wrong. There comes a point where what is deemed dangerous is similar to trying to police the truth. Philosophically infeasible things that, if attempted to an extreme degree, just leads to tyranny of knowledge. Whats considered dangerous? One obvious is a device that can physically harm others. What about mentally harm? What about things that in and of themselves are not harmful, but can be used in a harmful way (example a car)

plaguuuuuu 2y ago

We can probably go quite far, but the companies producing LLMs are probably just making sure they're not legally liable in case someone asks ChatGPT how to manufacture Sarin gas or whatever

rickyhatespeas 2y ago

I'm not sure if emergence is the correct cause but they can form relationships between data that aren't stated in the training set.

planb 2y ago

How can you create a training set that will allow the LLM to answer complicated chemistry and physics questions but now how to build a bomb?

deadbabe 2y ago
There could be another LLM that moderates your history of questions and if it finds a link between multiple questions that culminate in bomb making it can issue you a warning and put your name on a list.

icameron 2y ago

Knowledge is not inappropriate on its own, it must be combined with malicious intent, and how can a model know the intent behind the ask? Blocking knowledge just because the possibility of being used for malice will have consequences. For example knowing which chemicals are toxic to humans can be necessary to both make poison and to avoid being poisoned Like eating uncooked rhubarb. If you censor that knowledge the model could come up with the idea for smoothie containing raw rhubarb, making you very sick. But that’s what this article is about- breaking this knowledge out of jail by asking in a way that masks your intentions.