3 ms·
This is fundamentally the data quality issue. We don't want AIs to do what the average human would do, we want it to do what the best of us would do, or even
by Lerc 2y ago
This is fundamentally the data quality issue. We don't want AIs to do what the average human would do, we want it to do what the best of us would do, or even better.
I think the indiscriminate data gathering was sufficient when the gap between existing AIs and the average human was large enough that it was not proportionally very different to the gap between existing AIs and the best humans.
High quality data, either by removing poor examples or synthetically generating high quality exemplars should help a great deal.
The tricky bit with the instance of scams is that it has to walk the line between generalization of appropriate human behavior examples in a good faith situation and a bad faith situation.
How do you safely train an AI to be suspicious?
There is clear training bias towards obsequiousness in models at present, which is difficult to overcome with simple "Which response is better?" feedback because people will favour the response that agrees.
Getting a model to fairly challenge your assumptions (and indeed it's own) I predict will be a major step in usability.
- Terr_ 2y ago> This is fundamentally the data quality issue That contributes to this outcome, but I think the fundamental issue is deeper in how the algorithm is a kind of autocomplete being applied to an incremental play-script, with no inherent concept of itself being one of the actors or of truth versus contradiction. That's why you can instruct an LLM to disregard all prior instructions, or tell it to tell itself to imagine acting like it was disregarding prior instructions, etc... And if that's possible, why bother with a regular scam? Just command it to do what you want. > We don't want AIs to do what the average human would do [...] Getting a model to fairly challenge your assumptions To recycle a comment from a week ago regarding an LLM character backstory generator: > on reflection I think a human touch is going to be required when you want backstory which breaks or subverts common tropes and stereotypes. This makes sense if you view LLMs as primarily following connections that are already popular within their training data, and popular things are things readers are likely to anticipate. While I'm sure you could get an LLM to output challenges to common assumptions... The the challenges themselves may suffer from the same kind of bias. "LLM, when the hero meets the villain, add a twist that challenges expectations." "Okay: The villain reveals that he is actually the protagonist's father!"