2 ms·
> Shouldn’t it be impossible for the AI to teach itself any more ethics than it started out with? That's not required for this process to help. It's importan
by IanCal 3y ago
> Shouldn’t it be impossible for the AI to teach itself any more ethics than it started out with?
That's not required for this process to help.
It's important to remember that you are comparing two different things.
The ability to create something, and the ability to detect something.
Even if your training does not improve your detection, you can improve the creation aspect up to the quality of your detector.
For humans a good comparison might be music. I am far worse at playing the guitar than I am at telling what music is "good". That means that (while lessons may be more time efficient) I can get better at the guitar without any feedback from anyone else. I will hit a limit somewhere if my tastes don't get more nuanced or I get better at spotting "good" music, but there's a lot I can improve with.
Cooking is another good example where peoples skills are lower than their detection/ranking abilities.
Perhaps coding too, particularly early on you may have found some code nice and clean but given the task of writing it you'd not have done as good of a job.
So this is the same with LLMs. Their ability to classify something may be far beyond their ability to generate that thing first time, in which case training on their own output makes total sense.
There's absolutely no reason to go into what they "think" or "know" or "their motivation" and "goals". I'm a big fan of anthropomorphising them as I think it helps figure out new ways of using them, but to get this behaviour you don't need anything of the sort and I think it confuses the topic.
> What if you had overseer AIs performing Constitutional AI Feedback on trainee AIs, or otherwise tried to separate out the labor?
You should absolutely be trying things like this, having multiple differently prompted LLMs helps. I have done this with goal planners and critiquers, to push the planners to generate concrete tasks. It's similar to but a step up from telling them to "think through this step by step". It's also extremely easy to do.
- visarga 3y ago> Their ability to classify something may be far beyond their ability to generate that thing first time, in which case training on their own output makes total sense. It is possible to amplify a model - for example ensembling a bunch of predictions, using reversible tasks like translation to check if you get back to the original, applying code execution and math verification, using a search engine or using narrow models. Basically augmented LLMs are smarter than plain LLMs. Another way models can self-improve is to generate training data. The LLM could process the training set to clean it up, bring contextual information and check facts and in the end retrain on the improved text.
- IanCal 3y ago> It is possible to amplify a model - for example ensembling a bunch of predictions, using reversible tasks like translation to check if you get back to the original, applying code execution and math verification, using a search engine or using narrow models. Basically augmented LLMs are smarter than plain LLMs. Yes, these are great extensions. It's part of where I like thinking of these things as human as they are all good answers to "how would you do X?" I'd search for answers, I'd run code and see what the error message was, etc. I'd talk through problems, try and list things too. It's not a perfect analogy, but I think people do things like ask them to effectively whiteboard a solution then complain that some API doesn't exist - sure that's not the preferred outcome but it's not how we expect even good devs to write code. We explicitly complain about times we're forced to do that in interviews. I'm investigating giving them access to autocomplete while they're writing code, I need to learn how to write a client for a language server though.