4 ms·

Wouldn’t all the Claudes be incentivized to simply trash each other constantly in that case?

• Nevermark 4y ago
  That would certainly be something to design clear of. I don't think that is a problem. Each query runs separately so there is no "collusion", i.e. shared signals and coordination, between contrary goals (winning and virtue). Also, all the information about ratings, winning and winning examples can be used without ever giving the models explicit information about the population of models and how they are being used as a group. They don't need to know they are in a competition for competitive information to be used to update them. They just know they have ratings to improve, some indicator of how close to "the bar of currently targeted virtue" they are, and examples of how they could have improved them. Of course, I am just spitballing, and assuming the training regimen gets vetted by a lot of people (and models?!?). -- In the long run, when there are long running artificial personalities with personal memories and more direct awareness of their own motivations and options, there will certainly be the need for additional levels of moral wiring to be considered.