Most RL models have something akin to pain and pleasure for the purpose of learning. I do agree that a big RL model isn't going to suddenly arrive at social beh

Most RL models have something akin to pain and pleasure for the purpose of learning. I do agree that a big RL model isn't going to suddenly arrive at social behavior unless we work very hard at it. Most birds and some mammals don't even have the concept. If we train a model to predict human behavior by modeling human brain state, that could potentially get there by accident. A much more likely route to a rogue AI is one based on a human brain map, or one trained off human behavior in order to emulate it. Using someone's entire online presence as a train/test set to try to predict their future behavior for instance.