I agree that given an agreeable set of data, the training isn't hard. That's why I don't really see this as a technical problem. If humans can't even agree on

I agree that given an agreeable set of data, the training isn't hard. That's why I don't really see this as a technical problem. If humans can't even agree on what is right and wrong, moral or immoral, how can we come up with a corpus of data to train the bot on? Sure, Microsoft could control what they wanted the bot to say or not say, but would you really want a large corporation controlling the behavior of future AI systems that interact with people at large, and deciding how to filter information in cases like this? (Disclaimer: I work at MS.)