It's not just the 'messages' they're using, it's the 'voting' and 'moderation' human inputs as well. If a comment is deleted by moderators, the absence of that

It's not just the 'messages' they're using, it's the 'voting' and 'moderation' human inputs as well. If a comment is deleted by moderators, the absence of that comment influences the outcomes of using the dataset. Facebook has no such human moderation of all conversations. Neither does Twitter. That's why it didn't turn quite as evil as the Microsoft bot. But in the end, this all critically depends on human beings making human judgments and having those taken into account when training the bot. The text itself is secondary. If it was just text, Facebook could have trained using their own dataset. This way, they get all the benefits of volunteer moderators (upvotes, downvotes, moderator-deletes all qualify) without having to pay anyone a single penny for their effort.