I think if you're going to judge the tech, it's important to understand what it is. Basically, in both cases, you are bringing in the context to those questions

I think if you're going to judge the tech, it's important to understand what it is. Basically, in both cases, you are bringing in the context to those questions. You know what those questions "mean" in the collective brain-space, and why they are sensitive. You did not neutrally pick those questions; you know those questions are already loaded. Unsurprisingly, when you bring those pre-loaded questions to an AI whose job it is to continue the text in the way that was the most likely way to see it continued in the training text, you get exactly that. Prompt the bare, unfiltered AI with antisemitic text, get antisemitic continuations. That's unfortunate, but in terms of what was promised and what was delivered, not surprising. You asked for it. You got it. What I've never seen ChatGPT or any other GPT technology do is suddenly veer off into an antisemitic rant, unprompted. (At least, any technology that I know was GPT based. I'm not sure about Tay. Training the chat bot on its own input turns out to be a bad idea.) You can ask it to write a STTNG script where Picard discovers he has a son he never knew, but you won't see it suddenly spin off into an anti-Jewish rant. You didn't bring it. You didn't ask for it. So you didn't get it. An AI that spontaneously dropped racism and antisemitism and all that sort of thing into a significant portion of the responses could be a problem. But an AI that simply gives you what you want is something different, in my opinion. Worries about what that means in a larger context are currently not an issue, because the transformer-based AI that the GPT series is are fundamentally passive. There is no chance they will be fed a bad prompt and then go all Earworm [1] on $RACE. We still have time to deal with this. My main objection is training the AI on generalized Internet text, and then trying to patch it after the fact, is just fundamentally ineffective, in a number of ways. For the goal of trying to prevent the AI from ever saying anything bad, it is an arms race and I suspect it will end up being advantage attacker in the end. For the goal of having the highest possible quality AI, trying to mask out its outputs after the fact is highly inimical to that. There's just no way this is a good solution. At the very least, a far better one would be for the training set to be more discriminating in the first place. If nothing else, it is just generally bad engineering to put something into a system that didn't have to go in, then try to filter it out again after the fact. OpenAI would not have to be filtering out the antisemitism if they hadn't put it in the training set in the first place, for instance. It may be easy to just slam the entire internet in, but that doesn't make it a good idea. [1]: https://www.youtube.com/watch?v=-JlxuQ7tPgQ https://www.youtube.com/watch?v=-JlxuQ7tPgQ