Presumably there are humans flagging the inappropriate questions, and these are being fed into a training set for a smaller and more frequently retrained part o

Presumably there are humans flagging the inappropriate questions, and these are being fed into a training set for a smaller and more frequently retrained part of the model that provides a signal not to answer the question.