My Master thesis supervisor had a fun trick for this, that can be used not only on on NNs, but also on other classifiers: https://arxiv.org/pdf/1011.3177.pdf ht

My Master thesis supervisor had a fun trick for this, that can be used not only on on NNs, but also on other classifiers: https://arxiv.org/pdf/1011.3177.pdf https://arxiv.org/pdf/1011.3177.pdf The gist is, you train a single classifier that will work as two classifiers (with certain restrictions): one that says "false/not-false" and another that says "true/not-true". If the output of the classifier is "not-false" and "not-true", then you consider that as a "I don't know".