One of the major structural problems with using neural networks in production is that all failures are silent failures. You could create an ensemble of models t

One of the major structural problems with using neural networks in production is that all failures are silent failures. You could create an ensemble of models that base their conclusions on reliably different features and report on disagreements with the ensemble decision, but that doesn't actually tell you which models were in error, just that some models were in error. Also, reliably determining that models are using different features is difficult.