The other good thing with requiring proofs is that you can show specify exactly where the proof fails and the one giving the proof can use this feedback to corr

The other good thing with requiring proofs is that you can show specify exactly where the proof fails and the one giving the proof can use this feedback to correct their beliefs. I would actually be pretty okay with a model that can do that. Take feedback and correct themselves without resorting to retraining. It would still be pretty far away from a general AI but much much better than a black box.