Another way to phrase this is that, in the realm of theorem proving, a human is only going to 'validate' a response from an AI so long as there exists some math

Another way to phrase this is that, in the realm of theorem proving, a human is only going to 'validate' a response from an AI so long as there exists some mathematical or proof theoretic foundation to validate the response from an AI. To do otherwise is insanity, unless the AI can somehow provably demonstrate that they have invented a better system than all existing mathematics, computation, and logic. AI has plenty of room to grow in the realm of arts and creativity, because those are places where having a base foundation does not need to be so mechanically reinforced. Toppling all existing knowledge in the arts is fine to do, being that disruptive in the arts is perfectly acceptable and often welcome. When it comes to the safety of code - expectations of what a machine is going to do, for really important stuff, you want to have a solid, clear, strong reasoning system that is easily understood and easily conveyed to those looking to embrace it. Lots of math and abstraction exists either in the mind of humans, or on a computer, but to take one's hand and smack the whole house of cards down simply because it can't be proven to be absolute perfection - that's a ridiculous thing to do. We know what computers can do because we decide what computers can do. We need to retain some 'ground' in reasoning because what would happen if we collectively stepped away from it all for too long? 42.