We certainly cannot comprehend the entire thing at the same time. ”XAI” is a bit of a buzzword right now and every HCI researcher struggling with finding their

We certainly cannot comprehend the entire thing at the same time. ”XAI” is a bit of a buzzword right now and every HCI researcher struggling with finding their grip have jumped on the hype boat, meaning lots of them will just say things because it sounds nice to people with even less knowledge of NNs as them. However, to think we cannot comprehend them at all is too much. We can certainly ”query” a NN to understand the weights and activations that led to some decision in some specific scenario. How to consume and use that information is another matter, of course, but a debugging process is certainly feasible (as it is with any large scale computer system). Plus, the architectures of neural networks are not as complicated as most people think. Sure it gets weird after training, but the basic architecture of components are not nearly as complicated as the hardware of a GPU for example, or even that of a complex OS kernel.