Apologies for grossly oversimplifying ML/AI. Basically we have inputs, a massive array of layers of interconnections which end up with weighting factors as a re

Apologies for grossly oversimplifying ML/AI. Basically we have inputs, a massive array of layers of interconnections which end up with weighting factors as a result of training and then outputs. Like the human brain, the "knowledge" is stored in those weightings. There billions if not trillions of such weightings. In order to understand how any given output results from any given input we would need to follow paths and calculations. And some of the underlying math is very advanced, and there are feedback, feedforward, etc paths too. Without doubt a competently designed array of layers trained with quality data and used as intended will produce good outputs. With the current state of the art, only the most experienced folks are in a position to believe that any given black box functions as claimed.