How do you know whether the test data is distributed similarly to the training data without an underlying model of how the system works? Put differently, how do

How do you know whether the test data is distributed similarly to the training data without an underlying model of how the system works? Put differently, how do you know what matters for your tests? If you have a machine-learned model, you've likely trained it on sparse data. How do you know that your sparse data covers the set of signals required to make accurate predictions, and how do you know that your model actually uses those signals? Edit: I would say that to have a notion of confidence requires that you have an underlying model of how the system works (IE, explainability).