Something that is often left out when talking about interpretability is the relationships between the predictors and the dependent variable in simulations. For

Something that is often left out when talking about interpretability is the relationships between the predictors and the dependent variable in simulations. For example: - I fit a complex, difficult to interpret model to a dataset, attempting for forecast my sales (structure of the dataset largely irrelevant for this example) - I take an entry from the training set and decrease the value of some price attribute by 15%, leaving everything else unchanged - I try to predict the sales for the entry I just created using the trained model - What happens if the model now predicts lower sales? There is a clear relationship between price and sales volume going in the opposite direction. Would lowering my prices by 15% really lead to a decrease in my sales? How do you track what's happening in the model to create this forecast? Did I use the wrong model? Was my training data incorrect? How do you explain this to a client or to a product user?