OK, so we're assuming supervised learning. In that case, how were the features of the training set chosen? And how was the training set labelled? Most of the t

OK, so we're assuming supervised learning. In that case, how were the features of the training set chosen? And how was the training set labelled? Most of the time, outside of semi-supervised learning, those things are not in the data, someone has to (painstakingly) decide them. That can very well be explained I believe. More generally, what I mean to say is that you can learn any number of models from the same data, so to a great extent you choose what your model learns by manipulating your algorithm's (hyper)parameters, the training set's features, choosing what goes into the training set itself etc etc. That process leaves room for a lot of regulatory oversight. - in principle. In practice I expect since we're talking about people lending money or selling insurance we're only ever going to get those details out of their cold dead hands, and only with considerable effort at that.