The distinction that the OP is looking for is generative vs. discriminative inference. Generative inference: We know the process that generates our observed da

The distinction that the OP is looking for is generative vs. discriminative inference. Generative inference: We know the process that generates our observed data and can model it to sufficient precision, s.t. we can establish a forward synthesis: Parameters in, synthesised data out. With that we can compare a synthesized dataset from a configuration of parameters with real observed data and, using Bayes (a.k.a. MAP estimation), infer the correct configuration of parameters that most likely generated our observed data, which is the answer we're looking for. Examples: Kalman Filters, Linear Regression. Discriminative inference: We do not know the process that generates observed data or it's too difficult to model. In that case we ask machines to take a shot at modelling it. To do that we set up parametric data transformation pipelines (e.g. Neural Nets) and feed it lots of correct input/output pairs. Over time the model will learn how to transform the input (observed data) to the output (answer we're looking for) and hopefully generalise well when we feed it new input data that it hasn't seen before. Examples: NNs for classification. Of course there are interesting mixtures and hybrids between the two and the lines are blurred in some cases but this is the general distinction.