In statistics, there's a joint distribution of x and f(x) to be learned. In this piece, there's a distribution over trained models that naturally depends on the

In statistics, there's a joint distribution of x and f(x) to be learned. In this piece, there's a distribution over trained models that naturally depends on the initialized parameters. It's interesting when data is considered non-random, and the probabilistic variation arises from weight initialization. While this serves to characterize the training process, the authors see the goal of the training procedure to converge to a target function f(x), with known values for x. This is of course not the substantive goal of the DL/ML researcher or practitioner (where the target function is unknown, and the goal is to learn the joint distribution and hopefully not fit the latent f(x) function exactly). On the one hand, uncovering how a NN would behave with different lengths and widths for a fixed set of data seems an important step, and although I haven't read far enough to see any interesting results yet, I am sure the authors have much in store. On the other hand, I feel like most questions of practical relevance - like generalization, model choice given data, causality, statistical properties of the entire design process and so on - all require us to think about the data generating process, rather than only the weight initialization process.