So i'd put parametric vs. non-para on a scale. It can be done in terms of the final model parameters, even -- but this may seem initially weird. If the paramet

So i'd put parametric vs. non-para on a scale. It can be done in terms of the final model parameters, even -- but this may seem initially weird. If the parameters of the predictive model are a weakly compressive function of the dataset, then your method is non-parametric. If your parameters are extremely compressive it's parametric. Subject to both being low-loss models of the data. Why? Well a non-parametric method is basically one which "uses the data points as its statistical model"; and a parametric method fits the data to a prior low-parameterised model. Eg., linear regression essentially fits the data to a normal distribution = parametric on the mean/std, ie., pdf(Y|X) = N(ax +b, stdev). You fit (a,b) and thus essentially are just "finding a mean". Eg., knn just remembers the dataset itself. So there's a scale from "linear regression to knn", ie., Weights = (Mean, Stdev)... to Weights = (X, Y). The terms parametric and non-parametric are fairly overloaded, so this way of characterising the distinction may either improve or worsen that. Either way, my point is that NN model with a very high parameter count is essentially best analysed as KNN on a weakly compressed feature space. In that sense it is an incredibly obvious and simple algorithm. Incidentally, NN can just be linear regression or KNN if you set it up appropriately. So NN is an alg. which runs "from knn to linear regression" depending on, eg., activation-fns, how hard you regularise it, etc.