4 ms·

Is linear regression programming? A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves,

Is linear regression programming? A hammer, nail, saw and timber can also be used to solve problems but I wouldn't call those programming in and of themselves, but they could be used to built analog computers (where cogs, cams etc. Are like lines of code or procedures). Building a neutral network to get a result is not at all like programming. There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network.

qayxc 6y ago
> There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network. The same can be said about complex simulations. The difference lies in knowing and being able to determine the limits of the system and the verification process. In a simulation we can derive the accuracy of our model from parameters like numerical precision and -stability, coarseness and model used. In other words we know the function we want to model because we state it explicitly. Neural nets can model any function and the challenge is to extract the learned function from the trained network, to examine its limits and correctness. This is what I understood the author meant by the "operational" viewpoint. We can verify that a given network architecture combined with a given optimisation function will find a local minimum w.r.t a given set of training data. This can be verified and tested. What's not so easy to verify and test, however, are the properties of the modelled function as well as the function itself. That's why we still have to rely on proxies like error metrics on fixed datasets or failure cases. With a simulation on the other hand, we can easily control and predict the (quality of the-) outcome by manipulating well understood parameters (number of iterations, coarseness of the simulation, numerical precision, etc.). I picked simulations as an example, because many other classes of program can be verified using formal methods since the desired results are usually known beforehand. Again, just another reason why the author talks about a distinction in terms of operations, not the fundamental type of programming. I find this to be a very interesting and thought provoking idea.
mr_toad 6y ago
> Is linear regression programming? I’d argue not. Mostly. If you fit a model using lm() in R, and then apply that model it’s not the same as hand selecting the weights of a linear equation and coding that equation. You could in theory select the same weights, and code it by hand. But no one ever does.
- disgruntledphd2 6y ago
  Doesn't that imply that if one can fit an NN in a line of code, then it's not programming? That seems like a weird distinction to make.

qayxc 6y ago

> There is usually not a "perfect" structure, rather there are hardware, energy and time constraints to training and inference, balanced by over and under training the network. The same can be said about complex simulations. The difference lies in knowing and being able to determine the limits of the system and the verification process. In a simulation we can derive the accuracy of our model from parameters like numerical precision and -stability, coarseness and model used. In other words we know the function we want to model because we state it explicitly. Neural nets can model any function and the challenge is to extract the learned function from the trained network, to examine its limits and correctness. This is what I understood the author meant by the "operational" viewpoint. We can verify that a given network architecture combined with a given optimisation function will find a local minimum w.r.t a given set of training data. This can be verified and tested. What's not so easy to verify and test, however, are the properties of the modelled function as well as the function itself. That's why we still have to rely on proxies like error metrics on fixed datasets or failure cases. With a simulation on the other hand, we can easily control and predict the (quality of the-) outcome by manipulating well understood parameters (number of iterations, coarseness of the simulation, numerical precision, etc.). I picked simulations as an example, because many other classes of program can be verified using formal methods since the desired results are usually known beforehand. Again, just another reason why the author talks about a distinction in terms of operations, not the fundamental type of programming. I find this to be a very interesting and thought provoking idea.

mr_toad 6y ago

> Is linear regression programming? I’d argue not. Mostly. If you fit a model using lm() in R, and then apply that model it’s not the same as hand selecting the weights of a linear equation and coding that equation. You could in theory select the same weights, and code it by hand. But no one ever does.

disgruntledphd2 6y ago
Doesn't that imply that if one can fit an NN in a line of code, then it's not programming? That seems like a weird distinction to make.