3 ms·
To the first point, I believe it's a spectrum. To a practitioner, "better" can be defined along well-known dimensions. Suppose you know your data lies more or
by wenc 8y ago
To the first point, I believe it's a spectrum.
To a practitioner, "better" can be defined along well-known dimensions. Suppose you know your data lies more or less on a straight line -- you could fit a NN model or a run a linear regression. In this scenario, linear regression would be the "better" choice for some widely-accepted measures of "better" (interpretability, computational efficiency, parsimony, regularizability, etc.).
One almost never chooses NN just for the sake of choosing NN... there are well-understood trade-offs.
For instance, it is widely known that NN's generally require a larger amount of data than classical/statistical algorithms for weights to converge -- mostly because it's fitting a more general function than most classical ML algorithms. (The reason NNs have begun to show the results they have (vs in the 1990s) isn't just because we have more compute than before or that the theory has advanced significantly; it's also because we have more a lot more data to fit the more general function with.)
To the second point, I can see the point, but I wonder if the semantic meaning of the term is eroded by being overly encompassing. We could say a calculator implements AI.
- sgt101 8y agoAgree about data. Do you really think theory hasadvanced?
- wenc 8y agoIn my opinion fundamental NN theory hasn’t really advanced significantly since the 1990s. I guess it was phrased ambiguously in my comment But there has been a great deal of new techniques that make NN work better in practice like dropout, ReLU for vanishing gradients, CNNs and GANs for specific problem types, transfer learning, etc. The recent work with Neural ODEs show that the field is advancing in terms of ideas. This is a good trajectory IMO. Many engineering fields work this way — find out new ways of doing things that work and then take a step back to see if there’s anything fundamental that links everything together. Practice precedes theory.