6 ms·

Agreed - this just seems to be an extension of the calculator story listed above, except using machine learning. Why wouldn't you use state of the art technolog

by borkt 8y ago

Agreed - this just seems to be an extension of the calculator story listed above, except using machine learning. Why wouldn't you use state of the art technology (including those labeled as AI) to help push any field forward?

sgt101 8y ago
looking at the paper in PNAS I think that there is a clever arrangement and application of deep nets in play here. As ever the trick is to couple deep domain understanding to clever application of AI techniques.
- wenc 8y ago
  From what I gathered from the paper -- again I'm not an expert on what they're doing -- could the NN model have been replaced by any number of more parsimonious nonlinear regressors? (Not that an NN isn't a valid choice -- it is often used as a nonlinear regression model, but it is one of many valid choices in this scenario.) Which prompts the next question: does anything that uses NNs warrant the term AI? The use of NN here seems to be that of a model surrogate (i.e. model reduction). There's no perception problem, logic problem, decision problem, or anything that we commonly associate with "intelligence".
  - borkt 8y ago
    My undergrad is in Materials Science and I am very uninformed on the NN/AI stuff, but I completely agree that the term AI seems to be applied to all sorts of techniques with zero relation to true intelligence, seems more to be just working with patterns that have been fed into them.
  - sgt101 8y ago
    Well. Maybe it could be neater (in an information theoretic sense) but DNNs are a bit fuzzy on that, and if you have modern compute, who cares? I think that there's many questions about the validation of machine learning models which mean that your question "could the NN model have been replaced by any number of more parsimonious nonlinear regressors?" is open, in the sense - we don't have a sharp way of deciding which is best, because one out of 100 million isn't informative for statistical reasoning. The other question "does anything that uses NNs warrant the term AI?" is very, very difficult. Because as Marvin Minskey said "intelligence is a portmanteau term" by which he meant overloaded. It's full of meaning and non specific, so I like to say that AI is about technology and capability and not an association with human or animal cognition - which is the domain of Artificial Intelligence proper.
    - wenc 8y ago
      To the first point, I believe it's a spectrum. To a practitioner, "better" can be defined along well-known dimensions. Suppose you know your data lies more or less on a straight line -- you could fit a NN model or a run a linear regression. In this scenario, linear regression would be the "better" choice for some widely-accepted measures of "better" (interpretability, computational efficiency, parsimony, regularizability, etc.). One almost never chooses NN just for the sake of choosing NN... there are well-understood trade-offs. For instance, it is widely known that NN's generally require a larger amount of data than classical/statistical algorithms for weights to converge -- mostly because it's fitting a more general function than most classical ML algorithms. (The reason NNs have begun to show the results they have (vs in the 1990s) isn't just because we have more compute than before or that the theory has advanced significantly; it's also because we have more a lot more data to fit the more general function with.) To the second point, I can see the point, but I wonder if the semantic meaning of the term is eroded by being overly encompassing. We could say a calculator implements AI.
      - sgt101 8y ago
        Agree about data. Do you really think theory hasadvanced?
        wenc 8y ago
        In my opinion fundamental NN theory hasn’t really advanced significantly since the 1990s. I guess it was phrased ambiguously in my comment But there has been a great deal of new techniques that make NN work better in practice like dropout, ReLU for vanishing gradients, CNNs and GANs for specific problem types, transfer learning, etc. The recent work with Neural ODEs show that the field is advancing in terms of ideas. This is a good trajectory IMO. Many engineering fields work this way — find out new ways of doing things that work and then take a step back to see if there’s anything fundamental that links everything together. Practice precedes theory.
  - rsfern 8y ago
    The way they’ve formulated the problem, yes, I agree with you. There’s nothing really special about the deep learning model they used, and in fact they also tried gradient boosted decision trees.

by borkt 8y ago

sgt101 8y ago

looking at the paper in PNAS I think that there is a clever arrangement and application of deep nets in play here. As ever the trick is to couple deep domain understanding to clever application of AI techniques.

wenc 8y ago
From what I gathered from the paper -- again I'm not an expert on what they're doing -- could the NN model have been replaced by any number of more parsimonious nonlinear regressors? (Not that an NN isn't a valid choice -- it is often used as a nonlinear regression model, but it is one of many valid choices in this scenario.) Which prompts the next question: does anything that uses NNs warrant the term AI? The use of NN here seems to be that of a model surrogate (i.e. model reduction). There's no perception problem, logic problem, decision problem, or anything that we commonly associate with "intelligence".
- borkt 8y ago
  My undergrad is in Materials Science and I am very uninformed on the NN/AI stuff, but I completely agree that the term AI seems to be applied to all sorts of techniques with zero relation to true intelligence, seems more to be just working with patterns that have been fed into them.
- sgt101 8y ago
  Well. Maybe it could be neater (in an information theoretic sense) but DNNs are a bit fuzzy on that, and if you have modern compute, who cares? I think that there's many questions about the validation of machine learning models which mean that your question "could the NN model have been replaced by any number of more parsimonious nonlinear regressors?" is open, in the sense - we don't have a sharp way of deciding which is best, because one out of 100 million isn't informative for statistical reasoning. The other question "does anything that uses NNs warrant the term AI?" is very, very difficult. Because as Marvin Minskey said "intelligence is a portmanteau term" by which he meant overloaded. It's full of meaning and non specific, so I like to say that AI is about technology and capability and not an association with human or animal cognition - which is the domain of Artificial Intelligence proper.
  - wenc 8y ago
    To the first point, I believe it's a spectrum. To a practitioner, "better" can be defined along well-known dimensions. Suppose you know your data lies more or less on a straight line -- you could fit a NN model or a run a linear regression. In this scenario, linear regression would be the "better" choice for some widely-accepted measures of "better" (interpretability, computational efficiency, parsimony, regularizability, etc.). One almost never chooses NN just for the sake of choosing NN... there are well-understood trade-offs. For instance, it is widely known that NN's generally require a larger amount of data than classical/statistical algorithms for weights to converge -- mostly because it's fitting a more general function than most classical ML algorithms. (The reason NNs have begun to show the results they have (vs in the 1990s) isn't just because we have more compute than before or that the theory has advanced significantly; it's also because we have more a lot more data to fit the more general function with.) To the second point, I can see the point, but I wonder if the semantic meaning of the term is eroded by being overly encompassing. We could say a calculator implements AI.
    - sgt101 8y ago
      Agree about data. Do you really think theory hasadvanced?
      - wenc 8y ago
        In my opinion fundamental NN theory hasn’t really advanced significantly since the 1990s. I guess it was phrased ambiguously in my comment But there has been a great deal of new techniques that make NN work better in practice like dropout, ReLU for vanishing gradients, CNNs and GANs for specific problem types, transfer learning, etc. The recent work with Neural ODEs show that the field is advancing in terms of ideas. This is a good trajectory IMO. Many engineering fields work this way — find out new ways of doing things that work and then take a step back to see if there’s anything fundamental that links everything together. Practice precedes theory.
- rsfern 8y ago
  The way they’ve formulated the problem, yes, I agree with you. There’s nothing really special about the deep learning model they used, and in fact they also tried gradient boosted decision trees.