5 ms·

I love the article, but I don't agree with the premise that machine learning equals neural nets. In my understanding machine learning is a very broad term that

• lmilcin 6y ago
  Well... I guess most people equal ML with AI and use these terms interchangeably. If you just replace ML with AI everywhere in this article it is going to make sense. The article has other problems, one being the main premise. The problem isn't to drive a car around track (which is what the polynomials did) but rather write a program that can figure out how to drive a car without you knowing how to solve it.
  • maweki 6y ago
    That's not symbolic AI though. That's only statistical methods. The statistical methods are all the rage now, but explainable AI that can reason is an important area of computer science (and research) and uses formal methods. Edit: yeah, you can downvote this, but current AI research splits right along this line, whether it's symbolic or statistical. Some AI courses will use NNs, others will use Prolog and ASP. You can't just dismiss a whole field of research by reducing AI to statistical methods.
    • laichzeit0 6y ago
      When I see "symbolic AI" I immediately think of Gary Marcus and immediately feel disdain towards the topic because of his behaviour on Twitter and other places.
      • maweki 6y ago
        I don't know the dude. I "only" know that my field of research is deductive reasoning in interactive applications and that this area falls under "Logic Programming" and LP is an area of AI. I know that AI researchers are usually a bit dismissive about the other area. I don't like statistics either. Reducing the whole of AI research to statistical approaches (and NNs are one of those) is disingenious and dismisses hundreds of researchers doing important work. You may not want to have rule-based image recognition, but if your car decides to run over somebody, I feel we better have an explanation for this behaviour based on reasoning and logic.
        • laichzeit0 6y ago
          I don’t think anyone is dismissing symbolic AI. As far as I can see, it’s just not beating current SOTA results of NNs? It’s not really about ideology, it’s about what currently has superior performance. Model interpretability is not always a requirement.
    • unishark 6y ago
      "Expert systems" were the hot research area in AI prior to machine learning (data driven methods, basically). Old methods and problems from that era like automated reasoning still have some research and applications going on, but aren't remotely as big an area as machine learning.
  • karolkozub 6y ago
    In my understanding AI is an even broader term and means "any solution that imitates intelligent behavior". E.g. expert systems which are pretty much a bunch of if-then rules are also considered AI.
    • segfaultbuserr 6y ago
      It's my understanding as well, many things that a modern programmer thinks in term of "computation" were once considered to be "AI". Lisp and Prolog were "AI", even the A* algorithm is still considered a rudimentary form of "AI" in textbooks just because it uses heuristics. There's a joke that says "every time AI researchers figure out a piece of it, it stops being AI" [0]. It's why I use "AI" and "ML" interchangeably although I know it's technically incorrect - the formal definition doesn't match what people are currently thinking. [0] https://en.wikipedia.org/wiki/AI_effect https://en.wikipedia.org/wiki/AI_effect
    • Delk 6y ago
      There have traditionally been different approaches and definitions for AI. Some emphasize behaviour while others emphasize the logic behind the behaviour. (In some sense, while expert systems of course were an attempt at getting practical results, they might also have been an attempt to implement what was seen as human reasoning, while e.g. black box machine learning could be more about just getting the behaviour we want.) Some approaches view agents as intelligent if their action resembles humans or other beings that we consider intelligent, while other approaches are merely interested in whether they perform well at a specified task, perhaps more so than humans. So yes, "any solution that imitates intelligent behaviour" is probably right, but with nuances with regard to what that actually means.
  • dmos62 6y ago
    Well that depends on your definition of AI. Which isn't well defined. We call AI what we perceive as "magic". Black box algorithms have a higher chance of being perceived that way (e.g. neural nets). When you get some insight into how an algorithm works (easier for transparent box algos, but same holds for black box algorithms), you start to see it less and less as "magic", and, consequently, you're less likely to refer to it as an (artificial) intelligence. Because ultimately, that's what we mean by intelligence -- magic. When we say that something is intelligent, we liken it to ourselves: it evokes a sense of identification. It all comes back to a sense of humans being fundamentally separate from "the other" (computers in this case). If we saw the mathematical models and algorithms as just that, we wouldn't call them AI. Also, if we didn't think of our intelligence as more than the behaviour of our biological computer, we wouldn't be enchanted by the concept of non-biological systems mimicking some of our behaviour.
    • mjburgess 6y ago
      I disagree. We don't find these systems intelligent because, on inspection, they arent. We are intelligent. Not "magically", but actually nevertheless. Our intelligence, and that of dogs (, mice, etc.) consists in the ability to operate on partial models of environments; dynamically responsive to them; and to skilfully respond to changes in them. This sort of intelligence requires the environment to physically reconstitue the animal in order to non-cognitively develop skills. It is skillful action we are interested in; and precisely what I missing in naive rule-based models of congition.
      • dmos62 6y ago
        You provided an illustration of "magic". It's important to realise that you don't need a complex algorithm to produce complex behaviour (see Stephen Wolfram and his work on cellular automata).
    • Uberphallus 6y ago
      A professor once told in class "when it works and you don't understand why, it's called AI; when you do, it's called algorithm"
• maweki 6y ago
  The machine learning course at my university starts out with polynomial regression and estimators, statistics of classification, etc.. Neural networks are only one tool in a large toolbox. But they are all the rage and it is no surprise that a lot of people want to play with them. Cynically, neural networks are easier as you don't really have to think about your model. Give some examples with some classes and you're done. Or give examples of one class and let the neural net generate new ones. Doing away with the abstraction beforehand is an enticing prospect.
  • mjburgess 6y ago
    That's an excellent approach -- and how I try to introduce people to NNs. NNs are just polynomial regression with polynomial activations; and piece-wise linear regression with relu activations (etc.). A NN is just a highly parameterized regression model -- for better, or worse.
    • JackFr 6y ago
      That was an eye-opener for me. I had always thought of neural nets in terms of the massive connected graph, that in my head was somehow behaved like a machine. When I realized in the end its just a representation of a massive function, f:Rm->Rn, which needs to fitted to match inputs and outputs. I know this is not precisely correct and glosses over many, many details - but this change in viewpoint is what finally allowed me to increase the depth of my understanding.
      • deleted 6y ago
        [deleted]
      • mjburgess 6y ago
        It's unclear that there is such a thing as an NN, and in any case, that it is graph-like. What are the nodes and edges? There is a computational graph which corresponds to any mathematical function -- but it is not the NN diagram -- and not very interesting (eg., addition would be a node). NNs are neither neural nor networks.
  • 6y ago
• midjji 6y ago
  I agree, machine learning can certainly be over transparent models and classic models can certainly be non transparent. I tend to think of machine learning as any method which optimizes not only the model parameters, but also the model structure in a single step. Though then again the latter are just parameters of a more abstract model. So its all always optimization in the end.
• thesz 6y ago
  > Also one could argue that the polynomial model is just a perceptron One also can argue otherwise [1]. [1] https://matloff.wordpress.com/2018/06/20/neural-networks-are-essentially-polynomial-regression/ https://matloff.wordpress.com/2018/06/20/neural-networks-are...
• dr_dshiv 6y ago
  As soon as we recognize plain old regression as machine learning, then we start to see "averages" as models of systems and how practically useful could that be?
  • thegginthesky 6y ago
    Well, actually working with "averages" as baselines before you start experimenting with more complex ML models is a good habit. Sure, they are dummy regressors [1], but they can be so useful for proving that your whatever ML model you choose is at least better than a dummy baseline. If your model can't beat it, then you need to develop a better one. They can even be used as a place-holder model so you can develop your whole architecture surrounding it, while another teammate is iterating over more complex experiments. You could also settle in for a moving average process as a first model in a time-series [2], because they are easy to implement and simple to reason about. Never under-estimate the power of an "average". [1] https://scikit-learn.org/stable/modules/generated/sklearn.dummy.DummyRegressor.html https://scikit-learn.org/stable/modules/generated/sklearn.du... [2] https://en.wikipedia.org/wiki/Moving-average_model https://en.wikipedia.org/wiki/Moving-average_model
  • fractionalhare 6y ago
    I think you're being facetious, but on the off-chance you're not, and for the benefit of others: averages are incredibly practically useful for modeling systems. Parameter estimation (which generalizes averages and applies to other distribution features like variance) is a foundational modeling methodology. It's useful for both understanding and forecasting data. Measures of central tendency are nearly always good (if obviously imperfect) models of systems. Here is a trivial example: one of the best ways of modeling timeseries data, both in and out of sample, is to naively take the moving average. This is a rolling mean parameter estimate on n lagged values from the current timestep. Not only is this an excellent way of understanding the data (by decomposing it into seasonality, trend and residuals), it's a competitive benchmark for future values. The first step in timeseries analysis shouldn't be to reach for a neural network or even ARIMA. It should be to naively forecast forward using the mean. You might be surprised at how difficult it is to beat that benchmark with cross-validation and no overfitting or look-ahead bias.
• meatmanek 6y ago
  The author may have implemented ML when they optimized their polynomial constants: > If I was developing a racing game using this as the AI, I’d not just pick constants that successfully complete the track, but the ones that do it quickly. If they wrote code that automatically picked constants that successfully completed the track quickly, (even something as simple as sorting the results by completion time), then that's reinforcement learning.
• pmelendez 6y ago
  I came here to state the same. I am not sure when we changed the terms, but back in the day, this would happily fall into machine learning. As he mentioned, if you want a good driver you would execute thousands of experiments to pick a good set of parameters