4 ms·

This debate is meaningless for several reasons: 1. The original argument is a strawman. What do they mean by "data"? Is it survey results, microarrays, "Natura

This debate is meaningless for several reasons: 1. The original argument is a strawman. What do they mean by "data"? Is it survey results, microarrays, "Natural" images, "Natural" language text or readings from an audio sensor? No ML researcher would argue that applying complex models such as CNNs is useful for say survey data. But if the data is domain specific, such as Natural Language text, images taken in particular context, etc. using a model and parameters that exhibit good performance is a good starting point. 2. Unlike how statisticians view data (as say a matrix of measurements or "Data Frame"), machine learning researchers view data at a higher level of representation. E.g. An image is not merely a matrix but rather an object that can be augmented by horizontally flipping, changing contrast etc. In case of text you can render characters using different fonts, colors etc. 3. Finally the example used in the initial blog post, of predicting 1 vs 0 from images is itself incorrect. Sure a statistician would "train" a linear model to predict 1 vs 0, however I as an ML researcher would NOT train any model at all and would just use [1] which has state of the art performance in character recognition in widely varying conditions. When you have only 80 images, why risk assuming that they are sampled in an IID manner from population, instead why not simply use a model thats trained on far larger population. Now the final argument might look suspicious but its crucial in understanding the difference between AI/ML/CV vs Statistics. In AI/ML/CV the assumption is that there are higher level problems (Character recognition, Object recognition, Scene understanding, Audio recognition) which when solved enable us to apply them in wide variety of situations where they appear. Thus when you encounter a problem like digit recognition the answer an ML researcher would give is to use a state of the art model. [1] https://github.com/bgshih/crnn https://github.com/bgshih/crnn

Sean1708 9y ago
Are you using some accepted definition of Statistics that I'm unaware of? Because I always thought Machine Learning was a branch of Statistics.
- aub3bhat 9y ago
  >> Machine Learning was a branch of Statistics Thats incorrect. """ Machine learning is the subfield of computer science that, according to Arthur Samuel in 1959, gives "computers the ability to learn without being explicitly programmed." """ -- Wikipedia
  - Sean1708 9y ago
    That's fair enough, although I'm still not really sure why statisticians have to think about data in such a different way to machine learning researchers. I feel that if a statistician didn't look at the bigger picture of what their data is actually about and whether there are existing techniques to tackle that problem, they'd make a pretty terrible statistician.
    - autokad 9y ago
      I dont know if the commentator's assertions are correct, but my anecdotal experience follows: A statistician asks me if adding a feature could make a model worse (R^2), and I am like of course! He gets testy, and snidely chimes back 'i don't know why you think that could happen'. And I think 'I don't know why you don't think you can over fit data...' then it hit me, statisticians thinking revolves around running the model on the entire data set, where my thinking revolves around how it performs on test data
      - aub3bhat 9y ago
        There is a good paper by Leo Breiman [1] which discusses the exact issue that you mentioned. E.g. Statisticians believe that there is a "True" model that generates the data, and the errors observed are merely noise. ML on the other hand does not assumes existence of a "True" model, the assumption is that only the Data is source of the truth and any model that predicts the data with lowest error is preferable subject to performance on test/cross-validation etc. This is a powerful approach and distinguishes ML as separate field from statistics. My favorite example is predicting prices of real estate. A statistician will build a multi-level model that takes into account various effects (Zip code level, city level, school district level, year/month of acquisition etc.) and then build a regression model. The errors then would then be simply noise, an ML approach would be simply using weighted K-Nearest Neighbors, with geographic location as part of distance metric. Sure there are no effects to adjust but the K-NN regression model can account for difficult to capture quirks of geography by being able to represent local non-linear decision surface. http://projecteuclid.org/euclid.ss/1009213726 http://projecteuclid.org/euclid.ss/1009213726
- computerex 9y ago
  Machine learning is absolutely a branch of statistics.
  - aub3bhat 9y ago
    If you were to simply look at the overwhelming evidence (rather than resorting to use of vague words "absolutely" etc.) e.g. department affiliations of ML researchers (presenting at ICML/NIPS) and practitioners (at Google/FB/MSFT/etc) you would find that ML is firmly a branch of CS.
    - computerex 9y ago
      Why are you trying to deduce the nature of ML by looking at org charts? You do realize ML predates all the titles you listed above? Org charts are nuanced things often politically driven. You are looking at the wrong things of you want to truly learn instead of appeasing your personal biases. Computer science largely deals with algorithmic space and time complexity. Look up the Wikipedia page on logistic regression and tell me if you see computer science there. It is true that software giants are using and making huge breakthroughs in ML, specifically deep learning. That's largely because these companies invest heavily in bringing in the top ML researchers to their teams. Do you know the cost of deep mind? Individual engineers on the team have had multimillion dollar golden handcuff contracts from what I heard. Yes, software engineers can utilize the developed theory of ML and apply it to applications. For example, look at the new Google image search. You have a lot of interesting filter options now that require machine learning. Google speech recognition for instance has gotten so much better because Google moved off GMM's to dnn for the decoding. So just because Google and FB have interests in ML, doesn't mean that ML is a CS discipline. That's such faulty logic.

Sean1708 9y ago

Are you using some accepted definition of Statistics that I'm unaware of? Because I always thought Machine Learning was a branch of Statistics.

aub3bhat 9y ago
>> Machine Learning was a branch of Statistics Thats incorrect. """ Machine learning is the subfield of computer science that, according to Arthur Samuel in 1959, gives "computers the ability to learn without being explicitly programmed." """ -- Wikipedia
- Sean1708 9y ago
  That's fair enough, although I'm still not really sure why statisticians have to think about data in such a different way to machine learning researchers. I feel that if a statistician didn't look at the bigger picture of what their data is actually about and whether there are existing techniques to tackle that problem, they'd make a pretty terrible statistician.
  - autokad 9y ago
    I dont know if the commentator's assertions are correct, but my anecdotal experience follows: A statistician asks me if adding a feature could make a model worse (R^2), and I am like of course! He gets testy, and snidely chimes back 'i don't know why you think that could happen'. And I think 'I don't know why you don't think you can over fit data...' then it hit me, statisticians thinking revolves around running the model on the entire data set, where my thinking revolves around how it performs on test data
    - aub3bhat 9y ago
      There is a good paper by Leo Breiman [1] which discusses the exact issue that you mentioned. E.g. Statisticians believe that there is a "True" model that generates the data, and the errors observed are merely noise. ML on the other hand does not assumes existence of a "True" model, the assumption is that only the Data is source of the truth and any model that predicts the data with lowest error is preferable subject to performance on test/cross-validation etc. This is a powerful approach and distinguishes ML as separate field from statistics. My favorite example is predicting prices of real estate. A statistician will build a multi-level model that takes into account various effects (Zip code level, city level, school district level, year/month of acquisition etc.) and then build a regression model. The errors then would then be simply noise, an ML approach would be simply using weighted K-Nearest Neighbors, with geographic location as part of distance metric. Sure there are no effects to adjust but the K-NN regression model can account for difficult to capture quirks of geography by being able to represent local non-linear decision surface. http://projecteuclid.org/euclid.ss/1009213726 http://projecteuclid.org/euclid.ss/1009213726
computerex 9y ago
Machine learning is absolutely a branch of statistics.
- aub3bhat 9y ago
  If you were to simply look at the overwhelming evidence (rather than resorting to use of vague words "absolutely" etc.) e.g. department affiliations of ML researchers (presenting at ICML/NIPS) and practitioners (at Google/FB/MSFT/etc) you would find that ML is firmly a branch of CS.
  - computerex 9y ago
    Why are you trying to deduce the nature of ML by looking at org charts? You do realize ML predates all the titles you listed above? Org charts are nuanced things often politically driven. You are looking at the wrong things of you want to truly learn instead of appeasing your personal biases. Computer science largely deals with algorithmic space and time complexity. Look up the Wikipedia page on logistic regression and tell me if you see computer science there. It is true that software giants are using and making huge breakthroughs in ML, specifically deep learning. That's largely because these companies invest heavily in bringing in the top ML researchers to their teams. Do you know the cost of deep mind? Individual engineers on the team have had multimillion dollar golden handcuff contracts from what I heard. Yes, software engineers can utilize the developed theory of ML and apply it to applications. For example, look at the new Google image search. You have a lot of interesting filter options now that require machine learning. Google speech recognition for instance has gotten so much better because Google moved off GMM's to dnn for the decoding. So just because Google and FB have interests in ML, doesn't mean that ML is a CS discipline. That's such faulty logic.