4 ms·

So is the identity transform...

srean 6y ago
I wish your comment is voted up. Where do we go next ? -- look addition is a neural network, look weighted average is a neural network, look linear regression, logistic regression, Poisson regression are neural networks ...
- Certhas 6y ago
  I have definitely seen regressions referred to as linear machine learning. Not even kidding.
  - nikk1 6y ago
    Regression is an approach to machine learning https://en.wikipedia.org/wiki/Machine_learning#Regression_analysis https://en.wikipedia.org/wiki/Machine_learning#Regression_an...
  - bonoboTP 6y ago
    So? Machine learning is not some buzzword fad. Regression is part of ML for sure. Some people seem to think ML means fancy deep learning GPT-3 transformer running on a TPU farm. Actually ML is a discipline that has existed for several decades and has various theoretical results too, including VC theory etc. It is also not the same as statistics. They are adjacent but different fields.
    - Certhas 6y ago
      Yeah, but no. It definitely is a buzzword that randomly gets attached to every method of data analysis these days. If you disagree, then please give me a definition that is both distinct from statistics but also means that Regression is an ML technique. BTW, that doesn't mean there isn't also very substantial work done under the umbrella term. Regression analysis was first introduced by Legendre in the early 19th century. I never would have dreamed to pretend it is Machine Learning 10 years ago when the term ML was still used somewhat more sparingly, and carried a bit more meaning... But "I will use ML to analyse the data" gets more funding than "I will run a regression on the data".
      - bonoboTP 6y ago
        I won't type up a chapter about this now but ML is about learning from data. It is concerned with training/validation/test data, the generalization error etc. Statisticians care more about "modeling", actually estimating parameters that stand in for something real, to check assumptions and hypotheses. The cultures are very different. What makes total sense to one may baffle the other as lunacy (there's more convergence now though, by realizing the complementary nature of the two approaches). The terminology is different too: in stats, they call logistic regression regression, while in ML it would have been called classification (but the name is now stuck). ML also tends to be more Bayesian than frequentist, as opposed to classical stats. I could write more but having taken ML courses before the big hype times, I can assure you ML doesn't just mean hyped up startup fluff vaporware. Open up Kevin Murphy's ML book and compare the TOC with a Statistics textbook. There is overlap but it's certainly different in its goals and mentality. It seems like some bitter bickering from the side of stats people that they didn't manage to hype up their field as much. Yeah they did have many of the same tools at their disposal but didn't use them in this way.
        6gvONxR4sf7o 6y ago
        You're setting up a strawman of statistics. You're describing a small part of a large field. Stats has been more than what you're describing for longer than ML has been a phrase. The only real useful definition these days is that stats is learning from data that happens in the stats department and ML is learning from data that happens in the CS department.
        bonoboTP 6y ago
        I'm fine with that. I would also hope for more collab cross-departments, but the incentives in academia are unfortunately not exactly set up the right way. People become protective of "their turf". I wrote a bit more here: https://news.ycombinator.com/item?id=26984234 https://news.ycombinator.com/item?id=26984234 I think it's good if people are aware of the origins of the methods they use. Regression wasn't invented under the umbrella of ML, but is analyzed from a particular angle in ML more than in stats.
    - 6gvONxR4sf7o 6y ago
      > Regression is part of ML for sure. > ML is a discipline that has existed for several decades If you're going to say regression is part of ML, you can't say ML has only existed for decades. If you define it with that expansive scope, it's existed for centuries.
      - bonoboTP 6y ago
        ML uses regression in a particular way with particular aims and has developed prior approaches like OLS and Ridge regression, lasso, kriging etc. into a particular theoretical framework and analyses them from the angle of learning, i.e. heavy focus on generalization from training data to test data. The truth is, these are tools that many communities have used in parallel and these communities are in constant flux regarding what promising directions they find and exploit and when they make a big splash in one place, others take notice and incorporate those ideas etc. There are no rigidly predefined disciplines and fields over long timescales. When electrical engineers and signal processing communities worked on similar things they called it "pattern recognition". Machine learning as a field started out with more ambition than mere regression and classification (and indeed covers a lot more ground), but it turns out that supervised learning has had the most practical success. But it's a research program and community that goes beyond that. Similarly, there are parallels and similar equations between control theory and reinforcement learning. And indeed some controllers can be expressed as reinforcement learning agents. But the aims of the two communities are not the same. Maybe people would be happier if "statistical learning" (which is also used) was used more instead of "machine learning"? But indeed as another comment points out, ML as a paradigm does not necessarily require learning of a statistical kind. Labels grow and wane in popularity, it doesn't mean it's the same thing repackaged, rather that the aims and the focus changes depending on what we find most productive and fruitful. For example many of these things were also called "soft computing" a few years ago, but that term is rarely seen nowadays.
        Certhas 6y ago
        This sounds like in the end you agree that a lot of ML is applied stats? The problem in my mind is not that ML is using a lot of stats (obviously), it's that foundational mathematical concepts get labelled as ML techniques. This is why the title of the post is so annoying. This totally obscures the structure of the field. E.g. I wouldn't call linear algebra a quantum mechanics technique. I would say that QM uses (and spurred the development of) a lot of LinAlg.
  - aulin 6y ago
    why linear though
  - dunefox 6y ago
    You mean as in linear and logistic regression, two popular classical ML models whose parameters can be optimised using gradient descent?
- visarga 6y ago
  > Where do we go next ? Neural nets are all you need[*]. [*] if what you need is non-robust black boxes

I wish your comment is voted up. Where do we go next ? -- look addition is a neural network, look weighted average is a neural network, look linear regression, logistic regression, Poisson regression are neural networks ...

Certhas 6y ago
I have definitely seen regressions referred to as linear machine learning. Not even kidding.
- nikk1 6y ago
  Regression is an approach to machine learning https://en.wikipedia.org/wiki/Machine_learning#Regression_analysis https://en.wikipedia.org/wiki/Machine_learning#Regression_an...
- bonoboTP 6y ago
  So? Machine learning is not some buzzword fad. Regression is part of ML for sure. Some people seem to think ML means fancy deep learning GPT-3 transformer running on a TPU farm. Actually ML is a discipline that has existed for several decades and has various theoretical results too, including VC theory etc. It is also not the same as statistics. They are adjacent but different fields.
  - Certhas 6y ago
    Yeah, but no. It definitely is a buzzword that randomly gets attached to every method of data analysis these days. If you disagree, then please give me a definition that is both distinct from statistics but also means that Regression is an ML technique. BTW, that doesn't mean there isn't also very substantial work done under the umbrella term. Regression analysis was first introduced by Legendre in the early 19th century. I never would have dreamed to pretend it is Machine Learning 10 years ago when the term ML was still used somewhat more sparingly, and carried a bit more meaning... But "I will use ML to analyse the data" gets more funding than "I will run a regression on the data".
    - bonoboTP 6y ago
      I won't type up a chapter about this now but ML is about learning from data. It is concerned with training/validation/test data, the generalization error etc. Statisticians care more about "modeling", actually estimating parameters that stand in for something real, to check assumptions and hypotheses. The cultures are very different. What makes total sense to one may baffle the other as lunacy (there's more convergence now though, by realizing the complementary nature of the two approaches). The terminology is different too: in stats, they call logistic regression regression, while in ML it would have been called classification (but the name is now stuck). ML also tends to be more Bayesian than frequentist, as opposed to classical stats. I could write more but having taken ML courses before the big hype times, I can assure you ML doesn't just mean hyped up startup fluff vaporware. Open up Kevin Murphy's ML book and compare the TOC with a Statistics textbook. There is overlap but it's certainly different in its goals and mentality. It seems like some bitter bickering from the side of stats people that they didn't manage to hype up their field as much. Yeah they did have many of the same tools at their disposal but didn't use them in this way.
      - 6gvONxR4sf7o 6y ago
        You're setting up a strawman of statistics. You're describing a small part of a large field. Stats has been more than what you're describing for longer than ML has been a phrase. The only real useful definition these days is that stats is learning from data that happens in the stats department and ML is learning from data that happens in the CS department.
        bonoboTP 6y ago
        I'm fine with that. I would also hope for more collab cross-departments, but the incentives in academia are unfortunately not exactly set up the right way. People become protective of "their turf". I wrote a bit more here: https://news.ycombinator.com/item?id=26984234 https://news.ycombinator.com/item?id=26984234 I think it's good if people are aware of the origins of the methods they use. Regression wasn't invented under the umbrella of ML, but is analyzed from a particular angle in ML more than in stats.
  - 6gvONxR4sf7o 6y ago
    > Regression is part of ML for sure. > ML is a discipline that has existed for several decades If you're going to say regression is part of ML, you can't say ML has only existed for decades. If you define it with that expansive scope, it's existed for centuries.
    - bonoboTP 6y ago
      ML uses regression in a particular way with particular aims and has developed prior approaches like OLS and Ridge regression, lasso, kriging etc. into a particular theoretical framework and analyses them from the angle of learning, i.e. heavy focus on generalization from training data to test data. The truth is, these are tools that many communities have used in parallel and these communities are in constant flux regarding what promising directions they find and exploit and when they make a big splash in one place, others take notice and incorporate those ideas etc. There are no rigidly predefined disciplines and fields over long timescales. When electrical engineers and signal processing communities worked on similar things they called it "pattern recognition". Machine learning as a field started out with more ambition than mere regression and classification (and indeed covers a lot more ground), but it turns out that supervised learning has had the most practical success. But it's a research program and community that goes beyond that. Similarly, there are parallels and similar equations between control theory and reinforcement learning. And indeed some controllers can be expressed as reinforcement learning agents. But the aims of the two communities are not the same. Maybe people would be happier if "statistical learning" (which is also used) was used more instead of "machine learning"? But indeed as another comment points out, ML as a paradigm does not necessarily require learning of a statistical kind. Labels grow and wane in popularity, it doesn't mean it's the same thing repackaged, rather that the aims and the focus changes depending on what we find most productive and fruitful. For example many of these things were also called "soft computing" a few years ago, but that term is rarely seen nowadays.
      - Certhas 6y ago
        This sounds like in the end you agree that a lot of ML is applied stats? The problem in my mind is not that ML is using a lot of stats (obviously), it's that foundational mathematical concepts get labelled as ML techniques. This is why the title of the post is so annoying. This totally obscures the structure of the field. E.g. I wouldn't call linear algebra a quantum mechanics technique. I would say that QM uses (and spurred the development of) a lot of LinAlg.
- aulin 6y ago
  why linear though
- dunefox 6y ago
  You mean as in linear and logistic regression, two popular classical ML models whose parameters can be optimised using gradient descent?
visarga 6y ago
> Where do we go next ? Neural nets are all you need[*]. [*] if what you need is non-robust black boxes