7 ms·

If you want to draw the line between ML and not ML, I think you’ll have to put Kalman filters and linear regression on the non-ML side. You can put support vect

If you want to draw the line between ML and not ML, I think you’ll have to put Kalman filters and linear regression on the non-ML side. You can put support vector machines and neural networks on the ML side. In some sense the exact place you draw the distinction is arbitrary. You could try to characterize where the distinction is by saying that models with fewer parameters and lower complexity tend to be called “not ML”, and models with more parameters and higher complexity tend to be called “ML”.

wrsh07 2y ago
Linear regression is literally the second lecture of the Stanford ML class. https://cs229.stanford.edu/ https://cs229.stanford.edu/ If you want to say "not neural networks" or not dnn or not llm, sure. But it's obviously machine learning
- klodolph 2y ago
  When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such as differentiability.
  - computerex 2y ago
    Linear regression is ML. You are off base.
    - genewitch 2y ago
      That's cool. Can you explain what machines they were using in the 1800s to do learning on?
      - wrsh07 2y ago
        Does this mean if I simulate a neural network on pen and paper that stops being machine learning? All of these are machine learning techniques. Doing it by hand doesn't change anything. Today we use machines so it's machine learning
        zeroCalories 2y ago
        I've worked out a 3x3 neural network with two pieces of labeled data by hand. Give me my statistics degree!
      - windsignaling 2y ago
        You're using the word to define the concept rather than the concept to define the word. Wrong order. See: "Program".
  - Retric 2y ago
    If you want to draw a line you could say anything that’s not full AGI isn’t machine learning because of philosophical reasons. But, other than that there’s the only clear line is when the programmer isn’t hard coding results which puts Linear regression over the ML line. I guess you could argue about supervised vs unsupervised algorithms, but that’s going to exclude a lot of what is generally described as ML.
  - wrsh07 2y ago
    It is obviously machine learning. It is a machine learning algorithm. It is taught in machine learning classes. It is described as a machine learning algorithm literally everywhere. Here's wiki: > Linear regression is also a type of machine learning algorithm, more specifically a supervised algorithm, that learns from the labelled datasets and maps the data points to the most optimized linear functions that can be used for prediction on new datasets You can pretend it's not because it's not a sophisticated machine learning algorithm, but you are wrong.
  - windsignaling 2y ago
    After spending over a decade in both statistics and machine learning I'd say the only reason there isn't a "broad consensus" is because statisticians like to gate-keep, whether that's linear regression, Monte Carlo methods, or Kalman Filters. Linear regression appears in pretty much every ML textbook. Can you confidently say, "this model that appears in every ML textbook is the only model in the ML textbook that isn't an ML model"? Kalman Filters are like a continuous-state HMM. So why are HMMs considered ML and Kalman Filters not considered ML? IMO it's an ego thing. They spent decades rigorously analyzing everything about linear models and here come these CS cowboys producing amazing results without any of the careful rigor that statisticians normally apply. It's difficult to argue against real results so the inflexible, hard-nosed statisticians just hang on to whatever they can.
    - klodolph 2y ago
      That’s unfairly harsh to statisticians, IMO. You have two fields of study, statistics and ML. There’s a massive overlap. Gatekeeping? Practitioners from these two fields have different jargon and view things differently from each other. “X is taught in books about subject Y” is a pretty weak argument. I could use it to argue that group theory is quantum mechanics. Scientists and mathematics aren’t fighting over who gets to own group theory—the scientists get to put group theory in their toolboxes and the mathematicians get to study it for itself. Same with ML and statistics. When you do ML, you need certain statistical techniques in your toolbox, so they get taught in your ML books.
- deleted 2y ago
  [deleted]
jll29 2y ago
AI professor here. Anything that can separate data apoints can rightly been seen as a "supervised machine learning classifier". Todemystify the area, I literally introduce my intro to ML lecture by drawing a line on the board, give its equation y = 0.5 x on the backboard, reminding students that they already know this, and then explain how to use it as a spam filter by interpreting the points on either side of the line as good emails versus spam ones.
bigmadshoe 2y ago
Linear regression is machine learning. At their core neural networks are just repeated linear regression + a non-linearity arranged in interesting ways. The key is that they can be trained to fit data using some optimization protocol (e.g. gradient descent). Just because linear regression has a closed form solution and is conceptually simple doesn't mean anything here.

wrsh07 2y ago

Linear regression is literally the second lecture of the Stanford ML class. https://cs229.stanford.edu/ https://cs229.stanford.edu/ If you want to say "not neural networks" or not dnn or not llm, sure. But it's obviously machine learning

klodolph 2y ago
When you say it’s “obviously machine learning”, how could that statement possibly be correct? There’s not even broad consensus here… so you don’t get to say that your definition is obviously correct. There are pedagogical reasons why you’d include linear regression in a machine learning course. This is pretty clear to me—they have properties which are extremely important to the field of machine learning field, such as differentiability.
- computerex 2y ago
  Linear regression is ML. You are off base.
  - genewitch 2y ago
    That's cool. Can you explain what machines they were using in the 1800s to do learning on?
    - wrsh07 2y ago
      Does this mean if I simulate a neural network on pen and paper that stops being machine learning? All of these are machine learning techniques. Doing it by hand doesn't change anything. Today we use machines so it's machine learning
      - zeroCalories 2y ago
        I've worked out a 3x3 neural network with two pieces of labeled data by hand. Give me my statistics degree!
    - windsignaling 2y ago
      You're using the word to define the concept rather than the concept to define the word. Wrong order. See: "Program".
- Retric 2y ago
  If you want to draw a line you could say anything that’s not full AGI isn’t machine learning because of philosophical reasons. But, other than that there’s the only clear line is when the programmer isn’t hard coding results which puts Linear regression over the ML line. I guess you could argue about supervised vs unsupervised algorithms, but that’s going to exclude a lot of what is generally described as ML.
- wrsh07 2y ago
  It is obviously machine learning. It is a machine learning algorithm. It is taught in machine learning classes. It is described as a machine learning algorithm literally everywhere. Here's wiki: > Linear regression is also a type of machine learning algorithm, more specifically a supervised algorithm, that learns from the labelled datasets and maps the data points to the most optimized linear functions that can be used for prediction on new datasets You can pretend it's not because it's not a sophisticated machine learning algorithm, but you are wrong.
- windsignaling 2y ago
  After spending over a decade in both statistics and machine learning I'd say the only reason there isn't a "broad consensus" is because statisticians like to gate-keep, whether that's linear regression, Monte Carlo methods, or Kalman Filters. Linear regression appears in pretty much every ML textbook. Can you confidently say, "this model that appears in every ML textbook is the only model in the ML textbook that isn't an ML model"? Kalman Filters are like a continuous-state HMM. So why are HMMs considered ML and Kalman Filters not considered ML? IMO it's an ego thing. They spent decades rigorously analyzing everything about linear models and here come these CS cowboys producing amazing results without any of the careful rigor that statisticians normally apply. It's difficult to argue against real results so the inflexible, hard-nosed statisticians just hang on to whatever they can.
  - klodolph 2y ago
    That’s unfairly harsh to statisticians, IMO. You have two fields of study, statistics and ML. There’s a massive overlap. Gatekeeping? Practitioners from these two fields have different jargon and view things differently from each other. “X is taught in books about subject Y” is a pretty weak argument. I could use it to argue that group theory is quantum mechanics. Scientists and mathematics aren’t fighting over who gets to own group theory—the scientists get to put group theory in their toolboxes and the mathematicians get to study it for itself. Same with ML and statistics. When you do ML, you need certain statistical techniques in your toolbox, so they get taught in your ML books.
deleted 2y ago
[deleted]

jll29 2y ago

AI professor here. Anything that can separate data apoints can rightly been seen as a "supervised machine learning classifier". Todemystify the area, I literally introduce my intro to ML lecture by drawing a line on the board, give its equation y = 0.5 x on the backboard, reminding students that they already know this, and then explain how to use it as a spam filter by interpreting the points on either side of the line as good emails versus spam ones.

bigmadshoe 2y ago

Linear regression is machine learning. At their core neural networks are just repeated linear regression + a non-linearity arranged in interesting ways. The key is that they can be trained to fit data using some optimization protocol (e.g. gradient descent). Just because linear regression has a closed form solution and is conceptually simple doesn't mean anything here.