7 ms·

+1. What happened to ML? With the relatively recent craze precipitated by chatgpt, the term AI (perhaps in no small part due to "OpenAI") has completely taken

+1. What happened to ML? With the relatively recent craze precipitated by chatgpt, the term AI (perhaps in no small part due to "OpenAI") has completely taken over. ML is a more apt description of the current wave.

gwd 2y ago
> What happened to ML? Well for one, five years ago, anyone doing machine learning was doing their own training -- you know, overseeing the actual learning part. Now, although there is learning involved in LLMs, you as the consumer aren't involved in that part. You're given API access (or at best a set of pre-trained weights) as a black box, and you do what you can with it.
TrapLord_Rhodo 2y ago
AI -> Machine learning -> Deep Learning -> LLM's - LMM's -> ?
- notthatoldguy 2y ago
  AI -> Machine learning -> Deep Learning -> LLM's - LMM's new math for knowledge based systems <- ANN <- ML <- KRR <- LMM's
keiferski 2y ago
Yep, the difference is that ML doesn’t have the cultural legacy of the AI concept and thus isn’t as marketable.
floren 2y ago
After the AI Winter, people started calling it ML to avoid the stigma. Eventually ML got pretty good and a lot of the industry forgot the AI winter, so we're calling it AI again because it sounds cooler.
- notthatoldguy 2y ago
  Interesting opinion but no AI is a science ML is just one of the areas of this science
- gessha 2y ago
  There’s a little more to it. Learning-based systems didn’t prove themselves until the ImageNet moment around 2013. Machine learning wasn’t used in previous AI hype cycles because it wasn’t a learning system but what we now call good old fashioned AI - GOFAI - hard coded features, semantic web, etc.
  - sharma-arjun 2y ago
    Are classical machine learning techniques that don't involve neural networks / DL not considered "learning-based systems" anymore? I would argue that even something as simple as linear regression can and should be considered a learning-based system, even ignoring more sophisticated algorithms such as SVMs or boosted tree regression models. And these were in use for quite a while before the ImageNet moment, albeit not with the same level of visibility.
    - Jach 2y ago
      I've just accepted that these broad terms are really products of their time, and use of them just means people want to communicate at a higher level above the details. (Whether that's because they don't know the details, or because the details aren't important for the context.) It's a bit better than "magic" but not that different; the only real concern I have is vague terms making it into laws or regulations. I agree on linear regression, and I remember being excited by things like random forests in the ML sphere that no one seems to talk about anymore. I think under current standards of vagueness, even basic things like a color bucket-fill operation in a paint program count as "AI techniques". In the vein of slap a gear on it and call it steampunk, slap a search step on it and call it AI, or slap a past-data storage step on it and call it learning.
zer00eyz 2y ago
Open AI was betting on AI, AGI, Super inteligence. Look at the google engineer who thought they had an AI locked up in the basement... https://www.theverge.com/2022/6/13/23165535/google-suspends-ai-artificial-intelligence-engineer-sentient https://www.theverge.com/2022/6/13/23165535/google-suspends-... MS paper on sparks of AGI: https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-... The rumors that OpenAI deal with MS would give them everything till they got to AGI... A perpetual license to all new development. All the "Safety people" have left the OpenAi building. Even musk isnt talking about safety any more. I think the bet was that if you fed an LLM enough, got it big enough it would hit a tipping point, and become AGI, or sentient or sapient. That lines up nicely with the MS terms, and MS's on paper. I think they figured out that the math doesn't work that way (and never was going to). A prediction of the next token being better isnt intelligence any more than weather prediction will become weather.
- theGnuMe 2y ago
  Weather prediction is pretty on point for 5 days out. Also the state of the art is a graph neural network transformer.
- cyberax 2y ago
  > A prediction of the next token being better isnt intelligence Why? I think it absolutely can be intelligence.
  - RealityVoid 2y ago
    I think it depends on how you define intelligence, but _I_ mostly agree with Francois Collet's stance that intelligence is the ability to find novel solutions and adaptability to new challenges. He feels that memorisation is an important facet, but it is not enough for true intelligence ant that these systems excel at type2 thinking but gave huge gaps at type1. The alternative I'm considering is that It might just be that it's just a dataset problem, feeding these llms on words makes the lack a huge facet of embodied axistance that is needed to get context. I am a nobody though, so who knows....
jhbadger 2y ago
>What happened to ML? It's cyclical. The term "ML" was popular in the 1990s/early 2000s because "AI" had a bad odor of quackdom due to the failures of things like expert systems in the 1980s. The point was to reduce the hype and say "we're not interested in creating AGI; we just want computers to run classification tasks". We'll probably come up with a new term in the future to describe LLMs in the niches where they are actually useful after the current hype passes.
- daemin 2y ago
  Is there a good reference available that describes what happened with expert systems in the 80's? I'm only vaguely aware about such things but in my mind they seem to have some utility.
  - jhbadger 2y ago
    H.P. Newquist's "The Brain Makers: Genius, Ego, and Greed In The Search For Machines That Think" (1994) is a good book that centers on the second AI bubble (1980-1987) and its collapse. It is maybe a bit too focused on the companies and CEOs themselves rather than the technology, but it does cover the main problem of expert systems -- their brittleness when faced with a situation they haven't been designed for. One of the things that we learned with expert systems is it is better to have probabilistic weights for things rather than the IF THEN ELSE branches of a traditional expert system -- this led to things like Bayesian models which were popular before deep learning took over.
sk11001 2y ago
> the term AI (...) has completely taken over Maybe if you go by article titles it has. If you look at job titles, there are many more ML engineers than AI engineers.
konfusinomicon 2y ago
I make it a point to use ML in every work meeting or interaction where AI is the subject. it has not gained traction.

gwd 2y ago

> What happened to ML? Well for one, five years ago, anyone doing machine learning was doing their own training -- you know, overseeing the actual learning part. Now, although there is learning involved in LLMs, you as the consumer aren't involved in that part. You're given API access (or at best a set of pre-trained weights) as a black box, and you do what you can with it.

TrapLord_Rhodo 2y ago

AI -> Machine learning -> Deep Learning -> LLM's - LMM's -> ?

notthatoldguy 2y ago
AI -> Machine learning -> Deep Learning -> LLM's - LMM's new math for knowledge based systems <- ANN <- ML <- KRR <- LMM's

keiferski 2y ago

Yep, the difference is that ML doesn’t have the cultural legacy of the AI concept and thus isn’t as marketable.

floren 2y ago

After the AI Winter, people started calling it ML to avoid the stigma. Eventually ML got pretty good and a lot of the industry forgot the AI winter, so we're calling it AI again because it sounds cooler.

notthatoldguy 2y ago
Interesting opinion but no AI is a science ML is just one of the areas of this science
gessha 2y ago
There’s a little more to it. Learning-based systems didn’t prove themselves until the ImageNet moment around 2013. Machine learning wasn’t used in previous AI hype cycles because it wasn’t a learning system but what we now call good old fashioned AI - GOFAI - hard coded features, semantic web, etc.
- sharma-arjun 2y ago
  Are classical machine learning techniques that don't involve neural networks / DL not considered "learning-based systems" anymore? I would argue that even something as simple as linear regression can and should be considered a learning-based system, even ignoring more sophisticated algorithms such as SVMs or boosted tree regression models. And these were in use for quite a while before the ImageNet moment, albeit not with the same level of visibility.
  - Jach 2y ago
    I've just accepted that these broad terms are really products of their time, and use of them just means people want to communicate at a higher level above the details. (Whether that's because they don't know the details, or because the details aren't important for the context.) It's a bit better than "magic" but not that different; the only real concern I have is vague terms making it into laws or regulations. I agree on linear regression, and I remember being excited by things like random forests in the ML sphere that no one seems to talk about anymore. I think under current standards of vagueness, even basic things like a color bucket-fill operation in a paint program count as "AI techniques". In the vein of slap a gear on it and call it steampunk, slap a search step on it and call it AI, or slap a past-data storage step on it and call it learning.

zer00eyz 2y ago

Open AI was betting on AI, AGI, Super inteligence. Look at the google engineer who thought they had an AI locked up in the basement... https://www.theverge.com/2022/6/13/23165535/google-suspends-ai-artificial-intelligence-engineer-sentient https://www.theverge.com/2022/6/13/23165535/google-suspends-... MS paper on sparks of AGI: https://www.microsoft.com/en-us/research/publication/sparks-of-artificial-general-intelligence-early-experiments-with-gpt-4/ https://www.microsoft.com/en-us/research/publication/sparks-... The rumors that OpenAI deal with MS would give them everything till they got to AGI... A perpetual license to all new development. All the "Safety people" have left the OpenAi building. Even musk isnt talking about safety any more. I think the bet was that if you fed an LLM enough, got it big enough it would hit a tipping point, and become AGI, or sentient or sapient. That lines up nicely with the MS terms, and MS's on paper. I think they figured out that the math doesn't work that way (and never was going to). A prediction of the next token being better isnt intelligence any more than weather prediction will become weather.

theGnuMe 2y ago
Weather prediction is pretty on point for 5 days out. Also the state of the art is a graph neural network transformer.
cyberax 2y ago
> A prediction of the next token being better isnt intelligence Why? I think it absolutely can be intelligence.
- RealityVoid 2y ago
  I think it depends on how you define intelligence, but _I_ mostly agree with Francois Collet's stance that intelligence is the ability to find novel solutions and adaptability to new challenges. He feels that memorisation is an important facet, but it is not enough for true intelligence ant that these systems excel at type2 thinking but gave huge gaps at type1. The alternative I'm considering is that It might just be that it's just a dataset problem, feeding these llms on words makes the lack a huge facet of embodied axistance that is needed to get context. I am a nobody though, so who knows....

jhbadger 2y ago

>What happened to ML? It's cyclical. The term "ML" was popular in the 1990s/early 2000s because "AI" had a bad odor of quackdom due to the failures of things like expert systems in the 1980s. The point was to reduce the hype and say "we're not interested in creating AGI; we just want computers to run classification tasks". We'll probably come up with a new term in the future to describe LLMs in the niches where they are actually useful after the current hype passes.

daemin 2y ago
Is there a good reference available that describes what happened with expert systems in the 80's? I'm only vaguely aware about such things but in my mind they seem to have some utility.
- jhbadger 2y ago
  H.P. Newquist's "The Brain Makers: Genius, Ego, and Greed In The Search For Machines That Think" (1994) is a good book that centers on the second AI bubble (1980-1987) and its collapse. It is maybe a bit too focused on the companies and CEOs themselves rather than the technology, but it does cover the main problem of expert systems -- their brittleness when faced with a situation they haven't been designed for. One of the things that we learned with expert systems is it is better to have probabilistic weights for things rather than the IF THEN ELSE branches of a traditional expert system -- this led to things like Bayesian models which were popular before deep learning took over.

sk11001 2y ago

> the term AI (...) has completely taken over Maybe if you go by article titles it has. If you look at job titles, there are many more ML engineers than AI engineers.

konfusinomicon 2y ago

I make it a point to use ML in every work meeting or interaction where AI is the subject. it has not gained traction.