9 ms·

I see people saying this all over the internet, but for the entire history of AI, Machine Learning has either been the only, or the most effective method by far

I see people saying this all over the internet, but for the entire history of AI, Machine Learning has either been the only, or the most effective method by far of solving actual problems. Everyone who thinks otherwise should read this. http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html

wodenokoto 4y ago
Didn't expert systems rule in early AI research?
CuriousCosmic 4y ago
Ultimately the only thing that holds true about AI is Tesler's Theorem: Artificial intelligence is whatever hasn't been done yet Machine Learning is only still artificial intelligence because we haven't figured it out. Ultimately ML is just fancy control theory. Once the field is sufficiently explored, it will slowly stop being AI and will start being treated as a subset of control theory instead. I doubt it'll happen any time soon but it'll happen eventually. This is in the same way that algorithms, combinatorics, and graph theory when applied were seen as AI anywhere from 70 years ago to as little as ~25 years ago. Those fields' applications to unsolved computing problems (that humans could do) was what made them AI and once we solved those problems, they (and the techniques used) stopped being AI.
- FreakLegion 4y ago
  People in the field still think of these things as being part of it. The latest edition of Norvig's Artificial Intelligence: A Modern Approach[1] is two years old and kicks off with search, CSPs, and first-order logic, for example. Over the years AIMA has kept up with a bunch of stuff that historically has in fact been far more effective than machine learning, like automated planning, which is the basis for (among other things) most video game AI. An all-time great write-up and talk in this area is Three States and a Plan: The A.I. of F.E.A.R.[2]. And then there's linear and non-linear programming and the broader world of operations research, which not only have had more of an impact historically than machine learning but, when you get down to it, are also a useful lens for thinking about machine learning for people who use it but don't really understand it. Training a neural network and solving an MINLP problem are very similar things. 1. https://aima.cs.berkeley.edu/contents.html https://aima.cs.berkeley.edu/contents.html 2. https://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf https://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear... and https://www.gdcvault.com/play/1013282/Three-States-and-a-Plan https://www.gdcvault.com/play/1013282/Three-States-and-a-Pla...
- mozball 4y ago
  Indeed. Its surprising how quickly people are getting used to GPT's capabilities and dismissing it even. It wasn't that long ago that people were saying that making casual conversation, making an original joke or novel painting would be among the last things a computer would be able to do. Or that it would never happen.
Buttons840 4y ago
> for the entire history of AI, Machine Learning has either been the only, or the most effective method by far of solving actual problems. Did Deep Blue not act intelligently? It didn't use machine learning.
bnegreve 4y ago
Hum, I think you are misunderstanding Sutton here. He argues that the we should focus on search and learning, but nowhere he implies that ML is the only way to do it. In fact, many old school AI, non-ML techniques fit this description. For example, SAT/CSP solving are almost entirely about search, and sometime even about learning, but not in the machine learning sense. He even mentions "the methods defeated the world champion" at chess as a successful demonstration of AI, but these methods are certainly not what we would call machine learning today (deep search doesn't mean deep learning).

wodenokoto 4y ago

Didn't expert systems rule in early AI research?

CuriousCosmic 4y ago

Ultimately the only thing that holds true about AI is Tesler's Theorem: Artificial intelligence is whatever hasn't been done yet Machine Learning is only still artificial intelligence because we haven't figured it out. Ultimately ML is just fancy control theory. Once the field is sufficiently explored, it will slowly stop being AI and will start being treated as a subset of control theory instead. I doubt it'll happen any time soon but it'll happen eventually. This is in the same way that algorithms, combinatorics, and graph theory when applied were seen as AI anywhere from 70 years ago to as little as ~25 years ago. Those fields' applications to unsolved computing problems (that humans could do) was what made them AI and once we solved those problems, they (and the techniques used) stopped being AI.

FreakLegion 4y ago
People in the field still think of these things as being part of it. The latest edition of Norvig's Artificial Intelligence: A Modern Approach[1] is two years old and kicks off with search, CSPs, and first-order logic, for example. Over the years AIMA has kept up with a bunch of stuff that historically has in fact been far more effective than machine learning, like automated planning, which is the basis for (among other things) most video game AI. An all-time great write-up and talk in this area is Three States and a Plan: The A.I. of F.E.A.R.[2]. And then there's linear and non-linear programming and the broader world of operations research, which not only have had more of an impact historically than machine learning but, when you get down to it, are also a useful lens for thinking about machine learning for people who use it but don't really understand it. Training a neural network and solving an MINLP problem are very similar things. 1. https://aima.cs.berkeley.edu/contents.html https://aima.cs.berkeley.edu/contents.html 2. https://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear.pdf https://alumni.media.mit.edu/~jorkin/gdc2006_orkin_jeff_fear... and https://www.gdcvault.com/play/1013282/Three-States-and-a-Plan https://www.gdcvault.com/play/1013282/Three-States-and-a-Pla...
mozball 4y ago
Indeed. Its surprising how quickly people are getting used to GPT's capabilities and dismissing it even. It wasn't that long ago that people were saying that making casual conversation, making an original joke or novel painting would be among the last things a computer would be able to do. Or that it would never happen.

Buttons840 4y ago

> for the entire history of AI, Machine Learning has either been the only, or the most effective method by far of solving actual problems. Did Deep Blue not act intelligently? It didn't use machine learning.

bnegreve 4y ago

Hum, I think you are misunderstanding Sutton here. He argues that the we should focus on search and learning, but nowhere he implies that ML is the only way to do it. In fact, many old school AI, non-ML techniques fit this description. For example, SAT/CSP solving are almost entirely about search, and sometime even about learning, but not in the machine learning sense. He even mentions "the methods defeated the world champion" at chess as a successful demonstration of AI, but these methods are certainly not what we would call machine learning today (deep search doesn't mean deep learning).