13 ms·

We can’t trust AI systems built on deep learning alone

205 points by laurex 7y ago

cs702 7y ago
The opposing argument, by Rich Sutton, Distinguished Research Scientist at DeepMind: http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html
- justicezyx 7y ago
  Seems talking about different issues
- cjauvin 7y ago
  I would say this is also an opposing argument: https://twitter.com/ylecun/status/1066568396177842176 https://twitter.com/ylecun/status/1066568396177842176 i.e. gradient-based learning is the final word on the matter.
  - YeGoblynQueenne 7y ago
    Yes, gradient optimisation always gets the final word. But what does that word, well, say? _ / \ ,----------------------------. | Hey! I found your minimum! | '------------ ------------' / .----. \ / / \ / / \ \/ / \ / / \ @ / \ / / `'--' \ / / \ / / \ / / \ / / \ .-. / / \_/ `\ / \.__.' \ / \ / `;._.-'
- Barrin92 7y ago
  there are some surprisingly weak arguments in the text. It's correct to not treat computational resources as constant, but ot treat them as unimportant or negligible is awful as well. Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. If the goal is AGI this might get exponentially worse. Intelligence needs to take resource consumption into account. The models we produce aren't even close to high level reasoning and we're already consuming significantly more energy than humans or animals, something is wrong. The scale argument isn't great either because deep learning is running into the inverse issue of classical AI. Now instead of having to program all logic explicitly we have to formulate every individual problem as training data. This doesn't scale either. If an AI gets attacked by a wild animal the solution can't be to first produce 10k pictures of mauled victims, intelligence includes to reason about things in the abscence of data. We can't have autonomous cars constantly running into things until we provide huge amounts of data for every problem, this does not scale either.
  - roenxi 7y ago
    > Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. That is something of an illusion. Obviously there will be some sort of uneven distribution of computing power; some institutions will have more, some less. The institutions with more power will create models at the limit of what they can do, because that is the best use of their power. So if the thesis of more power = more results holds then truly cutting results will always be by people with resources that are practically unattainable by everyone else. Google's AlphaGo wasn't a particularly clever model, for example. It just had a lot of horsepower behind it to train it and the various ranging shot attempts Deepmind would have gone through. Someone else would have figured it out albeit more slowly in a few years as computing power became available. Computational power is still getting exponentially more affordable [0]. Costs aren't really rising, so much as the people who have spent more money get a few years ahead of everyone else and can preview what is about to become cheap. [0] https://aiimpacts.org/recent-trend-in-the-cost-of-computing/ https://aiimpacts.org/recent-trend-in-the-cost-of-computing/
    - 7y ago
- deleted 7y ago
  [deleted]
- 6gvONxR4sf7o 7y ago
  This argument completely ignores anything statistical, which is another limiting factor. It reminds me of the difference in bandit research and full RL research. RL researchers are fine throwing a thousand years of experience at their algorithm, because it can be simulated. Meanwhile people using bandits in the real world care about statistical efficiency (learning lots with little data), and it's reflected in the research. Most decisions aren't made with a huge abundance of data (most of us aren't google or facebook).
sebastianconcpt 7y ago
In cognitive science we talk about having cognitive models of things. So I’m sitting in a hotel room, and I understand that there’s a closet, there’s a bed, there’s the television that’s mounted in an unusual way. I know that there are all these things here, and I don’t just identify them. I also understand how they relate to one another. I have these ideas about how the outside world works. They’re not perfect. They’re fallible, but they’re pretty good. And I make a lot of inferences around them to guide my everyday actions. The opposite extreme is something like the Atari game system that DeepMind made, where it memorized what it needed to do as it saw pixels in particular places on the screen. If you get enough data, it can look like you’ve got understanding, but it’s actually a very shallow understanding. The proof is if you shift things by three pixels, it plays much more poorly. It breaks with the change. That’s the opposite of deep understanding. Of course. There are an infinte way to make interpretations of perceptions and a finite subset of possible valid ones. It's among those possible, that the AI will be a concrete implementation of an ideology. To select which one is always done by humans.
mark_l_watson 7y ago
I started reading Rebooting AI last night. I think that Marcus and Davis (so far in the book) take a reasonable approach by wanting to design robust AI. Robust AI requires general real world intelligence that is not provided by deep learning. I have earned over 90% of my income over the last five or six years as a deep learning practitioner. I am a fan of DL based on great results for perception tasks as well as solid NLP results like using BERT like models for things like anaphora resolution. But, I am in agreement with Marcus and Davis that our long term research priorities are wrong.
- turk73 7y ago
  Making 90% of your income off of this tech over the last n years is different than that tech being successful. I work at a very large company that is trying to use ML and AI in all kinds of places. The trend I am seeing is that most of that effort is falling flat, really flat, in fact. They have success in places where regular algorithms would also succeed, but just having regular developers design matching systems and do pretty basic statistics isn't sexy in terms of marketing, so they hush it all up and pretend that ML and neural nets and things are the only way forwards. I don't think it is. Our problem is that we have too many ETL robots who aren't very intelligent people and not very forward-thinking themselves! The cutting edge NLP stuff just showcased at my company was pretty lame, too. I barely saw any statistically significant results at all and yet they rather unscientifically proclaim success because they got any effect at all. Some of what we do in our field doesn't matter because it comes down to whether a customer got a 2nd call back and got converted to some minor sale or added to a program for them. It's throw away and creates good will at conferences and talks. We make a big deal out of it. We are spending hundreds of millions on projects, trying to save money on generating leads, reducing interactions with customers and vendors through staffed phone banks, and so on. My company has hired all kinds of academics and research type people and has given them titles of "Distinguished this" and "Principal that" and honestly there's not that much to show for it, maybe zero direct outcomes so far. What galls me the most is in all the conferences and demos they are showing off things like High School robotics vehicles and AI parlor tricks and astonishingly little has translated into the business we do. Meanwhile, there are people in the company who do know how to reduce costs and get more done and have outstanding outcomes, but their techniques are not sexy and thus unimportant to the PT Barnum MBAs running our company. I'm sure that's true most everywhere, of course. These Principals and Distinguisheds all keep proclaiming success while cashing fat paychecks. Meanwhile this year, our stock has had a tough go of it, so I'm curious whether these attempts will continue. The market takes no prisoners. Sure we get a lot of mileage out of looking cool for the recent grad crowd purposes of recruiting--kids want sexy, cool tech projects to work on and words like "insurance" turn them off, so there's that, I guess. My take on that is that it won't be long before all those new recruits will figure out they got bait and switched pretty bad and that they aren't going to get to work on any of this sexy ML and AI stuff anymore than I am in my role. I got lured in by Data Science (because PhD), which just shows how gullible I am, but at least some of that traditional statistical modeling is having an impact here and there. The problem again is that even that is overblown by a couple of orders of magnitude! In my project, we're simply trying to get more real-time data out to people who need it without having to call in to get it and that is ridiculously difficult because of all the systems we try to knit together and how overall terrible our data quality is. And now my boss wants to build out an "analytics engine" to capture some of this sexy ML and AI stuff. It leads me to believe that the people involved are most interested in getting promoted and not much more. Anyways, it is cool tech, but American taxpayers and people who are forced to buy our products are paying for it and I rather think they would prefer to spend their money in some better fashion.
keithyjohnson 7y ago
Understanding a sentence is fundamentally different from recognizing an object. But people are trying to use deep learning to do both. I agree with most of the article but I think this^^ skips over the different types of networks used to solve perception and language problems. A CNN is very different from say, word2vec, which isn't a very deep network at all.
- mlthoughts2018 7y ago
  I’d go further and say that deep networks are excellent for sentence understanding, and various types of RNN or 1D convolutional layers are very good at this in specialized domains just as CNNs and ResNets are good in specialized vision applications. It absolutely makes sense to use deep learning for both of these tasks. In fact, one very effective thing to do is to use a Siamese network to learn joint representational spaces of text and imagery in the same network. It’s really specious and disingenuous to say “boy, vision and language sure seem different but can you believe these DL researchers are using the same tools for both!?”
  - mattmcknight 7y ago
    or... "vision and language sure seem different, can you believe that networks of neurons in the brain do both?"
- jotakami 7y ago
  Perhaps the difference is in the nature of the information that is being probed and its larger context? Visual imagery often provides almost all of its own context, but the “meaning” of a sentence can be radically different depending upon its source. Humans produce words, so you almost need a working theory of mind to fully understand them. None of that context will ever make it into word2vec.
  - jkp56 7y ago
    Why not? It's true that the word "space" has different meanings when in appears in a math book, CS book or astronomy book. But we just have 3 different word2vec models. When I read something about math, I pick the math word2vec model and there "space" appears close to words "Hilbert" and "separable", while in the CS model, the same word is next to "complexity" and "memory". As I read more, I improve my word2vec models, but never mix them together. Now what happens if I'm reading something and don't understand the context? No, I don't switch to some general word2vec model. I rather try to guess which model to use and then reread the same text using that model.
mlthoughts2018 7y ago
This article doesn’t have any substance. It’s full of anecdata like shifting by 3 pixels to mess up a video game AI or some vague nonsense about “a model of this chair or this tv mounted to the wall.” It’s all casual hypotheticals. There’s plenty of research on Bayesian neural networks for causal inference. But even more, a lot of causal inference problems are “small data” problems where choosing a strongly informative prior to pair with simple models is needed to prevent overfitting and poor generalization and to account for domain expertise. Deep learning practitioners generally know plenty about this stuff and fully understand that deep neural networks are just one tool in the tool box, not applicable to all problems and certainly not approaching any kind of general AI solution that supersedes causal inference, feature engineering, etc. This article is just a sensationalist hit job trying to capitalize on public anxieties about AI to raise the profile of this academic and try to sell more copies of his book. I’d say, let’s not waste time on this crap. There are engineering problems that deep learning allows us to safely & reliably solve where other methods never could. We absolutely can trust these models for specific use cases. Let’s just get on with doing the work.
lngnmn1 7y ago
Why, yes. All life has the environmental constraints builtin in the structure of a brain and CNS. Brain also has implicit sets of filters in sensory channels and in pattern-marching machinery. Constraints and filters are the must have.
deleted 7y ago
[deleted]
wodenokoto 7y ago
In this context, what is classical AI? An SVM? A markov model? A large context free grammar with a dictionary?
- dreamcompiler 7y ago
  Yes. Or rules, logic, or other symbolic system. AI has always been divided into two camps: Connectionist and not. Current "AI" is all connectionist. What we're now calling "Classical AI" is the non-connectionist kind that was prominent in the 60s-80s but fell out of favor in the AI Winter.
- RobertDeNiro 7y ago
  I would say OR (operations research) is really what's meant here.
thelazydogsback 7y ago
I've been waiting for the Symbolic/NN pendulum to starting swinging back the other way and start settling in the center. NN/DL is great for the interface between the outer world and the inner world of the mind (pattern recognition and re-construction), and symbolic AI more straightforwardly represents more "language of the mind" tasks, and easily handles issues like explanation and other meta-behaviors that with DL is difficult due to its black-box nature. DL's reliance on extension/training vs. intention/rules can develop ad-hoc intentional emergent theories which is their strength but also their weakness as these theories may not be correct or complete. Each can be brittle in their own way - so it'll be interesting to see more cross-pollination.
- uoaei 7y ago
  SAT solvers are really fast now. Some sort of "neural SAT problem definition" followed by solving it seems to be an interesting direction, but I'm relatively naive on it all. Not sure how training would work since there's no backprop through Boolean logic.
  - w_t_payne 7y ago
    Differentiable SAT solvers is a thing.
    - ssivark 7y ago
      Pointers please; not just off-hand mentions leaving others to scratch their heads about what you mean! :-)
  - skdotdan 7y ago
    https://arxiv.org/abs/1905.12149 https://arxiv.org/abs/1905.12149
- brundolf 7y ago
  My company is one of the only ones still pushing on symbolic AI: https://www.cyc.com/ https://www.cyc.com/
gdubs 7y ago
So, I’ve been reading articles on this and I think I have a fuzzy idea of some of these solutions would entail. But what I’m hung up on is this: if Deep learning is about coming up with solutions to problems that are too hard for humans, how do we hope to understand the rationale behind whatever solutions the machine comes up with?
- ben_w 7y ago
  Sometimes it’s easier to understand a solution then it is to produce the solution in the first place. I don’t need to be Newton or Leibniz to understand calculus. That said, in general I don’t expect that we could understand any particular solution produced by an AI, be it deep or otherwise, but I do expect it to be possible quite often.
- dreamcompiler 7y ago
  A lot of DL is about teaching computers to solve problems that are easy for humans (like driving and recognizing your grandmother) but for which humans have a tough time explaining how they do it. The holy grail of neural nets has always been to build a simulation of the brain, figure out how it works, and apply that knowledge to how the human brain might work. We're not there yet but progress has been made. Eventually we'll understand NNs well enough to explain not only themselves but also human brains. In any case we have no choice because we cannot deploy NNs in life critical situations until we understand how they work, because that's the only way to understand how they fail.
  - thelazydogsback 7y ago
    > The holy grail of neural nets has always been to build a simulation of the brain, figure out how it works, and apply that knowledge to how the human brain might work. I'd say that's a goal for some people -- for those whose goal is to figure out how the brain works, rather than constructing a more ideal and powerful GI. Remember the brain is great at some things, but laughable at others -- such as a "7 +/- 2" items in short term memory, inability to immediately retain rote knowledge after one instance and in great numbers, etc. It's the merging of the fuzzy, goal-directed behavior of the mind, in conjunction with its ability to effect the "real world", and the super-human memory and computational capabilities of computers that makes possible future GAIs that are so powerful and possibly scary.
samstave 7y ago
I have a crazy idea: Make a system that allows one to scan their face, and OPT-OUT OF ALL FACIAL RECOGNITION.
- jayd16 7y ago
  But then that system would have to delete your face too! Great, now the only solution left for the robots is to kill all humans...
XuMiao 7y ago
One key question is whether symbolic AI is the right model of the world. It underperforms vector based AI on many specific tasks. But human experts heavily reply on it to communicate with each other. If symbolic AI is not the right model, P vs NP problem might be just irrelevant. Human philosophy is full of craps. We will lose a lot of beliefs. Elon will be right, we will abandon human languages, and connect through a cable in the brain. Everyone will relearn every thing from NN. If symbolic AI is the right model, but difficult to build algorithmically. Vector based model just help to make it faster and better. Then we humans are fine. We simply proxy the lower level optimization to AI. Our functionalities will be shifted just like what happened when engine was invented hundreds of years ago.
- jkp56 7y ago
  Of course symbolic AI is the way to go. We communicate with text messages that consist of words that we internally convert to word2vec style vectors to detect similar words. One more thing we do in our heads: we build a graph of those word2vec symbols. When I read in a book "a cat is sleeping on a tree" I instantly build a small graph where nodes Cat and Tree are connected with an edge labeled Sleeps. I may visualize this as a picture, but that's not necessary for AI. In fact, some people can't visualize anything, but they can definitely think. How? They can still build this knowledge graph. I believe that this graph representation is the limit of our intelligence: there are many facts out there that we can't possibly think about because they don't fit this graph model. It's like the set of real numbers can't be squeezed into the set of rational numbers: most of the numbers are irrational.
  - mbrock 7y ago
    Does one’s knowledge of how to ride a bike have anything to do with such graph structures? Or is intelligence unrelated to such skills? It seems that the intelligence involved in having a basic conversation would engage a lot of such skills, even just social tact.
    - jkp56 7y ago
      I think intelligence isn't involved in riding a bike. Understanding the theory about riding a bike is a different story, and that understanding is a knowledge graph.
jayd16 7y ago
Has work been done to formally prove general AI can't arise from deep learning? I can't help but feel its an assumption being made by those that prefer classical research.
- celeritascelery 7y ago
  First you need to find a formal definition for “general AI”.
  - jamiek88 7y ago
    Which get you into defining consciousness too. Sticky wicket.
  - jayd16 7y ago
    So I take it the answer is no. It seems like a lot of the discussion around what deep learning is not makes a lot of assumptions about what "actual" learning is but doesn't seem to back that up in any formal way.
- inimino 7y ago
  It is trivially proved that deep learning can represent any computable function so the proof you're asking for is not going to be possible. However, it's also completely obvious to anyone familiar with deep learning as it exists today that it is not moving in the direction of AGI and none of the research is ever going to lead to AGI short of some kind of miracle. The burden of proof here would be on those who think this is possible to give some plausible mechanism (or research agenda) for getting there.
  - jayd16 7y ago
    I'm no expert but it doesn't seem very far fetched to me. If we can use deep learning to create a digital assistant, computer vision, and navigation/traversal then we're not very far off from something surpassing a dog's level of intelligence even with what exists today. If that's possible then it seems plausible it could continue onward. Not saying its obviously possible but it doesn't seem obviously impossible and its a mistake to assume as such.
Iwan-Zotow 7y ago
We cannot trust NI (natural intelligence) systems built on mom-and-pop having sex alone
6gvONxR4sf7o 7y ago
I'm coming to suspect that even our data isn't enough for useful AI. Imagine you had a truly general sci-fi AI at your office. It still couldn't just look at your database and answer a simple question like "What was the difference in client churn rates between mobile and desktop last month?" or "What was the effect of experiment 1234 on per-client revenue?" Hell, a human couldn't do it. As far as the human or AI would know, you just presented it with a bunch of random tables. This matters because it's incredibly helpful to know which pieces are randomized. Which rows are repeated measurements as opposed to independent measurements. Which pieces are upstream of which others. There's so much domain knowledge baked into data, while we just expect an algorithm to learn from a simple table of floats. The human state of the art solution seems to be going on slack and asking questions about the data provenance, which will decidedly not work for an automated approach. A primary reason I can do better a better job than a generic algorithm is because you told me where the data came from (or I designed the schema and ETL myself), while the algo can't make any useful assumptions because all that info is hidden.
- eli_gottlieb 7y ago
  I mean, yeah, you can't do causal inference without a causal model. Uh, what's new about that?
  - 6gvONxR4sf7o 7y ago
    The point is that data provenance isn't encoded in the data. Like the difference between a schema where column X has type `int` versus X having type `do(int)` or something cleverer. If the way you get your causal model is to ask the person who ran the experiment, then it's very much an uphill battle for an algorithm to get a causal model. We want to enable automated causal inference, so we should better record our causal models (data lineage).
- allworknoplay 7y ago
  This is a fantastic point that may not be relevant now, but may be extraordinarily relevant in the future. I’m not aware of any work or even sci-fi that addresses AGI with regards to this question, and would be curious if there’s stuff out there?
fuguza 7y ago
Why in earth would you trust AI alone ? Same like driving my Tesla in deep sleep. So many times is been close encounters with obstacles..
aledalgrande 7y ago
Very curious about classical AI mixed with deep learning. Does anyone know of any examples?
brundolf 7y ago
A classical AI company that's working on "a different part of the brain": https://www.cyc.com/ https://www.cyc.com/
Mathnerd314 7y ago
> General AI also ought to be able to work just as comfortably reasoning about politics as reasoning about medicine. It’s the analogue of what people have; any reasonably bright person can do many, many different things. The average human has extreme difficulty reasoning about politics, while usually being reasonable on medicine (anti-vax being one of many exceptions). And it seems strange to expect a skilled pianist to also be a skilled neuroscientist or a skilled construction worker. On the other hand these people all use similar neural architectures (brains). So he seems pretty off-track when he criticizes "narrow AI" in favor of "general AI", as if there's some magic AI that will do everything perfectly, and even more off track when he criticizes researchers for using "one-size-fits-all" technologies, when indeed that is exactly what humans have been doing for millennia for their cognitive needs. And sure, ML models in publications so far are typically one-off things that react poorly to modified inputs or unexpected situations. But it's not clear this has any relevance to commercial use. Tesla is still selling self-driving cars despite the accidents.
- inimino 7y ago
  > some magic AI that will do everything perfectly Total straw man. He actually uses an intern as an example in the very next sentence after what you quoted, as you would expect them to be able to read and get up to speed on a new area regardless of what it was. Meanwhile SOTA in NLP is a system that can be built to answer a single kind of question but can't explain why it did so or do anything useful if given an explanation of why its answer was wrong.
  - Mathnerd314 7y ago
    There are deep models like BERT that do pre-training and then need minimal training to do multiple tasks such as question answering, entailment, sentiment analysis, etc. I don't know about "explaining" an answer but there are debuggers that find errors in data sets: https://arxiv.org/pdf/1603.07292.pdf https://arxiv.org/pdf/1603.07292.pdf. But as I said, I don't see why an artist would suddenly get up to speed as a construction worker. He seems to overestimate the capacity of interns as well.
mindgam3 7y ago
> AlphaGo can play very well on a 19x19 board but actually has to be retrained to play on a rectangular board. This right here is the soft underbelly of the entire “machine learning as step towards AGI” hype machine, fueled in no small part by DeepMind and its flashy but misleading demos. Once a human learns chess, you can give it a 10x10 board and she will perform at nearly the same skill level with zero retraining. Give the same challenge to DeepMind’s “superhuman” game-playing machine and it will be an absolute patzer. This is an obvious indicator that the state of the art in so-called “machine learning” doesn’t involve any actual learning in the way it is normally applied to intelligent systems like humans or animals. I am continually amazed by the failure of otherwise exceedingly intelligent tech people to grasp this problem.
- jayd16 7y ago
  >Once a human learns chess, you can give it a 10x10 board and she will perform at nearly the same skill level with zero retraining. Interesting. Has this actually been shown? I would assume a lot of the strategies a human is familiar with would fall apart as well. I'm no chess or go player but I would have to learn new strategies in a tic-tac-toe game scaled to 10x10. I would certainly not be as proficient although I would still consider myself to have intelligence.
  - mindgam3 7y ago
    Almost all of the human strategies and concepts would still apply: center control, square control, development, initiative, king safety, the opposition, etc. The only exceptions would be fringe concepts like opening theory (already moot in Chess960) and endgame edge cases. If you’re still not convinced, I’ll prove that skills transfer by playing bullet against anyone who can make a 10x10 variant playable online. [Edited to come across less egotistical]
    - csande17 7y ago
      > make a 10x10 variant playable online I'm not saying I'm definitely going to do this, but is there a rulebook somewhere for 10x10 chess? (What are the initial piece positions, and how would castling work?)
The_rationalist 7y ago
I always found it fascinatingly ironic that AGI research only fund deep learning which is a local minimum. People really interested in AGI should better look at Cyc and opencog
1e-9 7y ago
I haven't read the book, but the viewpoints he expresses in the interview are spot-on. DL can a great alert/suggestion mechanism in narrow domains, but it should never be trusted to make critical decisions. I believe that general intelligence will only be achieved through major advancements in general symbolic reasoning. Something like DL might play a small role in this breakthrough, but it will not be a core part of the solution.
- hadsed 7y ago
  Vectors may be the proper way to represent symbols as they are the link with perception. I would call that a pretty core part of AI.
gautamcgoel 7y ago
Gary Marcus literally gave a talk about this last week in my department's ML seminar. I asked him how sure he was that humanity would eventually achieve AGI and he said 100%. When I asked him when that would be he replied 30-100 years. Interesting perspective.
known 7y ago
AI cannot make Emotional decisions which are sometimes good e.g. https://en.wikipedia.org/wiki/Invisible_hand https://en.wikipedia.org/wiki/Invisible_hand

205 points by laurex 7y ago

The opposing argument, by Rich Sutton, Distinguished Research Scientist at DeepMind: http://www.incompleteideas.net/IncIdeas/BitterLesson.html http://www.incompleteideas.net/IncIdeas/BitterLesson.html

justicezyx 7y ago
Seems talking about different issues
cjauvin 7y ago
I would say this is also an opposing argument: https://twitter.com/ylecun/status/1066568396177842176 https://twitter.com/ylecun/status/1066568396177842176 i.e. gradient-based learning is the final word on the matter.
- YeGoblynQueenne 7y ago
  Yes, gradient optimisation always gets the final word. But what does that word, well, say? _ / \ ,----------------------------. | Hey! I found your minimum! | '------------ ------------' / .----. \ / / \ / / \ \/ / \ / / \ @ / \ / / `'--' \ / / \ / / \ / / \ / / \ .-. / / \_/ `\ / \.__.' \ / \ / `;._.-'
Barrin92 7y ago
there are some surprisingly weak arguments in the text. It's correct to not treat computational resources as constant, but ot treat them as unimportant or negligible is awful as well. Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. If the goal is AGI this might get exponentially worse. Intelligence needs to take resource consumption into account. The models we produce aren't even close to high level reasoning and we're already consuming significantly more energy than humans or animals, something is wrong. The scale argument isn't great either because deep learning is running into the inverse issue of classical AI. Now instead of having to program all logic explicitly we have to formulate every individual problem as training data. This doesn't scale either. If an AI gets attacked by a wild animal the solution can't be to first produce 10k pictures of mauled victims, intelligence includes to reason about things in the abscence of data. We can't have autonomous cars constantly running into things until we provide huge amounts of data for every problem, this does not scale either.
- roenxi 7y ago
  > Already computational resources are becoming prohibitive with only a few institutions producing state of the art models at high financial cost. That is something of an illusion. Obviously there will be some sort of uneven distribution of computing power; some institutions will have more, some less. The institutions with more power will create models at the limit of what they can do, because that is the best use of their power. So if the thesis of more power = more results holds then truly cutting results will always be by people with resources that are practically unattainable by everyone else. Google's AlphaGo wasn't a particularly clever model, for example. It just had a lot of horsepower behind it to train it and the various ranging shot attempts Deepmind would have gone through. Someone else would have figured it out albeit more slowly in a few years as computing power became available. Computational power is still getting exponentially more affordable [0]. Costs aren't really rising, so much as the people who have spent more money get a few years ahead of everyone else and can preview what is about to become cheap. [0] https://aiimpacts.org/recent-trend-in-the-cost-of-computing/ https://aiimpacts.org/recent-trend-in-the-cost-of-computing/
  - 7y ago
deleted 7y ago
[deleted]
6gvONxR4sf7o 7y ago
This argument completely ignores anything statistical, which is another limiting factor. It reminds me of the difference in bandit research and full RL research. RL researchers are fine throwing a thousand years of experience at their algorithm, because it can be simulated. Meanwhile people using bandits in the real world care about statistical efficiency (learning lots with little data), and it's reflected in the research. Most decisions aren't made with a huge abundance of data (most of us aren't google or facebook).

sebastianconcpt 7y ago

In cognitive science we talk about having cognitive models of things. So I’m sitting in a hotel room, and I understand that there’s a closet, there’s a bed, there’s the television that’s mounted in an unusual way. I know that there are all these things here, and I don’t just identify them. I also understand how they relate to one another. I have these ideas about how the outside world works. They’re not perfect. They’re fallible, but they’re pretty good. And I make a lot of inferences around them to guide my everyday actions. The opposite extreme is something like the Atari game system that DeepMind made, where it memorized what it needed to do as it saw pixels in particular places on the screen. If you get enough data, it can look like you’ve got understanding, but it’s actually a very shallow understanding. The proof is if you shift things by three pixels, it plays much more poorly. It breaks with the change. That’s the opposite of deep understanding. Of course. There are an infinte way to make interpretations of perceptions and a finite subset of possible valid ones. It's among those possible, that the AI will be a concrete implementation of an ideology. To select which one is always done by humans.

mark_l_watson 7y ago

I started reading Rebooting AI last night. I think that Marcus and Davis (so far in the book) take a reasonable approach by wanting to design robust AI. Robust AI requires general real world intelligence that is not provided by deep learning. I have earned over 90% of my income over the last five or six years as a deep learning practitioner. I am a fan of DL based on great results for perception tasks as well as solid NLP results like using BERT like models for things like anaphora resolution. But, I am in agreement with Marcus and Davis that our long term research priorities are wrong.

turk73 7y ago
Making 90% of your income off of this tech over the last n years is different than that tech being successful. I work at a very large company that is trying to use ML and AI in all kinds of places. The trend I am seeing is that most of that effort is falling flat, really flat, in fact. They have success in places where regular algorithms would also succeed, but just having regular developers design matching systems and do pretty basic statistics isn't sexy in terms of marketing, so they hush it all up and pretend that ML and neural nets and things are the only way forwards. I don't think it is. Our problem is that we have too many ETL robots who aren't very intelligent people and not very forward-thinking themselves! The cutting edge NLP stuff just showcased at my company was pretty lame, too. I barely saw any statistically significant results at all and yet they rather unscientifically proclaim success because they got any effect at all. Some of what we do in our field doesn't matter because it comes down to whether a customer got a 2nd call back and got converted to some minor sale or added to a program for them. It's throw away and creates good will at conferences and talks. We make a big deal out of it. We are spending hundreds of millions on projects, trying to save money on generating leads, reducing interactions with customers and vendors through staffed phone banks, and so on. My company has hired all kinds of academics and research type people and has given them titles of "Distinguished this" and "Principal that" and honestly there's not that much to show for it, maybe zero direct outcomes so far. What galls me the most is in all the conferences and demos they are showing off things like High School robotics vehicles and AI parlor tricks and astonishingly little has translated into the business we do. Meanwhile, there are people in the company who do know how to reduce costs and get more done and have outstanding outcomes, but their techniques are not sexy and thus unimportant to the PT Barnum MBAs running our company. I'm sure that's true most everywhere, of course. These Principals and Distinguisheds all keep proclaiming success while cashing fat paychecks. Meanwhile this year, our stock has had a tough go of it, so I'm curious whether these attempts will continue. The market takes no prisoners. Sure we get a lot of mileage out of looking cool for the recent grad crowd purposes of recruiting--kids want sexy, cool tech projects to work on and words like "insurance" turn them off, so there's that, I guess. My take on that is that it won't be long before all those new recruits will figure out they got bait and switched pretty bad and that they aren't going to get to work on any of this sexy ML and AI stuff anymore than I am in my role. I got lured in by Data Science (because PhD), which just shows how gullible I am, but at least some of that traditional statistical modeling is having an impact here and there. The problem again is that even that is overblown by a couple of orders of magnitude! In my project, we're simply trying to get more real-time data out to people who need it without having to call in to get it and that is ridiculously difficult because of all the systems we try to knit together and how overall terrible our data quality is. And now my boss wants to build out an "analytics engine" to capture some of this sexy ML and AI stuff. It leads me to believe that the people involved are most interested in getting promoted and not much more. Anyways, it is cool tech, but American taxpayers and people who are forced to buy our products are paying for it and I rather think they would prefer to spend their money in some better fashion.

keithyjohnson 7y ago

Understanding a sentence is fundamentally different from recognizing an object. But people are trying to use deep learning to do both. I agree with most of the article but I think this^^ skips over the different types of networks used to solve perception and language problems. A CNN is very different from say, word2vec, which isn't a very deep network at all.

mlthoughts2018 7y ago
I’d go further and say that deep networks are excellent for sentence understanding, and various types of RNN or 1D convolutional layers are very good at this in specialized domains just as CNNs and ResNets are good in specialized vision applications. It absolutely makes sense to use deep learning for both of these tasks. In fact, one very effective thing to do is to use a Siamese network to learn joint representational spaces of text and imagery in the same network. It’s really specious and disingenuous to say “boy, vision and language sure seem different but can you believe these DL researchers are using the same tools for both!?”
- mattmcknight 7y ago
  or... "vision and language sure seem different, can you believe that networks of neurons in the brain do both?"
jotakami 7y ago
Perhaps the difference is in the nature of the information that is being probed and its larger context? Visual imagery often provides almost all of its own context, but the “meaning” of a sentence can be radically different depending upon its source. Humans produce words, so you almost need a working theory of mind to fully understand them. None of that context will ever make it into word2vec.
- jkp56 7y ago
  Why not? It's true that the word "space" has different meanings when in appears in a math book, CS book or astronomy book. But we just have 3 different word2vec models. When I read something about math, I pick the math word2vec model and there "space" appears close to words "Hilbert" and "separable", while in the CS model, the same word is next to "complexity" and "memory". As I read more, I improve my word2vec models, but never mix them together. Now what happens if I'm reading something and don't understand the context? No, I don't switch to some general word2vec model. I rather try to guess which model to use and then reread the same text using that model.

mlthoughts2018 7y ago

This article doesn’t have any substance. It’s full of anecdata like shifting by 3 pixels to mess up a video game AI or some vague nonsense about “a model of this chair or this tv mounted to the wall.” It’s all casual hypotheticals. There’s plenty of research on Bayesian neural networks for causal inference. But even more, a lot of causal inference problems are “small data” problems where choosing a strongly informative prior to pair with simple models is needed to prevent overfitting and poor generalization and to account for domain expertise. Deep learning practitioners generally know plenty about this stuff and fully understand that deep neural networks are just one tool in the tool box, not applicable to all problems and certainly not approaching any kind of general AI solution that supersedes causal inference, feature engineering, etc. This article is just a sensationalist hit job trying to capitalize on public anxieties about AI to raise the profile of this academic and try to sell more copies of his book. I’d say, let’s not waste time on this crap. There are engineering problems that deep learning allows us to safely & reliably solve where other methods never could. We absolutely can trust these models for specific use cases. Let’s just get on with doing the work.

lngnmn1 7y ago

Why, yes. All life has the environmental constraints builtin in the structure of a brain and CNS. Brain also has implicit sets of filters in sensory channels and in pattern-marching machinery. Constraints and filters are the must have.

deleted 7y ago

[deleted]

wodenokoto 7y ago

In this context, what is classical AI? An SVM? A markov model? A large context free grammar with a dictionary?

dreamcompiler 7y ago
Yes. Or rules, logic, or other symbolic system. AI has always been divided into two camps: Connectionist and not. Current "AI" is all connectionist. What we're now calling "Classical AI" is the non-connectionist kind that was prominent in the 60s-80s but fell out of favor in the AI Winter.
RobertDeNiro 7y ago
I would say OR (operations research) is really what's meant here.

thelazydogsback 7y ago

I've been waiting for the Symbolic/NN pendulum to starting swinging back the other way and start settling in the center. NN/DL is great for the interface between the outer world and the inner world of the mind (pattern recognition and re-construction), and symbolic AI more straightforwardly represents more "language of the mind" tasks, and easily handles issues like explanation and other meta-behaviors that with DL is difficult due to its black-box nature. DL's reliance on extension/training vs. intention/rules can develop ad-hoc intentional emergent theories which is their strength but also their weakness as these theories may not be correct or complete. Each can be brittle in their own way - so it'll be interesting to see more cross-pollination.

uoaei 7y ago
SAT solvers are really fast now. Some sort of "neural SAT problem definition" followed by solving it seems to be an interesting direction, but I'm relatively naive on it all. Not sure how training would work since there's no backprop through Boolean logic.
- w_t_payne 7y ago
  Differentiable SAT solvers is a thing.
  - ssivark 7y ago
    Pointers please; not just off-hand mentions leaving others to scratch their heads about what you mean! :-)
- skdotdan 7y ago
  https://arxiv.org/abs/1905.12149 https://arxiv.org/abs/1905.12149
brundolf 7y ago
My company is one of the only ones still pushing on symbolic AI: https://www.cyc.com/ https://www.cyc.com/

gdubs 7y ago

So, I’ve been reading articles on this and I think I have a fuzzy idea of some of these solutions would entail. But what I’m hung up on is this: if Deep learning is about coming up with solutions to problems that are too hard for humans, how do we hope to understand the rationale behind whatever solutions the machine comes up with?

ben_w 7y ago
Sometimes it’s easier to understand a solution then it is to produce the solution in the first place. I don’t need to be Newton or Leibniz to understand calculus. That said, in general I don’t expect that we could understand any particular solution produced by an AI, be it deep or otherwise, but I do expect it to be possible quite often.
dreamcompiler 7y ago
A lot of DL is about teaching computers to solve problems that are easy for humans (like driving and recognizing your grandmother) but for which humans have a tough time explaining how they do it. The holy grail of neural nets has always been to build a simulation of the brain, figure out how it works, and apply that knowledge to how the human brain might work. We're not there yet but progress has been made. Eventually we'll understand NNs well enough to explain not only themselves but also human brains. In any case we have no choice because we cannot deploy NNs in life critical situations until we understand how they work, because that's the only way to understand how they fail.
- thelazydogsback 7y ago
  > The holy grail of neural nets has always been to build a simulation of the brain, figure out how it works, and apply that knowledge to how the human brain might work. I'd say that's a goal for some people -- for those whose goal is to figure out how the brain works, rather than constructing a more ideal and powerful GI. Remember the brain is great at some things, but laughable at others -- such as a "7 +/- 2" items in short term memory, inability to immediately retain rote knowledge after one instance and in great numbers, etc. It's the merging of the fuzzy, goal-directed behavior of the mind, in conjunction with its ability to effect the "real world", and the super-human memory and computational capabilities of computers that makes possible future GAIs that are so powerful and possibly scary.

samstave 7y ago

I have a crazy idea: Make a system that allows one to scan their face, and OPT-OUT OF ALL FACIAL RECOGNITION.

jayd16 7y ago
But then that system would have to delete your face too! Great, now the only solution left for the robots is to kill all humans...

XuMiao 7y ago

One key question is whether symbolic AI is the right model of the world. It underperforms vector based AI on many specific tasks. But human experts heavily reply on it to communicate with each other. If symbolic AI is not the right model, P vs NP problem might be just irrelevant. Human philosophy is full of craps. We will lose a lot of beliefs. Elon will be right, we will abandon human languages, and connect through a cable in the brain. Everyone will relearn every thing from NN. If symbolic AI is the right model, but difficult to build algorithmically. Vector based model just help to make it faster and better. Then we humans are fine. We simply proxy the lower level optimization to AI. Our functionalities will be shifted just like what happened when engine was invented hundreds of years ago.

jkp56 7y ago
Of course symbolic AI is the way to go. We communicate with text messages that consist of words that we internally convert to word2vec style vectors to detect similar words. One more thing we do in our heads: we build a graph of those word2vec symbols. When I read in a book "a cat is sleeping on a tree" I instantly build a small graph where nodes Cat and Tree are connected with an edge labeled Sleeps. I may visualize this as a picture, but that's not necessary for AI. In fact, some people can't visualize anything, but they can definitely think. How? They can still build this knowledge graph. I believe that this graph representation is the limit of our intelligence: there are many facts out there that we can't possibly think about because they don't fit this graph model. It's like the set of real numbers can't be squeezed into the set of rational numbers: most of the numbers are irrational.
- mbrock 7y ago
  Does one’s knowledge of how to ride a bike have anything to do with such graph structures? Or is intelligence unrelated to such skills? It seems that the intelligence involved in having a basic conversation would engage a lot of such skills, even just social tact.
  - jkp56 7y ago
    I think intelligence isn't involved in riding a bike. Understanding the theory about riding a bike is a different story, and that understanding is a knowledge graph.

jayd16 7y ago

Has work been done to formally prove general AI can't arise from deep learning? I can't help but feel its an assumption being made by those that prefer classical research.

celeritascelery 7y ago
First you need to find a formal definition for “general AI”.
- jamiek88 7y ago
  Which get you into defining consciousness too. Sticky wicket.
- jayd16 7y ago
  So I take it the answer is no. It seems like a lot of the discussion around what deep learning is not makes a lot of assumptions about what "actual" learning is but doesn't seem to back that up in any formal way.
inimino 7y ago
It is trivially proved that deep learning can represent any computable function so the proof you're asking for is not going to be possible. However, it's also completely obvious to anyone familiar with deep learning as it exists today that it is not moving in the direction of AGI and none of the research is ever going to lead to AGI short of some kind of miracle. The burden of proof here would be on those who think this is possible to give some plausible mechanism (or research agenda) for getting there.
- jayd16 7y ago
  I'm no expert but it doesn't seem very far fetched to me. If we can use deep learning to create a digital assistant, computer vision, and navigation/traversal then we're not very far off from something surpassing a dog's level of intelligence even with what exists today. If that's possible then it seems plausible it could continue onward. Not saying its obviously possible but it doesn't seem obviously impossible and its a mistake to assume as such.

Iwan-Zotow 7y ago

We cannot trust NI (natural intelligence) systems built on mom-and-pop having sex alone

6gvONxR4sf7o 7y ago

I'm coming to suspect that even our data isn't enough for useful AI. Imagine you had a truly general sci-fi AI at your office. It still couldn't just look at your database and answer a simple question like "What was the difference in client churn rates between mobile and desktop last month?" or "What was the effect of experiment 1234 on per-client revenue?" Hell, a human couldn't do it. As far as the human or AI would know, you just presented it with a bunch of random tables. This matters because it's incredibly helpful to know which pieces are randomized. Which rows are repeated measurements as opposed to independent measurements. Which pieces are upstream of which others. There's so much domain knowledge baked into data, while we just expect an algorithm to learn from a simple table of floats. The human state of the art solution seems to be going on slack and asking questions about the data provenance, which will decidedly not work for an automated approach. A primary reason I can do better a better job than a generic algorithm is because you told me where the data came from (or I designed the schema and ETL myself), while the algo can't make any useful assumptions because all that info is hidden.

eli_gottlieb 7y ago
I mean, yeah, you can't do causal inference without a causal model. Uh, what's new about that?
- 6gvONxR4sf7o 7y ago
  The point is that data provenance isn't encoded in the data. Like the difference between a schema where column X has type `int` versus X having type `do(int)` or something cleverer. If the way you get your causal model is to ask the person who ran the experiment, then it's very much an uphill battle for an algorithm to get a causal model. We want to enable automated causal inference, so we should better record our causal models (data lineage).
allworknoplay 7y ago
This is a fantastic point that may not be relevant now, but may be extraordinarily relevant in the future. I’m not aware of any work or even sci-fi that addresses AGI with regards to this question, and would be curious if there’s stuff out there?

fuguza 7y ago

Why in earth would you trust AI alone ? Same like driving my Tesla in deep sleep. So many times is been close encounters with obstacles..

aledalgrande 7y ago

Very curious about classical AI mixed with deep learning. Does anyone know of any examples?

brundolf 7y ago

A classical AI company that's working on "a different part of the brain": https://www.cyc.com/ https://www.cyc.com/

Mathnerd314 7y ago

> General AI also ought to be able to work just as comfortably reasoning about politics as reasoning about medicine. It’s the analogue of what people have; any reasonably bright person can do many, many different things. The average human has extreme difficulty reasoning about politics, while usually being reasonable on medicine (anti-vax being one of many exceptions). And it seems strange to expect a skilled pianist to also be a skilled neuroscientist or a skilled construction worker. On the other hand these people all use similar neural architectures (brains). So he seems pretty off-track when he criticizes "narrow AI" in favor of "general AI", as if there's some magic AI that will do everything perfectly, and even more off track when he criticizes researchers for using "one-size-fits-all" technologies, when indeed that is exactly what humans have been doing for millennia for their cognitive needs. And sure, ML models in publications so far are typically one-off things that react poorly to modified inputs or unexpected situations. But it's not clear this has any relevance to commercial use. Tesla is still selling self-driving cars despite the accidents.

inimino 7y ago
> some magic AI that will do everything perfectly Total straw man. He actually uses an intern as an example in the very next sentence after what you quoted, as you would expect them to be able to read and get up to speed on a new area regardless of what it was. Meanwhile SOTA in NLP is a system that can be built to answer a single kind of question but can't explain why it did so or do anything useful if given an explanation of why its answer was wrong.
- Mathnerd314 7y ago
  There are deep models like BERT that do pre-training and then need minimal training to do multiple tasks such as question answering, entailment, sentiment analysis, etc. I don't know about "explaining" an answer but there are debuggers that find errors in data sets: https://arxiv.org/pdf/1603.07292.pdf https://arxiv.org/pdf/1603.07292.pdf. But as I said, I don't see why an artist would suddenly get up to speed as a construction worker. He seems to overestimate the capacity of interns as well.

mindgam3 7y ago

> AlphaGo can play very well on a 19x19 board but actually has to be retrained to play on a rectangular board. This right here is the soft underbelly of the entire “machine learning as step towards AGI” hype machine, fueled in no small part by DeepMind and its flashy but misleading demos. Once a human learns chess, you can give it a 10x10 board and she will perform at nearly the same skill level with zero retraining. Give the same challenge to DeepMind’s “superhuman” game-playing machine and it will be an absolute patzer. This is an obvious indicator that the state of the art in so-called “machine learning” doesn’t involve any actual learning in the way it is normally applied to intelligent systems like humans or animals. I am continually amazed by the failure of otherwise exceedingly intelligent tech people to grasp this problem.

jayd16 7y ago
>Once a human learns chess, you can give it a 10x10 board and she will perform at nearly the same skill level with zero retraining. Interesting. Has this actually been shown? I would assume a lot of the strategies a human is familiar with would fall apart as well. I'm no chess or go player but I would have to learn new strategies in a tic-tac-toe game scaled to 10x10. I would certainly not be as proficient although I would still consider myself to have intelligence.
- mindgam3 7y ago
  Almost all of the human strategies and concepts would still apply: center control, square control, development, initiative, king safety, the opposition, etc. The only exceptions would be fringe concepts like opening theory (already moot in Chess960) and endgame edge cases. If you’re still not convinced, I’ll prove that skills transfer by playing bullet against anyone who can make a 10x10 variant playable online. [Edited to come across less egotistical]
  - csande17 7y ago
    > make a 10x10 variant playable online I'm not saying I'm definitely going to do this, but is there a rulebook somewhere for 10x10 chess? (What are the initial piece positions, and how would castling work?)

The_rationalist 7y ago

I always found it fascinatingly ironic that AGI research only fund deep learning which is a local minimum. People really interested in AGI should better look at Cyc and opencog

1e-9 7y ago

I haven't read the book, but the viewpoints he expresses in the interview are spot-on. DL can a great alert/suggestion mechanism in narrow domains, but it should never be trusted to make critical decisions. I believe that general intelligence will only be achieved through major advancements in general symbolic reasoning. Something like DL might play a small role in this breakthrough, but it will not be a core part of the solution.

hadsed 7y ago
Vectors may be the proper way to represent symbols as they are the link with perception. I would call that a pretty core part of AI.

gautamcgoel 7y ago

Gary Marcus literally gave a talk about this last week in my department's ML seminar. I asked him how sure he was that humanity would eventually achieve AGI and he said 100%. When I asked him when that would be he replied 30-100 years. Interesting perspective.

known 7y ago

AI cannot make Emotional decisions which are sometimes good e.g. https://en.wikipedia.org/wiki/Invisible_hand https://en.wikipedia.org/wiki/Invisible_hand