11 ms·

It surprises many non techies I talk to (and techies too) to learn that AI hasn't really come very far in a few decades. We're a bit faster (by orders of magnit

It surprises many non techies I talk to (and techies too) to learn that AI hasn't really come very far in a few decades. We're a bit faster (by orders of magnitude to be sure), but AI "summers" are characterized primarily by it being a bit cheaper to put old limited things in products, and not by some abstract march of tech progress. Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out.

gambiting 8y ago
In the 60s, there was a rather famous project where scientists thought that solving image recognition will take them few months at most. It's 2018 and the best algorithms on the planet will answer with 99% confidence that a sofa in a zebra print is in fact, a zebra, unless they were very specifically trained against this scenario. I feel this exact same thing is happening with autonomous cars - yes, it's possible to get them to be extremely good at recognising the road and surroundings - but the last few percent, those crucial few percent that make the technology actually usable, I don't see those happening for another 50 years at least.
- nostrademons 8y ago
  My unpopular (to me, even) prediction is that wartime will be the catalyst that makes self-driving cars ubiquitous. In times of peace having a car that kills its occupant even 1/10,000 of the time is unacceptable. In times of war having a car (or more likely, truck/tanker/tank) with no occupant is a huge competitive advantage, because someone is actively trying to kill the occupant. Build something with even a 90% chance of crashing and you still win, because you can take people off the battlefield entirely while your opponent loses precious soldiers with every vehicle that's destroyed. Then after the war, people's risk tolerances get reset because quibbling over a 1/10,000 chance of death seems ridiculous when people have been actively trying to kill you for the last 5 years and a countable percentage of your friends are now dead. The way global politics is going we probably don't even have to wait 10 years for this.
  - zazen 8y ago
    You seem to have a war like WW2 in mind. But a war between big powers in the 21st century couldn't drag on for five years while you frantically work on new technology. Even without use of nukes, one side or the other is going to be flattened a lot faster than in the days when bombers used propellers. Now, a new Cold War, that might push technological competition, although you won't get that "reset" of people's attitudes that you're after.
    - reitzensteinm 8y ago
      It sounds more like he has a war like Iraq and Afghanistan in mind. Imagine resupply convoys being driven autonomously, capable of launching drones for defense. https://www.army-technology.com/features/feature77200/ https://www.army-technology.com/features/feature77200/ "[...] every 55,702 barrels of fuel burned in Afghanistan by the US military forces corresponded to one casualty."
      - Cthulhu_ 8y ago
        I've seen a number of projects already for self-driving supply trucks, and even walkers from Boston Dynamics. There's a (probably highly scripted) one featured in a later episode of Top Gear, for example.
      - zazen 8y ago
        Well, you probably won't have to wait 10 years for another non-peer war like that. I think he's envisaging a big peer war with mass mobilization ala WW1/WW2, but he imagines such a war would be like a big Iraq/Afghanistan. It wouldn't.
        larkeith 8y ago
        It seems like it might be feasible for a proxy war (like the one in Syria) to escalate significantly, to the point where multiple major powers are contributing manpower and materiel directly to the front lines; In such a case, it may be possible for an extended war to take place, while still incentivizing deployment of autonomous weaponized platforms - which are already in development today [1]. [1] https://en.wikipedia.org/wiki/Lethal_autonomous_weapon https://en.wikipedia.org/wiki/Lethal_autonomous_weapon
- denzil_correa 8y ago
  > In the 60s, there was a rather famous project where scientists thought that solving image recognition will take them few months at most. What project was this?
  - tome 8y ago
    Probably a reference to this https://en.wikipedia.org/wiki/Computer_vision#History https://en.wikipedia.org/wiki/Computer_vision#History
  - code_duck 8y ago
    https://www.explainxkcd.com/wiki/index.php/1425:_Tasks https://www.explainxkcd.com/wiki/index.php/1425:_Tasks
  - FabHK 8y ago
    I thought it might have been a reference to (LISP inventor) McCarthy's famous Dartmouth Conference in 1956, where they thought that 10 people over 2 months could make "significant advances" in making machines "use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves". https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop EDIT: Probably the Summer Vision Project, as Tome said in sibling comment. https://dspace.mit.edu/handle/1721.1/6125 https://dspace.mit.edu/handle/1721.1/6125
- dalbasal 8y ago
  To me, a fundamental question is "is this a problem?" Humans have similar problems too. Our intelligence is trained by experience and evolution to operate within certain parameters. More concerning to me is the fact that a human can conceptualize "couch" from a single example. ML algorithims needs to see thousands of couches before they can classify them.
  - deleted 8y ago
    [deleted]
  - ksdale 8y ago
    Preface: I didn't intend for this to be so long... I just got on a roll... I'm not sure it's true that we conceptualize couch from a single example. We've all seen thousands of couches in many different contexts, homes, schools, doctor's offices, on TV. If you had a person who had never seen a couch or a zebra before, and all you could tell them was right or wrong, it would probably take them (a lot?) more than a single try before they could distinguish between couch and zebra without fail. The only reason it's easy for us is that we have this giant scaffolding built around zebras as living things that look like a horses and donkeys, and couches as inanimate objects that look like things people sit on and regularly have some pattern on them. I think people tend to underestimate just how much "training" in the ML sense goes into a human brain. After all, humans spend the first... decade? of life incapable of all but the simplest tasks. That's a decade in which our brains are consuming petabytes of information and processing it constantly. Watching my children grow up, the way they learn seems remarkably similar to the way computers learn. If you watch a baby learn to move, it is purely an exercise in going too far in one direction and then too far in the other direction, repeated for basically years until they're coordinated enough to move roughly like an adult by the time they're 3 or so. It's the same with words and concepts too, they're just guessing based on things they already know (and the guesses are often waaaay off, because they don't yet know much), but they're constantly filing things away into their frameworks until their frameworks get big enough that this too resembles the way adults learn. By the time we're adults, the human brain makes ML algorithms look pathetic, but that doesn't take into the account the decades long head start that our brains got.
azernik 8y ago
> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. On the other hand, the article notes that today's AI Summer gets its funding from a constant stream of profitable applications (though less dramatic than the blue-sky hype). The field is less reliant on animal spirits of academia, and has a more reliable source of funding. I still think things like self-driving cars are far in the future, but we're at a point where ongoing research into e.g. machine vision can be funded by less safety-critical commercial applications.
- romaniv 8y ago
  Expert systems had industry funding in the 80s. And contrary to what some people here keep implying, they delivered results. Some of them are still used right now.
2sk21 8y ago
I completed my PhD in the subject of improving back propagation - in 1992! Many people are startled when I tell them how old the idea of NNs are :-)
- YorkshireSeason 8y ago
  It's not widely known, but Alan Turing tinkered with neural nets in 1948, see [1], with his boss at the time calling Turing's work a "schoolboy essay". For a history of the subject, see Section 5.1 of [2]. Note that some parts of [2] are rather controversial. For a historical account of back-progagation, which itself was 'invented' many times, see [3, 4]. It is also not widely known that regular expressions, one of programmers' favourite tools, were introduced by Kleene in [5]: he was interested in characterising the behaviour of McCulloch-Pitts "nerve nets" (early versions of neural nets) and finite automata. [1] http://www.alanturing.net/turing_archive/pages/reference%20articles/connectionism/Turing's%20neural%20networks.html http://www.alanturing.net/turing_archive/pages/reference%20a... [2] https://arxiv.org/abs/1404.7828 https://arxiv.org/abs/1404.7828 [3] https://www.math.uni-bielefeld.de/documenta/vol-ismp/52_griewank-andreas-b.pdf https://www.math.uni-bielefeld.de/documenta/vol-ismp/52_grie... [4] http://people.idsia.ch/~juergen/who-invented-backpropagation.html http://people.idsia.ch/~juergen/who-invented-backpropagation... [5] S. Kleene, Representation of events in nerve nets and finite automata, see https://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM704.pdf https://www.rand.org/content/dam/rand/pubs/research_memorand...
  - FabHK 8y ago
    Great references (though I can't open [4]). I sometimes think the contributions of numerical analysis pioneers is undervalued (eg Wilkinson, Kahan, Golub). Backpropagation is really just reverse mode AD, and has been discovered many times, as you point out. https://en.wikipedia.org/wiki/Automatic_differentiation https://en.wikipedia.org/wiki/Automatic_differentiation EDIT: [4] works now.
- s-macke 8y ago
  Show them this video: https://www.youtube.com/watch?v=cNxadbrN_aI https://www.youtube.com/watch?v=cNxadbrN_aI
ahartmetz 8y ago
I like your pessimism. But I'm also optimistic that interesting (mostly non-AI) stuff will happen when Moore's Law isn't a substitute for other innovations anymore. I am especially looking forward to Ray Kurzweil eating his hat when the singularity is delayed indefinitely.
- tim333 8y ago
  You'll have a long wait - I think he projected 2045.
  - ahartmetz 8y ago
    Well crap, I remembered something much closer.
    - tim333 8y ago
      He said Turing test 2029. That may have better hat eating possibilities.
- Pica_soO 8y ago
  Why was it again, that everyone reels in distaste from the church of singularity... Oh, yes... "Oh, powerful deus ex machina yet to come, the one above me is lost, take him into the loving arms of your personality review and do not judge your faithful followers by his threshold." the same reason all other powerful organized religion is distasteful. Rampant Stasi denunciation and a constant attempt to subdue all opposition voices.
vegardx 8y ago
People are often surprised that many planes can land completely autonomously on airports that support autoland (I think they call it ILS CAT 3). And that work on systems like that started in the 60s.
- adrianN 8y ago
  Automating systems to interact with other automatic systems, likes planes and ILS, is a lot easier than automating systems that need to interact with humans.
  - alxlaz 8y ago
    And it's even harder to automate systems that have to derive data from environmental cues. A line-following robot is something simple enough that undergrad students can build, and making it stop if it detects obstacles is a good choice for extra credit. A road-following car that stops when detecting obstacles is way, way, way more complicated than that.
    - adrianN 8y ago
      I don't think stopping for obstacles is the hard part of autonomous driving.
      - reitanqild 8y ago
        It gets harder when you are: * Out of the lab * On roads that aren't level * When the obstacles move * etc
      - rsynnott 8y ago
        It's one surprisingly hard part of it. Note how Teslas enjoy driving into stationary objects with certain properties, say.
        adrianN 8y ago
        It is my understanding that this is due to having the wrong kinds of sensors.
agumonkey 8y ago
To me the major difference is the pressure differential of today's wave. It's a lot larger market than 80s AI, so it may provide more resources to overcome the hurdles. Or maybe it will get into another "comatose" decade.
Shivetya 8y ago
We get bombarded with the term at nearly every turn. We see it in movies, television, and video games. So for the part what researches expect AI to become is not what the public thinks AI is. Marketing has a way of taking something that sounds cool, is just too complex to fully understand, but not so complex that they cannot convince you that you need it, and from there the terminology gets corrupted. Sure most people dismiss most over the top depictions in tv, movies, and games, but it is surprising how many out there still believe governments and the mega rich have something "close".
- ianai 8y ago
  I agree that linear thinking has its faults but I wonder if certain things can’t “add up” to opportunities for breakthroughs. Ie exascale computing, quantum scale Boltzmann machines, and faster communication between cpu cores. I’m wondering what new technologies will be possible after modeling at the scales possible by exascale.
MaxBarraclough 8y ago
"hasn't really come very far in a few decades"? How about AlphaGo Zero?
- romaniv 8y ago
  What about AlphaGo Zero? I'm honestly not sure why a system that plays board games is seen as some kind of giant leap in AI. Seems mostly a result of Google's marketing.
  - MaxBarraclough 8y ago
    You're really going to pretend that cracking Go wasn't a big moment for AI?
    - romaniv 8y ago
      In 1996 IBM made a system that was better than any person at playing chess. Chess has roughly 400 moves possible at any point. In 2016 Google made a system that was better at any person at playing Go. Go has roughly 130,000 moves possible at any point. That's approximately equal to 400 * 2^9. Moore's law states that the number of transistors (which is a proxy for computing power) in CPUs doubles every year. 2016 - 1996 = 20 years of computing power growth. In short, after our computing power has increased roughly by a factor of 2 ^ 20 someone made a system that plays a game that's 2 ^ 9 time more complicated than chess. Why is this seen as some giant, surprising leap forward?
      - knight-of-lambd 8y ago
        Because a chess engine isn't scanning 400 possibilities, and a Go engine isn't scanning 130k (where'd you even get that number?) possible game states. For example, a brute force search of a 19x19 Go board to a depth of 20 would yield on the order of 361^20 = 1.4E51 game states. With a reduction in search depth and better algorithms, state-of-the-art engines might cut this down by ten orders of magnitude, but can still be beaten by rank amateurs. Deepmind's approach to board game engines blows all previous approaches out of the water. The claim that their success is incidental to Moore's law is categorically false.
rsynnott 8y ago
"Ever" is a long time. But yeah, I wouldn't be surprised to see it die down again few a few decades at least.
dalbasal 8y ago
At least to some extent, how you sees this relates to what you want/think AI to eventually be. A while back, I heard a particularly curmudgeonly interview with Noam Chomsky, just commenting on the state of AI and recent AI programs (Watson, Deep blue..). Chomsky basically called all these statsistical AI programs brute force parlour tricks, nothing to do with computer science or artificial intelligences. He stopped just short of calling the whole thing fraud. Real AI are things like bird flocking algorithims (did he work on something like this, can't remember). I think the jist was that to be an AI program, a program must embody a theory of intelligence. The theory-less nature of modern statistical algorithims seemed to really peeve him. maybe he's right. Anyway... I don't remember what point 9if any) i was driving at. I think there are abstract values at play, in how we narrate progress. Since we still don't know what inteligence "is," or even if it is a distinct thing, it's hard to know what the real milestones are. Personally, I like the fully realized, running at scale "proofs," but that probably says more about my ability to understand theory than anything else.
- braythwayt 8y ago
  At some level: Alice: “Computers aren’t doing real intelligence, because they don’t think the way we theorize we think.” Bob: “Computers are doing real intelligence, because we don’t think the way we theorize we think.”
- nkozyra 8y ago
  > Chomsky basically called all these statsistical AI programs brute force parlour tricks, nothing to do with computer science or artificial intelligences. The counterargument is our own brains use brute force parlor tricks to solve problems. We learn by a priori (pre-built models), by experience (reinforcement learning) and viewing and weighing features that we observe against prior experience (NNs/ANNs). The entire point of ML is to produce approximately the same result a human does. Whether it's a parlor trick or a perfect simulation of the brain is irrelevant. There's no advantage to something novel and more "organic" unless it produces better results. This is a common problem I see - people are always trying to compare human reasoning to artificial reasoning rather than looking at the output. The output is all that matters.
nkozyra 8y ago
Given how much the "brains" of AI is built upon fairly old (and very old) mathematics, I find this sentiment to be somewhat myopic. Yes, deep learning has its roots in something 50+ years old. Yes, we still use Bayesian concepts unearthed a century or more ago. Probability theory took foot 80 years ago. So what? Do we throw out the old models and approaches simply because they're old? But to say that AI "hasn't really come very far" because its technological achievements outweigh its computational side doesn't really make much sense to me. Progress is slow. Seeing a story like this and saying "wow, and we're only now seeing this in use in practice today" makes an implicit assumption that once something is proved it should become widespread. The original self-driving cars were not practical; it took advancements in computational power and speed to make it so. That's "coming far." We don't need some novel ML approach to advance AI.
- carlmr 8y ago
  Another nice example are Fourier transforms. It took quite a while from 1822 until we had great applications for it in microcontrollers and DSPs for sound processing. However I can also kind of see the argument here that the deep learning ideas don't seem to have such a fundamental idea behind them. We modeled how biology works at a very simple and abstract level. Now we have more computing power (and especially more data) so we could find a few applications. But to me at least it doesn't look like a solid theory to build upon. It doesn't deliver insights into how things work like Fourier transforms did with signals and Bayes did with probabilities.
  - bunderbunder 8y ago
    > We modeled how biology works That's the popular story behind neural networks. Like any good popular explanation of a STEM concept, it's as useful as it is inaccurate.
    - carlmr 8y ago
      >at a simple and abstract level It would be nice if you quoted honestly.
sergiosgc 8y ago
> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. I assume you are referring to the often declared demise of Moore's Law. Reports of such demise have been greatly exaggerated: https://ourworldindata.org/grapher/transistors-per-microprocessor https://ourworldindata.org/grapher/transistors-per-microproc...
romaniv 8y ago
Another way of putting it: AI research of the 60s-80s was extremely impressive and is ridiculously underappreciated right now. Many ideas were simply way ahead of their time. It's sad that most software engineers today do not bother learning anything about the past of computing, assuming everything made in those decades is automatically "outdated". (This is not restricted to AI. For example, most OO programmers don't know what is Smalltalk.) The guy the article about says as much. “I’ve stopped giving general advice to other researchers,” said Dickmanns, now 82 years old. “Only this much: One should never completely lose sight of approaches that were once very successful.” I really feel for those researchers. Seeing how your life's work is ignored by people who brute-force their ways through the problems you've already solved must be rather depressing.
- squirrelicus 8y ago
  So this
kwhitefoot 8y ago
> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. This reminds me of a remark, possibly made by P. M. S. Blackett, which went something like this: "We've run out of money, now we'll have to start to think." It is possible that stagnation in computing performance will be just the stimulus that the field needs.

gambiting 8y ago

In the 60s, there was a rather famous project where scientists thought that solving image recognition will take them few months at most. It's 2018 and the best algorithms on the planet will answer with 99% confidence that a sofa in a zebra print is in fact, a zebra, unless they were very specifically trained against this scenario. I feel this exact same thing is happening with autonomous cars - yes, it's possible to get them to be extremely good at recognising the road and surroundings - but the last few percent, those crucial few percent that make the technology actually usable, I don't see those happening for another 50 years at least.

nostrademons 8y ago
My unpopular (to me, even) prediction is that wartime will be the catalyst that makes self-driving cars ubiquitous. In times of peace having a car that kills its occupant even 1/10,000 of the time is unacceptable. In times of war having a car (or more likely, truck/tanker/tank) with no occupant is a huge competitive advantage, because someone is actively trying to kill the occupant. Build something with even a 90% chance of crashing and you still win, because you can take people off the battlefield entirely while your opponent loses precious soldiers with every vehicle that's destroyed. Then after the war, people's risk tolerances get reset because quibbling over a 1/10,000 chance of death seems ridiculous when people have been actively trying to kill you for the last 5 years and a countable percentage of your friends are now dead. The way global politics is going we probably don't even have to wait 10 years for this.
- zazen 8y ago
  You seem to have a war like WW2 in mind. But a war between big powers in the 21st century couldn't drag on for five years while you frantically work on new technology. Even without use of nukes, one side or the other is going to be flattened a lot faster than in the days when bombers used propellers. Now, a new Cold War, that might push technological competition, although you won't get that "reset" of people's attitudes that you're after.
  - reitzensteinm 8y ago
    It sounds more like he has a war like Iraq and Afghanistan in mind. Imagine resupply convoys being driven autonomously, capable of launching drones for defense. https://www.army-technology.com/features/feature77200/ https://www.army-technology.com/features/feature77200/ "[...] every 55,702 barrels of fuel burned in Afghanistan by the US military forces corresponded to one casualty."
    - Cthulhu_ 8y ago
      I've seen a number of projects already for self-driving supply trucks, and even walkers from Boston Dynamics. There's a (probably highly scripted) one featured in a later episode of Top Gear, for example.
    - zazen 8y ago
      Well, you probably won't have to wait 10 years for another non-peer war like that. I think he's envisaging a big peer war with mass mobilization ala WW1/WW2, but he imagines such a war would be like a big Iraq/Afghanistan. It wouldn't.
      - larkeith 8y ago
        It seems like it might be feasible for a proxy war (like the one in Syria) to escalate significantly, to the point where multiple major powers are contributing manpower and materiel directly to the front lines; In such a case, it may be possible for an extended war to take place, while still incentivizing deployment of autonomous weaponized platforms - which are already in development today [1]. [1] https://en.wikipedia.org/wiki/Lethal_autonomous_weapon https://en.wikipedia.org/wiki/Lethal_autonomous_weapon
denzil_correa 8y ago
> In the 60s, there was a rather famous project where scientists thought that solving image recognition will take them few months at most. What project was this?
- tome 8y ago
  Probably a reference to this https://en.wikipedia.org/wiki/Computer_vision#History https://en.wikipedia.org/wiki/Computer_vision#History
- code_duck 8y ago
  https://www.explainxkcd.com/wiki/index.php/1425:_Tasks https://www.explainxkcd.com/wiki/index.php/1425:_Tasks
- FabHK 8y ago
  I thought it might have been a reference to (LISP inventor) McCarthy's famous Dartmouth Conference in 1956, where they thought that 10 people over 2 months could make "significant advances" in making machines "use language, form abstractions and concepts, solve kinds of problems now reserved for humans, and improve themselves". https://en.wikipedia.org/wiki/Dartmouth_workshop https://en.wikipedia.org/wiki/Dartmouth_workshop EDIT: Probably the Summer Vision Project, as Tome said in sibling comment. https://dspace.mit.edu/handle/1721.1/6125 https://dspace.mit.edu/handle/1721.1/6125
dalbasal 8y ago
To me, a fundamental question is "is this a problem?" Humans have similar problems too. Our intelligence is trained by experience and evolution to operate within certain parameters. More concerning to me is the fact that a human can conceptualize "couch" from a single example. ML algorithims needs to see thousands of couches before they can classify them.
- deleted 8y ago
  [deleted]
- ksdale 8y ago
  Preface: I didn't intend for this to be so long... I just got on a roll... I'm not sure it's true that we conceptualize couch from a single example. We've all seen thousands of couches in many different contexts, homes, schools, doctor's offices, on TV. If you had a person who had never seen a couch or a zebra before, and all you could tell them was right or wrong, it would probably take them (a lot?) more than a single try before they could distinguish between couch and zebra without fail. The only reason it's easy for us is that we have this giant scaffolding built around zebras as living things that look like a horses and donkeys, and couches as inanimate objects that look like things people sit on and regularly have some pattern on them. I think people tend to underestimate just how much "training" in the ML sense goes into a human brain. After all, humans spend the first... decade? of life incapable of all but the simplest tasks. That's a decade in which our brains are consuming petabytes of information and processing it constantly. Watching my children grow up, the way they learn seems remarkably similar to the way computers learn. If you watch a baby learn to move, it is purely an exercise in going too far in one direction and then too far in the other direction, repeated for basically years until they're coordinated enough to move roughly like an adult by the time they're 3 or so. It's the same with words and concepts too, they're just guessing based on things they already know (and the guesses are often waaaay off, because they don't yet know much), but they're constantly filing things away into their frameworks until their frameworks get big enough that this too resembles the way adults learn. By the time we're adults, the human brain makes ML algorithms look pathetic, but that doesn't take into the account the decades long head start that our brains got.

azernik 8y ago

> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. On the other hand, the article notes that today's AI Summer gets its funding from a constant stream of profitable applications (though less dramatic than the blue-sky hype). The field is less reliant on animal spirits of academia, and has a more reliable source of funding. I still think things like self-driving cars are far in the future, but we're at a point where ongoing research into e.g. machine vision can be funded by less safety-critical commercial applications.

romaniv 8y ago
Expert systems had industry funding in the 80s. And contrary to what some people here keep implying, they delivered results. Some of them are still used right now.

2sk21 8y ago

I completed my PhD in the subject of improving back propagation - in 1992! Many people are startled when I tell them how old the idea of NNs are :-)

YorkshireSeason 8y ago
It's not widely known, but Alan Turing tinkered with neural nets in 1948, see [1], with his boss at the time calling Turing's work a "schoolboy essay". For a history of the subject, see Section 5.1 of [2]. Note that some parts of [2] are rather controversial. For a historical account of back-progagation, which itself was 'invented' many times, see [3, 4]. It is also not widely known that regular expressions, one of programmers' favourite tools, were introduced by Kleene in [5]: he was interested in characterising the behaviour of McCulloch-Pitts "nerve nets" (early versions of neural nets) and finite automata. [1] http://www.alanturing.net/turing_archive/pages/reference%20articles/connectionism/Turing's%20neural%20networks.html http://www.alanturing.net/turing_archive/pages/reference%20a... [2] https://arxiv.org/abs/1404.7828 https://arxiv.org/abs/1404.7828 [3] https://www.math.uni-bielefeld.de/documenta/vol-ismp/52_griewank-andreas-b.pdf https://www.math.uni-bielefeld.de/documenta/vol-ismp/52_grie... [4] http://people.idsia.ch/~juergen/who-invented-backpropagation.html http://people.idsia.ch/~juergen/who-invented-backpropagation... [5] S. Kleene, Representation of events in nerve nets and finite automata, see https://www.rand.org/content/dam/rand/pubs/research_memoranda/2008/RM704.pdf https://www.rand.org/content/dam/rand/pubs/research_memorand...
- FabHK 8y ago
  Great references (though I can't open [4]). I sometimes think the contributions of numerical analysis pioneers is undervalued (eg Wilkinson, Kahan, Golub). Backpropagation is really just reverse mode AD, and has been discovered many times, as you point out. https://en.wikipedia.org/wiki/Automatic_differentiation https://en.wikipedia.org/wiki/Automatic_differentiation EDIT: [4] works now.
s-macke 8y ago
Show them this video: https://www.youtube.com/watch?v=cNxadbrN_aI https://www.youtube.com/watch?v=cNxadbrN_aI

ahartmetz 8y ago

I like your pessimism. But I'm also optimistic that interesting (mostly non-AI) stuff will happen when Moore's Law isn't a substitute for other innovations anymore. I am especially looking forward to Ray Kurzweil eating his hat when the singularity is delayed indefinitely.

tim333 8y ago
You'll have a long wait - I think he projected 2045.
- ahartmetz 8y ago
  Well crap, I remembered something much closer.
  - tim333 8y ago
    He said Turing test 2029. That may have better hat eating possibilities.
Pica_soO 8y ago
Why was it again, that everyone reels in distaste from the church of singularity... Oh, yes... "Oh, powerful deus ex machina yet to come, the one above me is lost, take him into the loving arms of your personality review and do not judge your faithful followers by his threshold." the same reason all other powerful organized religion is distasteful. Rampant Stasi denunciation and a constant attempt to subdue all opposition voices.

vegardx 8y ago

People are often surprised that many planes can land completely autonomously on airports that support autoland (I think they call it ILS CAT 3). And that work on systems like that started in the 60s.

adrianN 8y ago
Automating systems to interact with other automatic systems, likes planes and ILS, is a lot easier than automating systems that need to interact with humans.
- alxlaz 8y ago
  And it's even harder to automate systems that have to derive data from environmental cues. A line-following robot is something simple enough that undergrad students can build, and making it stop if it detects obstacles is a good choice for extra credit. A road-following car that stops when detecting obstacles is way, way, way more complicated than that.
  - adrianN 8y ago
    I don't think stopping for obstacles is the hard part of autonomous driving.
    - reitanqild 8y ago
      It gets harder when you are: * Out of the lab * On roads that aren't level * When the obstacles move * etc
    - rsynnott 8y ago
      It's one surprisingly hard part of it. Note how Teslas enjoy driving into stationary objects with certain properties, say.
      - adrianN 8y ago
        It is my understanding that this is due to having the wrong kinds of sensors.

agumonkey 8y ago

To me the major difference is the pressure differential of today's wave. It's a lot larger market than 80s AI, so it may provide more resources to overcome the hurdles. Or maybe it will get into another "comatose" decade.

Shivetya 8y ago

We get bombarded with the term at nearly every turn. We see it in movies, television, and video games. So for the part what researches expect AI to become is not what the public thinks AI is. Marketing has a way of taking something that sounds cool, is just too complex to fully understand, but not so complex that they cannot convince you that you need it, and from there the terminology gets corrupted. Sure most people dismiss most over the top depictions in tv, movies, and games, but it is surprising how many out there still believe governments and the mega rich have something "close".

ianai 8y ago
I agree that linear thinking has its faults but I wonder if certain things can’t “add up” to opportunities for breakthroughs. Ie exascale computing, quantum scale Boltzmann machines, and faster communication between cpu cores. I’m wondering what new technologies will be possible after modeling at the scales possible by exascale.

MaxBarraclough 8y ago

"hasn't really come very far in a few decades"? How about AlphaGo Zero?

romaniv 8y ago
What about AlphaGo Zero? I'm honestly not sure why a system that plays board games is seen as some kind of giant leap in AI. Seems mostly a result of Google's marketing.
- MaxBarraclough 8y ago
  You're really going to pretend that cracking Go wasn't a big moment for AI?
  - romaniv 8y ago
    In 1996 IBM made a system that was better than any person at playing chess. Chess has roughly 400 moves possible at any point. In 2016 Google made a system that was better at any person at playing Go. Go has roughly 130,000 moves possible at any point. That's approximately equal to 400 * 2^9. Moore's law states that the number of transistors (which is a proxy for computing power) in CPUs doubles every year. 2016 - 1996 = 20 years of computing power growth. In short, after our computing power has increased roughly by a factor of 2 ^ 20 someone made a system that plays a game that's 2 ^ 9 time more complicated than chess. Why is this seen as some giant, surprising leap forward?
    - knight-of-lambd 8y ago
      Because a chess engine isn't scanning 400 possibilities, and a Go engine isn't scanning 130k (where'd you even get that number?) possible game states. For example, a brute force search of a 19x19 Go board to a depth of 20 would yield on the order of 361^20 = 1.4E51 game states. With a reduction in search depth and better algorithms, state-of-the-art engines might cut this down by ten orders of magnitude, but can still be beaten by rank amateurs. Deepmind's approach to board game engines blows all previous approaches out of the water. The claim that their success is incidental to Moore's law is categorically false.

rsynnott 8y ago

"Ever" is a long time. But yeah, I wouldn't be surprised to see it die down again few a few decades at least.

dalbasal 8y ago

At least to some extent, how you sees this relates to what you want/think AI to eventually be. A while back, I heard a particularly curmudgeonly interview with Noam Chomsky, just commenting on the state of AI and recent AI programs (Watson, Deep blue..). Chomsky basically called all these statsistical AI programs brute force parlour tricks, nothing to do with computer science or artificial intelligences. He stopped just short of calling the whole thing fraud. Real AI are things like bird flocking algorithims (did he work on something like this, can't remember). I think the jist was that to be an AI program, a program must embody a theory of intelligence. The theory-less nature of modern statistical algorithims seemed to really peeve him. maybe he's right. Anyway... I don't remember what point 9if any) i was driving at. I think there are abstract values at play, in how we narrate progress. Since we still don't know what inteligence "is," or even if it is a distinct thing, it's hard to know what the real milestones are. Personally, I like the fully realized, running at scale "proofs," but that probably says more about my ability to understand theory than anything else.

braythwayt 8y ago
At some level: Alice: “Computers aren’t doing real intelligence, because they don’t think the way we theorize we think.” Bob: “Computers are doing real intelligence, because we don’t think the way we theorize we think.”
nkozyra 8y ago
> Chomsky basically called all these statsistical AI programs brute force parlour tricks, nothing to do with computer science or artificial intelligences. The counterargument is our own brains use brute force parlor tricks to solve problems. We learn by a priori (pre-built models), by experience (reinforcement learning) and viewing and weighing features that we observe against prior experience (NNs/ANNs). The entire point of ML is to produce approximately the same result a human does. Whether it's a parlor trick or a perfect simulation of the brain is irrelevant. There's no advantage to something novel and more "organic" unless it produces better results. This is a common problem I see - people are always trying to compare human reasoning to artificial reasoning rather than looking at the output. The output is all that matters.

nkozyra 8y ago

Given how much the "brains" of AI is built upon fairly old (and very old) mathematics, I find this sentiment to be somewhat myopic. Yes, deep learning has its roots in something 50+ years old. Yes, we still use Bayesian concepts unearthed a century or more ago. Probability theory took foot 80 years ago. So what? Do we throw out the old models and approaches simply because they're old? But to say that AI "hasn't really come very far" because its technological achievements outweigh its computational side doesn't really make much sense to me. Progress is slow. Seeing a story like this and saying "wow, and we're only now seeing this in use in practice today" makes an implicit assumption that once something is proved it should become widespread. The original self-driving cars were not practical; it took advancements in computational power and speed to make it so. That's "coming far." We don't need some novel ML approach to advance AI.

carlmr 8y ago
Another nice example are Fourier transforms. It took quite a while from 1822 until we had great applications for it in microcontrollers and DSPs for sound processing. However I can also kind of see the argument here that the deep learning ideas don't seem to have such a fundamental idea behind them. We modeled how biology works at a very simple and abstract level. Now we have more computing power (and especially more data) so we could find a few applications. But to me at least it doesn't look like a solid theory to build upon. It doesn't deliver insights into how things work like Fourier transforms did with signals and Bayes did with probabilities.
- bunderbunder 8y ago
  > We modeled how biology works That's the popular story behind neural networks. Like any good popular explanation of a STEM concept, it's as useful as it is inaccurate.
  - carlmr 8y ago
    >at a simple and abstract level It would be nice if you quoted honestly.

sergiosgc 8y ago

> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. I assume you are referring to the often declared demise of Moore's Law. Reports of such demise have been greatly exaggerated: https://ourworldindata.org/grapher/transistors-per-microprocessor https://ourworldindata.org/grapher/transistors-per-microproc...

romaniv 8y ago

Another way of putting it: AI research of the 60s-80s was extremely impressive and is ridiculously underappreciated right now. Many ideas were simply way ahead of their time. It's sad that most software engineers today do not bother learning anything about the past of computing, assuming everything made in those decades is automatically "outdated". (This is not restricted to AI. For example, most OO programmers don't know what is Smalltalk.) The guy the article about says as much. “I’ve stopped giving general advice to other researchers,” said Dickmanns, now 82 years old. “Only this much: One should never completely lose sight of approaches that were once very successful.” I really feel for those researchers. Seeing how your life's work is ignored by people who brute-force their ways through the problems you've already solved must be rather depressing.

squirrelicus 8y ago
So this

kwhitefoot 8y ago

> Given Moore's law, I'd be surprised if we ever see another AI summer after the current one peters out. This reminds me of a remark, possibly made by P. M. S. Blackett, which went something like this: "We've run out of money, now we'll have to start to think." It is possible that stagnation in computing performance will be just the stimulus that the field needs.