5 ms·

This appears to be the pattern of AI development overall: booms and busts of hype and disillusionment that leave behind useful tools that most don’t consciously

This appears to be the pattern of AI development overall: booms and busts of hype and disillusionment that leave behind useful tools that most don’t consciously consider to be remnants of a previous AI bubble.

mdoms 5y ago
The same will happen with deep learning networks because if we're honest no serious researcher would really consider this "AI" either. It's just a big self-correcting linear algebra machine. Researchers will keep using that "AI" label for as long as it gets them research grants or selling products.
- bopbeepboop 5y ago
  Why do you think DNNs are unrelated to “AI”? They develop an internal language[*] through experience in which they model the world, and make decisions based upon that language; what more do you want from them? [*] You can interpret that “linear algebra machine” as the Euclidean VM modeling some latent type theory; Curry-Howard + algebra-geometry correspondences connect systems of difference equations to type theories.
- WanderPanda 5y ago
  I wouldn't call it a "linear algebra machine" if the central point (of NNs) is that they are highly non-linear (the deeper the more non-linear, roughly)
flir 5y ago
When it works, it's no longer AI. I know I'm paraphrasing someone smarter than me there. (Edit: Larry Tesler maybe)
- Jensson 5y ago
  People believe that achieving X means the agent will be intelligent and can thus do Y, Z, A, B, C etc. But then someone builds an agent that can do X, but it can't do anything else. So people don't view it as intelligent, since it doesn't do any of all the other things people associated with being able to do X. So it has nothing to do with being used to computers solving new problems, it is that we now realize that solving those problems doesn't really require the level of intelligence we thought it would. Of course solving new problems is great, but it isn't nearly as exciting as being able to create intelligent agents. Edit: And it isn't just hobbyists making this mistake, you see big AI researchers often make it as well and they have for a long time. You will find tons of articles of AI researchers saying something along the lines of "Now we solved X, therefore solving A, B, C, D, E, F is just around the corner!", but they basically never deliver on that. Intelligence was harder than they thought.
  - 6gvONxR4sf7o 5y ago
    Exactly. People make the case that something along the lines of AGI is necessary and sufficient for some task, then someone comes along and solve it with something completely unlike AGI, proving that it's not a necessary condition. Or in more straightforward but less precise terms, solving this is AI and then it isn't.
  - IggleSniggle 5y ago
    So, our brains are just a little ecosystem of competing functions, and our pre-frontal cortex / prediction engine (the piece most AI researchers care about) is trained on species survival, but beholden to the "lizard brain." I'm still on the bandwagon that AGI is already solved, and we just don't recognize it because we aren't as complex or magical as we like to pretend we are. We are just currently selected-for species survival, and not even in the present environment as the dominant species of the planet. That doesn't make us "Generally intelligent" any more than many other systems on the planet (including the computational ones). I'm not even convinced that "intelligence" is especially a thing. If we're trying to replicate "human intelligence," then I think that's something else entirely, and would require removing a lot of capability from the AI that I don't think anybody would want to remove from AI, given that we have plenty of serviceable humans to do the work of human-intelligence style tasks. No, when folks say AGI, they mean "cares about the things I care about, and is more accurate, but can speak in a way I understand." But how is a network of neurons going to care about what you care about if they see the world in a different way, or have been trained on solving a different set of problems than "living a life as a human?" It just doesn't make sense to expect, or even want, human intelligence from a synthetic intelligence.
Blammar 5y ago
Don't you think, though, that each boom and bust cycle leaves us closer to real accomplishments? We now have protein folders and superhuman Go players -- that's new. I agree that ML ("AI") is currently at the alchemy stage. And, guess what? A neural network isn't even Turing complete! [citation needed - correct me if I am wrong.] So ML can only compute SOME functions. AGI, when it comes, and believe me, it will, will have ML as part of its structure, but only a small part.
- ashtonkem 5y ago
  I have no idea if each boom brings us closer. Scientific discovery is not exactly a linear process; we can’t observe before the fact if we’re getting closer or are on a dead end path, and that’s even assuming we can ever get there at all. But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.
- 6gvONxR4sf7o 5y ago
  I mean, each cycle does leave us with real accomplishments. If the question is whether it leaves us closer to AGI, then it's an open question. Like, when AGI happens, it will certainly trace its roots back to things happening in each period, but its roots will also go back to Gauss and Newton and co, so nobody knows whether it gets us closer in the way you probably mean.
- Dr_Birdbrain 5y ago
  ML is Turing complete in the sense that every computable function can be approximated to arbitrary precision by a 3-layer neural network. Classic result from the 90s, the paper in question (iirc) is titled something like “neural networks are universal approximators” Turing machines also can only approximate to arbitrary precision, so the computation models are equivalent.
- randcraw 5y ago
  Of course that depends on what you mean by "real accomplishments". It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall. Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam. Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.
Animats 5y ago
This round got a lot further than the last round. I went through Stanford CS in the mid 1980s, just as it was becoming clear that expert systems didn't really do much. There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. Big-time denial as the field collapsed. Almost all the 1980s AI startups went bust. The "AI winter" followed. Today, expert systems are almost forgotten. This time around, there are large, successful industries using machine learning. Tens of thousands of people understand it. It's still not "strong AI", but it's useful and profitable. So work will continue. We're still at least one big idea short of "strong AI". A place to look for it is in "common sense", narrowly defined as "not doing something really bad in the next 30 seconds". This just requires lower mammal level AI, not human level. Something with the motion planning and survival capabilities of a squirrel, for example, would be a big advance. (I once had a conversation with Rod Brooks about this. He'd been building six-legged robot insects, and was giving a talk about how his group was making the jump to human level AI, with a project called "Cog". I asked why such a big jump? Why not try for a robot mouse, which might be within reach. He said "I don't want to go down in history as the man who created the world's best robot mouse". Cog was a flop, and Brooks goes down in history as the the creator of the robot vacuum cleaner.)
- dreamcompiler 5y ago
  > There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. That would be Ed Feigenbaum, the man who (IMHO) almost single-handedly brought on the AI winter of the 80s. Because of him we had to call all the AI research we did in the 90s something other than "AI" lest it get shut down instantly.
  - Animats 5y ago
    Um, yes. His book. "The Fifth Generation - Artificial Intelligence and Japan's Computer Challenge to the World" lays out his position. For a short version, here's his testimony before a congressional committee, asking for funding.[1] "The era of reasoning machines is inevitable. It is the "manifest destiny" of computing." He was taken seriously at the time. Chief Scientist of the USAF at one point. Turing Award. [1] https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw4621.pdf https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw46...
m463 5y ago
that's the right answer. New technologies are always overestimated in the short term, but underestimated in the long term. The long-term usually is not a generalized solution, but is still a true solution. speech recognition hype... now most phone trees use them. electric car hype... now california is full of them. self driving cars... now many new cars have driver assist.
- giardini 5y ago
  Post must be machine-generated - it says nothing correct. Electric cars are NOT AI! But it is true that "california is full of them", whatever "them" may be. That may have something to do with the crowd (whores and madmen) attracted by the California Gold Rush and whose descendants now mine their wealth in VC endeavors, including deep learning. https://en.wikipedia.org/wiki/California_Gold_Rush https://en.wikipedia.org/wiki/California_Gold_Rush "driver assist" is a long way from a "self driving car".
  - m463 5y ago
    I was making a general statement about "technologies"... I'm saying AI is a technology that is being overestimated now. People are trying to use it for everything now. I think in the future when the hype has worn off it will not have worked for everything hyped. But it will have worked well for a couple things to the point they are taken for granted or have become invisible.
narrator 5y ago
I think there are these technological problems, many that overlap with AI, that a lot of politically powerful people want to come to fruition to enable the World Economic Forum lauded "Fourth Industrial Revolution" [1] and they are willing to invest insane amounts of money in them without expecting much of a return, thus creating really big p/e ratios. Examples include: 1. Self-driving cars. The ultimate being no steering wheel and carefully software controlled and permissioned as to where you are allowed to go. 2. AI powered Body monitoring devices (e.g Fitbit) that can be used to biologically monitor millions of consumers body states and functions. 3. Any new kind of surveillance technology. Miracle mass surveillance is something that's really interesting to this market. 4. Large centralized social media networks that aggressively moderate content with AI. 5. Various kinds of transhumanist biotech that design all sorts of biotech stuff with AI. What the heck is Calico up to anyway? 6. Blockchain stuff. AI enabled or otherwise. They are also interested in environment tech like any kind of alternative energy, no matter how speculative or impractical and fake meat, but I digress. So I would guess the bubble is the WEF crowd and friends with absurd amounts of money trying to make their future, for good or ill, a reality and investing without much regard for the economics of the project. [1]https://www.weforum.org/agenda/2016/01/what-is-the-fourth-industrial-revolution/ https://www.weforum.org/agenda/2016/01/what-is-the-fourth-in...
- nbzso 5y ago
  It is logical. Protect the elites with automation and control the masses to prevent a "Second French Revolution". Which may be postponed but doubtfully avoided. You will own nothing and will be happy.:)

mdoms 5y ago

The same will happen with deep learning networks because if we're honest no serious researcher would really consider this "AI" either. It's just a big self-correcting linear algebra machine. Researchers will keep using that "AI" label for as long as it gets them research grants or selling products.

bopbeepboop 5y ago
Why do you think DNNs are unrelated to “AI”? They develop an internal language[*] through experience in which they model the world, and make decisions based upon that language; what more do you want from them? [*] You can interpret that “linear algebra machine” as the Euclidean VM modeling some latent type theory; Curry-Howard + algebra-geometry correspondences connect systems of difference equations to type theories.
WanderPanda 5y ago
I wouldn't call it a "linear algebra machine" if the central point (of NNs) is that they are highly non-linear (the deeper the more non-linear, roughly)

flir 5y ago

When it works, it's no longer AI. I know I'm paraphrasing someone smarter than me there. (Edit: Larry Tesler maybe)

Jensson 5y ago
People believe that achieving X means the agent will be intelligent and can thus do Y, Z, A, B, C etc. But then someone builds an agent that can do X, but it can't do anything else. So people don't view it as intelligent, since it doesn't do any of all the other things people associated with being able to do X. So it has nothing to do with being used to computers solving new problems, it is that we now realize that solving those problems doesn't really require the level of intelligence we thought it would. Of course solving new problems is great, but it isn't nearly as exciting as being able to create intelligent agents. Edit: And it isn't just hobbyists making this mistake, you see big AI researchers often make it as well and they have for a long time. You will find tons of articles of AI researchers saying something along the lines of "Now we solved X, therefore solving A, B, C, D, E, F is just around the corner!", but they basically never deliver on that. Intelligence was harder than they thought.
- 6gvONxR4sf7o 5y ago
  Exactly. People make the case that something along the lines of AGI is necessary and sufficient for some task, then someone comes along and solve it with something completely unlike AGI, proving that it's not a necessary condition. Or in more straightforward but less precise terms, solving this is AI and then it isn't.
- IggleSniggle 5y ago
  So, our brains are just a little ecosystem of competing functions, and our pre-frontal cortex / prediction engine (the piece most AI researchers care about) is trained on species survival, but beholden to the "lizard brain." I'm still on the bandwagon that AGI is already solved, and we just don't recognize it because we aren't as complex or magical as we like to pretend we are. We are just currently selected-for species survival, and not even in the present environment as the dominant species of the planet. That doesn't make us "Generally intelligent" any more than many other systems on the planet (including the computational ones). I'm not even convinced that "intelligence" is especially a thing. If we're trying to replicate "human intelligence," then I think that's something else entirely, and would require removing a lot of capability from the AI that I don't think anybody would want to remove from AI, given that we have plenty of serviceable humans to do the work of human-intelligence style tasks. No, when folks say AGI, they mean "cares about the things I care about, and is more accurate, but can speak in a way I understand." But how is a network of neurons going to care about what you care about if they see the world in a different way, or have been trained on solving a different set of problems than "living a life as a human?" It just doesn't make sense to expect, or even want, human intelligence from a synthetic intelligence.

Blammar 5y ago

Don't you think, though, that each boom and bust cycle leaves us closer to real accomplishments? We now have protein folders and superhuman Go players -- that's new. I agree that ML ("AI") is currently at the alchemy stage. And, guess what? A neural network isn't even Turing complete! [citation needed - correct me if I am wrong.] So ML can only compute SOME functions. AGI, when it comes, and believe me, it will, will have ML as part of its structure, but only a small part.

ashtonkem 5y ago
I have no idea if each boom brings us closer. Scientific discovery is not exactly a linear process; we can’t observe before the fact if we’re getting closer or are on a dead end path, and that’s even assuming we can ever get there at all. But it’s hard to deny that each boom doesn’t give us something useful. Neural Nets might not exactly make an AGI, but they do have uses.
6gvONxR4sf7o 5y ago
I mean, each cycle does leave us with real accomplishments. If the question is whether it leaves us closer to AGI, then it's an open question. Like, when AGI happens, it will certainly trace its roots back to things happening in each period, but its roots will also go back to Gauss and Newton and co, so nobody knows whether it gets us closer in the way you probably mean.
Dr_Birdbrain 5y ago
ML is Turing complete in the sense that every computable function can be approximated to arbitrary precision by a 3-layer neural network. Classic result from the 90s, the paper in question (iirc) is titled something like “neural networks are universal approximators” Turing machines also can only approximate to arbitrary precision, so the computation models are equivalent.
randcraw 5y ago
Of course that depends on what you mean by "real accomplishments". It seems to me that that deep nets have effectively maximized the potential of using gradient pursuit to model patterns. But if you remove gradients from your data, or shrink your data down to tens of samples, or shift the problem to logic, or need to use functions that aren't convex or differentiable, deep nets run smack into a wall. Luckily human perception makes extensive use of gradients, as does most search, so problems in those arenas have been unsurprisingly amenable to solution using deep nets (vision, speech, game play, etc). But many of the problems that remain untouched by DL, like human cognition, are NOT driven by gradients. Will deep nets eventually fill that void? I doubt it. You can convert only so many problems with big data into gradients to pursue them efficiently with DL before that transformation trick runs out of steam. Personally I think deep net language modeling is one of those areas, and soon we'll encounter the limit of their generalizable contextual phrase association. Then because deep nets are so difficult to selectively revise or extend the specifics that they have learned, the vanguard of ever more complex deep nets (transformers) will eventually sink beneath their own weight, taking the last best hope for DL-based general AI with them.

Animats 5y ago

This round got a lot further than the last round. I went through Stanford CS in the mid 1980s, just as it was becoming clear that expert systems didn't really do much. There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. Big-time denial as the field collapsed. Almost all the 1980s AI startups went bust. The "AI winter" followed. Today, expert systems are almost forgotten. This time around, there are large, successful industries using machine learning. Tens of thousands of people understand it. It's still not "strong AI", but it's useful and profitable. So work will continue. We're still at least one big idea short of "strong AI". A place to look for it is in "common sense", narrowly defined as "not doing something really bad in the next 30 seconds". This just requires lower mammal level AI, not human level. Something with the motion planning and survival capabilities of a squirrel, for example, would be a big advance. (I once had a conversation with Rod Brooks about this. He'd been building six-legged robot insects, and was giving a talk about how his group was making the jump to human level AI, with a project called "Cog". I asked why such a big jump? Why not try for a robot mouse, which might be within reach. He said "I don't want to go down in history as the man who created the world's best robot mouse". Cog was a flop, and Brooks goes down in history as the the creator of the robot vacuum cleaner.)

dreamcompiler 5y ago
> There had been Stanford faculty running around claiming "strong AI real soon now" to Congress. That would be Ed Feigenbaum, the man who (IMHO) almost single-handedly brought on the AI winter of the 80s. Because of him we had to call all the AI research we did in the 90s something other than "AI" lest it get shut down instantly.
- Animats 5y ago
  Um, yes. His book. "The Fifth Generation - Artificial Intelligence and Japan's Computer Challenge to the World" lays out his position. For a short version, here's his testimony before a congressional committee, asking for funding.[1] "The era of reasoning machines is inevitable. It is the "manifest destiny" of computing." He was taken seriously at the time. Chief Scientist of the USAF at one point. Turing Award. [1] https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw4621.pdf https://stacks.stanford.edu/file/druid:mv321kw4621/mv321kw46...

m463 5y ago

that's the right answer. New technologies are always overestimated in the short term, but underestimated in the long term. The long-term usually is not a generalized solution, but is still a true solution. speech recognition hype... now most phone trees use them. electric car hype... now california is full of them. self driving cars... now many new cars have driver assist.

giardini 5y ago
Post must be machine-generated - it says nothing correct. Electric cars are NOT AI! But it is true that "california is full of them", whatever "them" may be. That may have something to do with the crowd (whores and madmen) attracted by the California Gold Rush and whose descendants now mine their wealth in VC endeavors, including deep learning. https://en.wikipedia.org/wiki/California_Gold_Rush https://en.wikipedia.org/wiki/California_Gold_Rush "driver assist" is a long way from a "self driving car".
- m463 5y ago
  I was making a general statement about "technologies"... I'm saying AI is a technology that is being overestimated now. People are trying to use it for everything now. I think in the future when the hype has worn off it will not have worked for everything hyped. But it will have worked well for a couple things to the point they are taken for granted or have become invisible.

narrator 5y ago

I think there are these technological problems, many that overlap with AI, that a lot of politically powerful people want to come to fruition to enable the World Economic Forum lauded "Fourth Industrial Revolution" [1] and they are willing to invest insane amounts of money in them without expecting much of a return, thus creating really big p/e ratios. Examples include: 1. Self-driving cars. The ultimate being no steering wheel and carefully software controlled and permissioned as to where you are allowed to go. 2. AI powered Body monitoring devices (e.g Fitbit) that can be used to biologically monitor millions of consumers body states and functions. 3. Any new kind of surveillance technology. Miracle mass surveillance is something that's really interesting to this market. 4. Large centralized social media networks that aggressively moderate content with AI. 5. Various kinds of transhumanist biotech that design all sorts of biotech stuff with AI. What the heck is Calico up to anyway? 6. Blockchain stuff. AI enabled or otherwise. They are also interested in environment tech like any kind of alternative energy, no matter how speculative or impractical and fake meat, but I digress. So I would guess the bubble is the WEF crowd and friends with absurd amounts of money trying to make their future, for good or ill, a reality and investing without much regard for the economics of the project. [1]https://www.weforum.org/agenda/2016/01/what-is-the-fourth-industrial-revolution/ https://www.weforum.org/agenda/2016/01/what-is-the-fourth-in...

nbzso 5y ago
It is logical. Protect the elites with automation and control the masses to prevent a "Second French Revolution". Which may be postponed but doubtfully avoided. You will own nothing and will be happy.:)