3 ms·

An AI "winter" is a long period in which [edit: funding is cut because...] researchers are in disbelief about having a path to real intelligence. I think that i

An AI "winter" is a long period in which [edit: funding is cut because...] researchers are in disbelief about having a path to real intelligence. I think that is not the case at this time, because we have (or approaching) adequate tools to rationally dismiss that disbelief. The current AI "spring" has brought back the belief that connectionism may by the ultimate tool to explain the human brain. I mean you can't deny that DL models of vision look eerily like the early stages in visual processing in the brain (which is a very large part of it). Even if DL researchers lose their path in search for "true AI", the neuroscientists can keep probing the blueprint to find new clues to its intelligence. Even AI companies are starting to create plausible models that link to biology. So at this time, it's unlikely that progress will be abandoned any time soon. E.g. https://arxiv.org/abs/1610.00161 https://arxiv.org/abs/1610.00161 https://arxiv.org/abs/1706.04698 https://arxiv.org/abs/1706.04698 https://www.ncbi.nlm.nih.gov/pubmed/28095195 https://www.ncbi.nlm.nih.gov/pubmed/28095195

nl 8y ago
That’s not what the AI Winter was. It was when anything that used the term “AI” (including academic research) was unfundable.
dekhn 8y ago
No, AI winter was when the AI people oversold the tech, then failed to deliver, and lost their funding. This is well documented in histories of the field.
- zerostar07 8y ago
  I think the scientific pessimism preceded the funding cuts: https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter
jdonaldson 8y ago
The brain has a number of functional parts that we don't understand all that well. Research on the brain hits a wall every now and then, but you never hear the phrase "Neuroscience Winter". We're starting to train models that match biological brain behavior, at least in some crude functional/structural sense. Maybe this is just a string of coincidences, but my guess is that discoveries of analogous biological/model components will continue to happen, and we'll be able to learn more about AI and the brain by linking related phenomenon in vitro and in silica. A few more recent examples: https://deepmind.com/blog/grid-cells/ https://deepmind.com/blog/grid-cells/ https://www.nature.com/articles/nature04485 https://www.nature.com/articles/nature04485
- zerostar07 8y ago
  Neuroscience has been in constant winter, mainly because the methods are too crude and small scale, and that by observing 100 neurons out of billions makes it impossible to tell the whole story. The deep mind papers are interesting, but they are just a start imho. It may be that they are focusing on a mere coincidence. Nevertheless it is exciting to see progress in that direction, thats why i don't think DL research is going to lose steam soon. It would require a major show-stopper discovery (like the minsky-papert paper). Tesla crashes are no such thing. It seems CS people are sick and tired of the hype, but i feel neuroscientists are now warming up to it.
  - jdonaldson 8y ago
    I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain activations work better than others in DL is a little weak imho. Right now ML practitioners just throw a lot of parameter combinations at the wall and see what sticks. That's fine, but it's not really indicative of a robust understanding of model behavior.
    - zerostar07 8y ago
      I 'm actually of the opposite idea - let the two fields evolve by their own darwinian process as this will yield more interesting results. DL itself has created its own scientific questions and puzzles that may lead to important discoveries (which may transfer to neuroscience). E.g. "why does batch normalization / dropout work."
mr_toad 8y ago
From what I’ve read of the last AI winters, funding was more centralised, & projects were larger and fewer. Hardware was scarcer. These days anyone with a few dollars to spend on compute time and some free software can do machine learning. I can’t see that going away.

nl 8y ago

That’s not what the AI Winter was. It was when anything that used the term “AI” (including academic research) was unfundable.

dekhn 8y ago

No, AI winter was when the AI people oversold the tech, then failed to deliver, and lost their funding. This is well documented in histories of the field.

zerostar07 8y ago
I think the scientific pessimism preceded the funding cuts: https://en.wikipedia.org/wiki/AI_winter https://en.wikipedia.org/wiki/AI_winter

jdonaldson 8y ago

The brain has a number of functional parts that we don't understand all that well. Research on the brain hits a wall every now and then, but you never hear the phrase "Neuroscience Winter". We're starting to train models that match biological brain behavior, at least in some crude functional/structural sense. Maybe this is just a string of coincidences, but my guess is that discoveries of analogous biological/model components will continue to happen, and we'll be able to learn more about AI and the brain by linking related phenomenon in vitro and in silica. A few more recent examples: https://deepmind.com/blog/grid-cells/ https://deepmind.com/blog/grid-cells/ https://www.nature.com/articles/nature04485 https://www.nature.com/articles/nature04485

zerostar07 8y ago
Neuroscience has been in constant winter, mainly because the methods are too crude and small scale, and that by observing 100 neurons out of billions makes it impossible to tell the whole story. The deep mind papers are interesting, but they are just a start imho. It may be that they are focusing on a mere coincidence. Nevertheless it is exciting to see progress in that direction, thats why i don't think DL research is going to lose steam soon. It would require a major show-stopper discovery (like the minsky-papert paper). Tesla crashes are no such thing. It seems CS people are sick and tired of the hype, but i feel neuroscientists are now warming up to it.
- jdonaldson 8y ago
  I'd like to see neuroscience take more of a fundamental role in grounding/situating deep learning approaches. VGG is often mentioned as being roughly analogous to the visual cortex, but it differs in important ways. There's all kinds of why questions there. E.g. Why does deep learning work better with (e.g. ReLU) activation based on pooling while biological models use an inhibitory mechanism? Theory on why certain activations work better than others in DL is a little weak imho. Right now ML practitioners just throw a lot of parameter combinations at the wall and see what sticks. That's fine, but it's not really indicative of a robust understanding of model behavior.
  - zerostar07 8y ago
    I 'm actually of the opposite idea - let the two fields evolve by their own darwinian process as this will yield more interesting results. DL itself has created its own scientific questions and puzzles that may lead to important discoveries (which may transfer to neuroscience). E.g. "why does batch normalization / dropout work."

mr_toad 8y ago

From what I’ve read of the last AI winters, funding was more centralised, & projects were larger and fewer. Hardware was scarcer. These days anyone with a few dollars to spend on compute time and some free software can do machine learning. I can’t see that going away.