5 ms·

The reason AI has been so successful recently is that the research community has assumed a ruthlessly empirical philosophy: no idea, no matter how beautiful or

The reason AI has been so successful recently is that the research community has assumed a ruthlessly empirical philosophy: no idea, no matter how beautiful or interesting, is considered truly useful until it bears measurable results on some dataset. The reason neuromorphic computing gets such skepticism from AI researchers is that so far it has resisted any attempts at this kind of empiricism. No neuromorphic implementation has shown state of the art results on any important problem. If/When neuromorphic computers show groundbreaking results, the community will pivot quickly to using them. But expecting AI researchers to show deference to neuromorphic computing because it "mimics the brain" is to ignore the empirical philosophy that has led to AI's success.

andreyk 10y ago
To be fair, this whole Deep Learning renaissance was made possible and kicked off only after decades of research in multi layer neural nets (going back to the 80s) by Hinton, Lecun, etc. They stuck to their chosen method despite it not having great empirical results (the research community shunned NNs in the 90s for SVNs cause they worked better), because they believed it should and will work - and it did, eventually. So a similar argument for 'basic research' could be made for neuromorphic computing.
- bcatanzaro 10y ago
  Yes, I totally agree. Yann LeCun, Geoff Hinton, Jurgen Schmidhuber and others did unpopular work for a long time. And they deserve tons of credit for their perseverance which paid off. Similarly, I think it's great that there are AI researchers working on techniques which are currently out of favor. It's important to have diversity of viewpoint. What irritates me about neuromorphic computing is that much of the work I see publicized (including the work in this article) isn't being presented as basic research on a risky hypothesis. Instead it's presented as the future of AI, despite the current lack of any demonstrated utility, and the almost complete disconnect between the AI researchers building the future of AI and the neuromorphic community. The burden of proof is always on the researcher to show utility, and if the neuromorphic computing community can do that, I'll be super excited! Until then, I'll be waiting for something measurable and concrete, and rolling my eyes at brain analogies.
  - Russell91 10y ago
    > Yes, I totally agree. Yann LeCun, Geoff Hinton, Jurgen Schmidhuber and others did unpopular work for a long time. ... > Until then, I'll be ... rolling my eyes at brain analogies. Maybe you don't realize this, but these guys made more brain analogies than you can count over the same period to which you attribute their greatness. Meanwhile, they were attacked year after year by state-of-the-art land grabbers saying the same things you just did. > isn't being presented as basic research on a risky hypothesis. It is basic research, but it's not a risky hypothesis. Existing neuromorphic computers achieve 10^14 ops/s at 20 W. Thats 5 Tops/Watt. The best GPUs currently achieve less than 200 Gops/Watt. Where is the risk in saying that a man-made neuromorphic chip can achieve more per dollar than a GPU. There is no risk, and suggesting that this field is somehow has too much risk for advances to be celebrated is absolutely crazy.
    - deepnotderp 10y ago
      Non-neuromorphic (analog) deep learning chip startup here. We're forecasting AT LEAST ~50 TOPS/watt for inference.
      - Russell91 10y ago
        Sure - I guess it's productive for me to answer why this doesn't disagree with my comment. By the time you get the software to hook up that kind of low bit precision (READ: neuromorphic) compute performance with extreme communication-minimizing strategies (READ: neuromorphic), which will invariable require compute colocated, persistent storage (READ: neuromorphic) in any type of general AI application, you're not exactly making the argument that neuromorphic chips are a bad idea. We literally have to start taking neuromorphic to mean some silly semantics like "exactly like the brain in every possible way" in order to disagree with it. Edit: also, to ground this discussion, there are extremely concrete reason why current neural net architectures will NOT work with the above optimizations. That's the primary motivation for talking about "neuromorphic", or any other synonym you want to coin, as fundamentally different hardware. AI software ppl need to have a term for hardware of the future, which simply won't be capable of running AlexNet well at all, in the same way that a GPU can't run CPU code well. I think the term "neuromorphic" to describe this hardware is as productive as any.

andreyk 10y ago

To be fair, this whole Deep Learning renaissance was made possible and kicked off only after decades of research in multi layer neural nets (going back to the 80s) by Hinton, Lecun, etc. They stuck to their chosen method despite it not having great empirical results (the research community shunned NNs in the 90s for SVNs cause they worked better), because they believed it should and will work - and it did, eventually. So a similar argument for 'basic research' could be made for neuromorphic computing.

bcatanzaro 10y ago
Yes, I totally agree. Yann LeCun, Geoff Hinton, Jurgen Schmidhuber and others did unpopular work for a long time. And they deserve tons of credit for their perseverance which paid off. Similarly, I think it's great that there are AI researchers working on techniques which are currently out of favor. It's important to have diversity of viewpoint. What irritates me about neuromorphic computing is that much of the work I see publicized (including the work in this article) isn't being presented as basic research on a risky hypothesis. Instead it's presented as the future of AI, despite the current lack of any demonstrated utility, and the almost complete disconnect between the AI researchers building the future of AI and the neuromorphic community. The burden of proof is always on the researcher to show utility, and if the neuromorphic computing community can do that, I'll be super excited! Until then, I'll be waiting for something measurable and concrete, and rolling my eyes at brain analogies.
- Russell91 10y ago
  > Yes, I totally agree. Yann LeCun, Geoff Hinton, Jurgen Schmidhuber and others did unpopular work for a long time. ... > Until then, I'll be ... rolling my eyes at brain analogies. Maybe you don't realize this, but these guys made more brain analogies than you can count over the same period to which you attribute their greatness. Meanwhile, they were attacked year after year by state-of-the-art land grabbers saying the same things you just did. > isn't being presented as basic research on a risky hypothesis. It is basic research, but it's not a risky hypothesis. Existing neuromorphic computers achieve 10^14 ops/s at 20 W. Thats 5 Tops/Watt. The best GPUs currently achieve less than 200 Gops/Watt. Where is the risk in saying that a man-made neuromorphic chip can achieve more per dollar than a GPU. There is no risk, and suggesting that this field is somehow has too much risk for advances to be celebrated is absolutely crazy.
  - deepnotderp 10y ago
    Non-neuromorphic (analog) deep learning chip startup here. We're forecasting AT LEAST ~50 TOPS/watt for inference.
    - Russell91 10y ago
      Sure - I guess it's productive for me to answer why this doesn't disagree with my comment. By the time you get the software to hook up that kind of low bit precision (READ: neuromorphic) compute performance with extreme communication-minimizing strategies (READ: neuromorphic), which will invariable require compute colocated, persistent storage (READ: neuromorphic) in any type of general AI application, you're not exactly making the argument that neuromorphic chips are a bad idea. We literally have to start taking neuromorphic to mean some silly semantics like "exactly like the brain in every possible way" in order to disagree with it. Edit: also, to ground this discussion, there are extremely concrete reason why current neural net architectures will NOT work with the above optimizations. That's the primary motivation for talking about "neuromorphic", or any other synonym you want to coin, as fundamentally different hardware. AI software ppl need to have a term for hardware of the future, which simply won't be capable of running AlexNet well at all, in the same way that a GPU can't run CPU code well. I think the term "neuromorphic" to describe this hardware is as productive as any.