4 ms·

How come they aim for GPU technology rather than for neuromorphic computing ASICs?

by sawwit 11y ago

kitcar 11y ago
Just a guess, but they project the cost of large scale GPU manufacturing to decline faster than ASIC due to the shared applications of gaming/VR?
- blackguardx 11y ago
  You would think AI would be bigger than VR. Edit: going with GPUs makes sense now, but it doesn't indicate a big bet on Facebook's part. I don't think opensourcing a server cabinet is that interesting, but I'm a hardware engineer. In 10 years I think more people will touch AI than VR but I don't know how that translates to investment.
  - unchocked 11y ago
    Gaming is a proven market. Capital hates risk. (Real capital, not the fringe type we talk about here.)
  - argonaut 11y ago
    AI might be bigger than VR, but the timeframes are messed up. VR is an engineering problem now; I don't think there are major field-shattering breakthroughs needed to make reasonably realistic, cheap VR within the next 5 years. AI is nowhere close to strong AI or reasonable general-purpose intelligent programs that don't require loads of human tuning. It will take several breakthroughs.
vonnik 11y ago
Which neuromorphic ASICs are you thinking of? GPUs are fast and they work now. Some "neuromorphic" chips, like IBM's True North, are just low-energy GPUs, so neuromorphic is in the eyes of the beholder...
- emcq 11y ago
  This is not very accurate. The true North chip places memory next to compute elements on chip. GPUs have most memory off chip. The true North hardware is specialized to neural net stuff, and doesn't allow general floating point compute, have any sort of instruction set decoder, etc. That turns into big performance and power wins.
modeless 11y ago
Contrary to what you might read in puff pieces from IBM PR, all existing "neuromorphic" hardware is terrible. The neural nets that work don't look anything like the brain, and neural nets that look like the brain don't work. People constructing neural nets that look like the brain are engaged in cargo cult science, because the truth is we have no idea yet how the brain works and attempting to imitate it without knowing that is doomed to failure.
- dangirsh 11y ago
  I'm curious to hear your opinions on Numenta's work on emulating parts of the neocortex.
  - howlin 11y ago
    He just gave it to you. Their system is not really competitive in any metric people are measuring.
  - chimtim 11y ago
    A good measure of any of the techniques are the results. We see record object recognition and speech recognition results. How many of these results are from Numenta or IBM neomorphic chips? Most successes have been on deep learning architectures over GPUs (and large datasets). While these fancy architectures may have their applications, they have nothing to do with recent advancements in last 3 years.
    - emcq 11y ago
      To be fair, the goal of those new computing architectures is not to advance algorithms. That at best is a side benefit. Those algorithmic performance improvements seen in DNN have come from improved datasets and training. Numenta and IBM arent focusing on training AFAIK. Google's Quantum Annealing [0] is the only hardware I'm aware of focused on training, although there are rumors Nervana Systems may produce something [1]. I'm sure there are others; accelerating training of DNN isnt a particularly new idea. The goal of these other computing architectures is typically to provide lower power, higher frequency/lower latency, or smaller form factor execution of trained models, but there is a question of how much value they can provide over more conventional chips to be worth the chip design costs. However without these architectures becoming as mainstream as say a GPU, I think we will continue to see advances come from the typical everyday computer. The ML community seems to be much more democratic than others. [0] http://googleresearch.blogspot.com/2015/12/when-can-quantum-annealing-win.html http://googleresearch.blogspot.com/2015/12/when-can-quantum-... [1] http://www.nervanasys.com/about/ http://www.nervanasys.com/about/
nickpsecurity 11y ago
Basically, you need hardware designers and a ton of money to develop ASIC's that will accelerate as good as current GPU's. I've seen designs along lines of SIMD's, DSP's, etc that might be a nice alternative in custom hardware. Still tons of money to get them to work. Doesn't even count developing compilers and such to utilize them properly. Existing tools on GPU's are easier despite me wanting more work on FPGA's and ASIC's in this area. ;)
michael_h 11y ago
I was heavily into neuromorphic hardware ~5 years ago when the hype seemed to be at its peak. We funneled a load of time and money into it. It was working okay for what we were doing and everybody was generally happy. Then the NVidia Tesla C2050 came out and CUDA reached a somewhat stable release. The neuromorphic plane has crashed into the proverbial mountain. EDIT: Maybe I should elaborate - we accomplished the same task using the Tesla card, but it was about 100x faster and each card was $2500 and usable for other tasks. The general rule of thumb became: design a neuromorphic system, wait for the next Tesla chip, then simulate it in software.

How come they aim for GPU technology rather than for neuromorphic computing ASICs?

by sawwit 11y ago

How come they aim for GPU technology rather than for neuromorphic computing ASICs?

Just a guess, but they project the cost of large scale GPU manufacturing to decline faster than ASIC due to the shared applications of gaming/VR?

blackguardx 11y ago
You would think AI would be bigger than VR. Edit: going with GPUs makes sense now, but it doesn't indicate a big bet on Facebook's part. I don't think opensourcing a server cabinet is that interesting, but I'm a hardware engineer. In 10 years I think more people will touch AI than VR but I don't know how that translates to investment.
- unchocked 11y ago
  Gaming is a proven market. Capital hates risk. (Real capital, not the fringe type we talk about here.)
- argonaut 11y ago
  AI might be bigger than VR, but the timeframes are messed up. VR is an engineering problem now; I don't think there are major field-shattering breakthroughs needed to make reasonably realistic, cheap VR within the next 5 years. AI is nowhere close to strong AI or reasonable general-purpose intelligent programs that don't require loads of human tuning. It will take several breakthroughs.

vonnik 11y ago

Which neuromorphic ASICs are you thinking of? GPUs are fast and they work now. Some "neuromorphic" chips, like IBM's True North, are just low-energy GPUs, so neuromorphic is in the eyes of the beholder...

emcq 11y ago
This is not very accurate. The true North chip places memory next to compute elements on chip. GPUs have most memory off chip. The true North hardware is specialized to neural net stuff, and doesn't allow general floating point compute, have any sort of instruction set decoder, etc. That turns into big performance and power wins.

modeless 11y ago

Contrary to what you might read in puff pieces from IBM PR, all existing "neuromorphic" hardware is terrible. The neural nets that work don't look anything like the brain, and neural nets that look like the brain don't work. People constructing neural nets that look like the brain are engaged in cargo cult science, because the truth is we have no idea yet how the brain works and attempting to imitate it without knowing that is doomed to failure.

dangirsh 11y ago
I'm curious to hear your opinions on Numenta's work on emulating parts of the neocortex.
- howlin 11y ago
  He just gave it to you. Their system is not really competitive in any metric people are measuring.
- chimtim 11y ago
  A good measure of any of the techniques are the results. We see record object recognition and speech recognition results. How many of these results are from Numenta or IBM neomorphic chips? Most successes have been on deep learning architectures over GPUs (and large datasets). While these fancy architectures may have their applications, they have nothing to do with recent advancements in last 3 years.
  - emcq 11y ago
    To be fair, the goal of those new computing architectures is not to advance algorithms. That at best is a side benefit. Those algorithmic performance improvements seen in DNN have come from improved datasets and training. Numenta and IBM arent focusing on training AFAIK. Google's Quantum Annealing [0] is the only hardware I'm aware of focused on training, although there are rumors Nervana Systems may produce something [1]. I'm sure there are others; accelerating training of DNN isnt a particularly new idea. The goal of these other computing architectures is typically to provide lower power, higher frequency/lower latency, or smaller form factor execution of trained models, but there is a question of how much value they can provide over more conventional chips to be worth the chip design costs. However without these architectures becoming as mainstream as say a GPU, I think we will continue to see advances come from the typical everyday computer. The ML community seems to be much more democratic than others. [0] http://googleresearch.blogspot.com/2015/12/when-can-quantum-annealing-win.html http://googleresearch.blogspot.com/2015/12/when-can-quantum-... [1] http://www.nervanasys.com/about/ http://www.nervanasys.com/about/

nickpsecurity 11y ago

Basically, you need hardware designers and a ton of money to develop ASIC's that will accelerate as good as current GPU's. I've seen designs along lines of SIMD's, DSP's, etc that might be a nice alternative in custom hardware. Still tons of money to get them to work. Doesn't even count developing compilers and such to utilize them properly. Existing tools on GPU's are easier despite me wanting more work on FPGA's and ASIC's in this area. ;)

michael_h 11y ago

I was heavily into neuromorphic hardware ~5 years ago when the hype seemed to be at its peak. We funneled a load of time and money into it. It was working okay for what we were doing and everybody was generally happy. Then the NVidia Tesla C2050 came out and CUDA reached a somewhat stable release. The neuromorphic plane has crashed into the proverbial mountain. EDIT: Maybe I should elaborate - we accomplished the same task using the Tesla card, but it was about 100x faster and each card was $2500 and usable for other tasks. The general rule of thumb became: design a neuromorphic system, wait for the next Tesla chip, then simulate it in software.