4 ms·

Could you elaborate as to why this is a huge opportunity business wise?

mryan 11y ago
In the coming years we will see a lot more applications powered by deep learning. If someone releases a chip that provides 10x performance compared to the current GPU-based method, they will sell a lot of chips.
- x0x0 11y ago
  No doubt -- the problem is the assumption that 10x performance is possible. Though if anyone can deliver it will (imo) be Intel: it will be a process war, and they seem to be able to do more transistors than anyone else.
  - azinman2 11y ago
    Or IBM: http://research.ibm.com/cognitive-computing/neurosynaptic-chips.shtml#fbid=YUVZuhPyPsg http://research.ibm.com/cognitive-computing/neurosynaptic-ch...
  - maaku 11y ago
    Intel may have to find a different niche. Their Xeon Phi approach to parallelism is very interesting, speaking as someone dabbling in AGI, but is not comparably well suited for deep learning.
- azeirah 11y ago
  How well suit are fpga's for deep learning?
  - amund 11y ago
    https://gigaom.com/2015/02/23/microsoft-is-building-fast-low-power-neural-networks-with-fpgas/ https://gigaom.com/2015/02/23/microsoft-is-building-fast-low...
  - Thimothy 11y ago
    The answer to the question "How well suited are FPGAs for -insert field where GPUs or vanilla processors do no excel at-?" is always "Far better than the GPUs or proccessors but with an abysmal power usage". In this kind of new developments FPGAs are usually used as a test before moving to ASICs.
maaku 11y ago
Basically everything you interact with on a daily basis involves (deep) machine learning. Everything from advertising to logistics to drug discovery to credit evaluations to packet routing on the internet either involves deep learning in its operation, or is presently implemented with some optimal strategy that was discovered via deep learning. Machine learning runs the world. As for the business opportunity, if you are in one of these industries then one of the prime ways you differentiate from your competitors is how well your algorithm works. And how well it works depends a great deal on how much training data you are able to feed into it, which in this era of information saturation is basically a hardware limitation. Give me a 10x more powerful machine learning system, and I'll give you a few basis points advantage over the competition, and that'll make you, not them, the dominant player.
- hueving 11y ago
  Packet routing? If there is an example of this, it is very far from the norm. Routing on the Internet is driven first and foremost by peering arrangements.
  - maaku 11y ago
    Now ask yourself how decisions are made regarding new infrastructure deployments. But what I actually had in mind was QoS.
    - mryan 11y ago
      Who is using deep learning for packet routing and QoS decisions?
      - pilooch 11y ago
        No one I know of, from the last time I did check on this topic. However, it seems possible to plug ML on top of something like OpenFlow. PM me if interested to look this up further.
        11y ago
modeless 11y ago
Deep learning has already completely taken over speech recognition, face recognition, and object recognition. Going forward deep learning is the technology that will solve sensing and perception for machines. You're going to want sensing and perception in your phone and every computer you use, but even more than that: every drone, every self-driving car, every kind of robot is going to need multiple deep learning chips. Each of those chips is going to need vastly more FLOPS and more memory bandwidth than the biggest CPUs and GPUs of today. Beyond sensing and perception, I believe that deep learning will also be the technology that solves planning, natural language, reasoning, and creativity for machines. This is much more speculative, but you can see the beginnings of planning in the DeepMind Atari work, the beginnings of natural language processing and reasoning in machine translation and various question answering systems, and the beginnings of creativity in Deep Dream and other generative models people have done. Of course once all those pieces are solved then AI is solved. The market for a technology that solves AI is practically unlimited. Intel is constantly searching for the next big application that will require more processing power so people need to buy faster CPUs and they can justify spending $X billion on their next fab. In recent years they've been struggling to find it. Deep learning is it. The appetite for FLOPS and memory bandwidth is unbounded for the foreseeable future. Unfortunately for Intel, CPUs are weak on both compared to GPUs. Maybe Xeon Phi can morph into a deep learning system?
- lucidrains 11y ago
  I am similarly very bullish on this field, even for what I once thought were incredible claims that it can eventually understand the semantics behind documents, do question and answering, and even reason. What blew my mind was this lecture, given by Hinton https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc/view https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc... The main idea is that reasoning is just a sequence of 'thought vectors' that can be encoded within a recurrent neural network. Sounded almost outlandish to me until I watched Richard Socher's lectures and started to understand that words can be represented as vectors, and that these vectors can be then encoded into new vectors of even higher representation. 'Thought vectors' may not be so outlandish after all.
  - 11y ago

mryan 11y ago

In the coming years we will see a lot more applications powered by deep learning. If someone releases a chip that provides 10x performance compared to the current GPU-based method, they will sell a lot of chips.

x0x0 11y ago
No doubt -- the problem is the assumption that 10x performance is possible. Though if anyone can deliver it will (imo) be Intel: it will be a process war, and they seem to be able to do more transistors than anyone else.
- azinman2 11y ago
  Or IBM: http://research.ibm.com/cognitive-computing/neurosynaptic-chips.shtml#fbid=YUVZuhPyPsg http://research.ibm.com/cognitive-computing/neurosynaptic-ch...
- maaku 11y ago
  Intel may have to find a different niche. Their Xeon Phi approach to parallelism is very interesting, speaking as someone dabbling in AGI, but is not comparably well suited for deep learning.
azeirah 11y ago
How well suit are fpga's for deep learning?
- amund 11y ago
  https://gigaom.com/2015/02/23/microsoft-is-building-fast-low-power-neural-networks-with-fpgas/ https://gigaom.com/2015/02/23/microsoft-is-building-fast-low...
- Thimothy 11y ago
  The answer to the question "How well suited are FPGAs for -insert field where GPUs or vanilla processors do no excel at-?" is always "Far better than the GPUs or proccessors but with an abysmal power usage". In this kind of new developments FPGAs are usually used as a test before moving to ASICs.

maaku 11y ago

Basically everything you interact with on a daily basis involves (deep) machine learning. Everything from advertising to logistics to drug discovery to credit evaluations to packet routing on the internet either involves deep learning in its operation, or is presently implemented with some optimal strategy that was discovered via deep learning. Machine learning runs the world. As for the business opportunity, if you are in one of these industries then one of the prime ways you differentiate from your competitors is how well your algorithm works. And how well it works depends a great deal on how much training data you are able to feed into it, which in this era of information saturation is basically a hardware limitation. Give me a 10x more powerful machine learning system, and I'll give you a few basis points advantage over the competition, and that'll make you, not them, the dominant player.

hueving 11y ago
Packet routing? If there is an example of this, it is very far from the norm. Routing on the Internet is driven first and foremost by peering arrangements.
- maaku 11y ago
  Now ask yourself how decisions are made regarding new infrastructure deployments. But what I actually had in mind was QoS.
  - mryan 11y ago
    Who is using deep learning for packet routing and QoS decisions?
    - pilooch 11y ago
      No one I know of, from the last time I did check on this topic. However, it seems possible to plug ML on top of something like OpenFlow. PM me if interested to look this up further.
      - 11y ago

modeless 11y ago

Deep learning has already completely taken over speech recognition, face recognition, and object recognition. Going forward deep learning is the technology that will solve sensing and perception for machines. You're going to want sensing and perception in your phone and every computer you use, but even more than that: every drone, every self-driving car, every kind of robot is going to need multiple deep learning chips. Each of those chips is going to need vastly more FLOPS and more memory bandwidth than the biggest CPUs and GPUs of today. Beyond sensing and perception, I believe that deep learning will also be the technology that solves planning, natural language, reasoning, and creativity for machines. This is much more speculative, but you can see the beginnings of planning in the DeepMind Atari work, the beginnings of natural language processing and reasoning in machine translation and various question answering systems, and the beginnings of creativity in Deep Dream and other generative models people have done. Of course once all those pieces are solved then AI is solved. The market for a technology that solves AI is practically unlimited. Intel is constantly searching for the next big application that will require more processing power so people need to buy faster CPUs and they can justify spending $X billion on their next fab. In recent years they've been struggling to find it. Deep learning is it. The appetite for FLOPS and memory bandwidth is unbounded for the foreseeable future. Unfortunately for Intel, CPUs are weak on both compared to GPUs. Maybe Xeon Phi can morph into a deep learning system?

lucidrains 11y ago
I am similarly very bullish on this field, even for what I once thought were incredible claims that it can eventually understand the semantics behind documents, do question and answering, and even reason. What blew my mind was this lecture, given by Hinton https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc/view https://drive.google.com/file/d/0B8i61jl8OE3XdHRCSkV1VFNqTWc... The main idea is that reasoning is just a sequence of 'thought vectors' that can be encoded within a recurrent neural network. Sounded almost outlandish to me until I watched Richard Socher's lectures and started to understand that words can be represented as vectors, and that these vectors can be then encoded into new vectors of even higher representation. 'Thought vectors' may not be so outlandish after all.
- 11y ago