5 ms·

People are making fun of facebook/google for expanding beyond core competency, but this is a market (AI training / inference) where Nvidia has a very real monop

People are making fun of facebook/google for expanding beyond core competency, but this is a market (AI training / inference) where Nvidia has a very real monopoly that facebook and google must rely on for core competency products, so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer and partnering with TSMC (or whoever) to make training and inference chips. It's absolutely the right decision today, and if there was one viable alternative to Nvidia (which AMD and Intel are not), it wouldn't be necessary.

hoodoof 9y ago
I don't agree that because the primary supplier is a monopoly is justification for doing something as distracting and risky as designing your own chips. So they need a second supplier? Do what the government do, and demand that there be a second source https://en.wikipedia.org/wiki/Second_source https://en.wikipedia.org/wiki/Second_source or you won't buy their product, or give money to some other company or companies for whom that is a core competency. >> so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer You say this with great certainty, that almost got me on board, but saying something with certainty does not make it right. It's not critically important - what's the worst that can happen in the short term - they pay a bit more because of the monopoly? You make it sound like if the chip monopoly isn't broken then doom will befall them all. Piffle. If there is such great demand for these chips then there are plenty of other silicon companies who can be attracted by the smell of demand. What would make sense is for Facebook to claim to be making their own to strengthen their negotiating position with the supplier, but with no real plan to design their own chip.
- wslh 9y ago
  > ...doing something as distracting and risky as designing your own chips. This is what you do when your company is a cash cow, you earn so much money that even reinvesting it in the core business doesn't impact your always growing revenue and profits. Also, it is refreshing that new companies are entering the classical semiconductors market, lots of positive externalities.
- noselasd 9y ago
  Why is designing custom chips risky and distracting ? It seems to have worked pretty ok for e.g. google and amazon.
- pm90 9y ago
  Apple designed its own ARM chips and they design some pretty damn good ones. More competition and choice in the semiconductor space is universally a good thing.
  - yndoendo 9y ago
    Can you actually purchase an Apple ARM SOC or make your own SOC with their processor for product development? If not, then I wouldn't classify them has a competitor since they only allow for internal usage.
opencl 9y ago
The entire reason that AMD hardware isn't considered a viable alternative to NVIDIA is that it doesn't run CUDA, which is going to be true of these custom chips too. The chips FB are working on are probably far more similar to Google's TPUs than anything NVIDIA makes though.
- flamedoge 9y ago
  So the problem is programmability
- Symmetry 9y ago
  For researchers, yes. At the scale Facebook and Google operate at the lack of CUDA isn't much of a problem. The big difference is that AMD's GPUs only work on vectors whereas Google's TPUs (and NVidia's latest card) can work on matrices directly for a large increase in throughput.
  - p1esk 9y ago
    Most DL researchers don't care about CUDA, they use higher level frameworks (Tensorflow, Pytorch, Caffe, etc). The frameworks can be extended to support other backends (e.g. OpenCL). If someone builds their own hardware, provides integrated support for that hardware in Tensorflow, and demonstrates speed advantages, people will use it.
- danielh96 9y ago
  Why do you say that? Because they already have a large code base that would be too much work to move over? Or superior features in CUDA? This is actually related to some ongoing research for me, be very interested in any insight.
  - p1esk 9y ago
    Because there's no compelling reason not to use CUDA.
- tsomctl 9y ago
  This is something that I've wondered about. There is nothing about Tensorflow (for example) that is specific to CUDA. Why doesn't AMD dedicate a couple of engineers to port it? They don't even have to do a complete job. Only about half of the Tensorflow operations are used for most models.
arbuge 9y ago
The other factor at play is that Nvidia chips were originally designed for graphics processing tasks, which may not be the best architecture for AI.
- twtw 9y ago
  ... and were completely redesigned starting in the mid-2000s to be good general purpose parallel processors. Modern GPUs have very little specialized graphics hardware. This doesn't mean there aren't better architectures, but this idea that it's bad cause it's a graphics accelerator is horribly out of date.
- deepnotderp 9y ago
  A widely propagated myth. GPUs were largely programmable parallel processors by the time they became in vogue for deep learning. Bill Dally (nVidia chief scientist) has said that the hard wired graphics takes up so little die area that it is essentially negligible in terms of cost for them. Plus, the latest Volta chips have "tensor cores" the equivalent of little TPUs.
jacksmith21006 9y ago
What? Google has their TPUs to do their AI work. On the device they now have their PVC. https://www.wired.com/2017/04/building-ai-chip-saved-google-building-dozen-new-data-centers/ https://www.wired.com/2017/04/building-ai-chip-saved-google-... Google's TPU Chip Helped It Avoid Building Dozens of New Data ...
justicezyx 9y ago
I am not sure many still argue core compute is not gf's core competency. If there are still many, I'd say they did a good job of hiding core competency.
shmerl 9y ago
They can ditch CUDA, and start using AMD in addition. It's still not a huge increase in choice, but better than one.
- twtw 9y ago
  Ditching CUDA would mean accepting at least 10-30% performance hit on NVIDIA cards and losing the ability to use the newest NVIDIA improvements. OpenCL doesn't know about HMMA (aka "Tensor Cores"), so it would be about an order of magnitude slower for DL on Volta. I think you'd be hard pressed to find people willing trade significant performance for the option of running applications on AMD's less power efficient processors.
  - shmerl 9y ago
    Nvidia should provide Vulkan extensions to access those I suppose. So they should still be accessible. There is some plan to converge Vulkan and OpenCL into one coherent combination.

hoodoof 9y ago

I don't agree that because the primary supplier is a monopoly is justification for doing something as distracting and risky as designing your own chips. So they need a second supplier? Do what the government do, and demand that there be a second source https://en.wikipedia.org/wiki/Second_source https://en.wikipedia.org/wiki/Second_source or you won't buy their product, or give money to some other company or companies for whom that is a core competency. >> so it's critically important that one or both of them break the monopoly by becoming a fabless chip designer You say this with great certainty, that almost got me on board, but saying something with certainty does not make it right. It's not critically important - what's the worst that can happen in the short term - they pay a bit more because of the monopoly? You make it sound like if the chip monopoly isn't broken then doom will befall them all. Piffle. If there is such great demand for these chips then there are plenty of other silicon companies who can be attracted by the smell of demand. What would make sense is for Facebook to claim to be making their own to strengthen their negotiating position with the supplier, but with no real plan to design their own chip.

wslh 9y ago
> ...doing something as distracting and risky as designing your own chips. This is what you do when your company is a cash cow, you earn so much money that even reinvesting it in the core business doesn't impact your always growing revenue and profits. Also, it is refreshing that new companies are entering the classical semiconductors market, lots of positive externalities.
noselasd 9y ago
Why is designing custom chips risky and distracting ? It seems to have worked pretty ok for e.g. google and amazon.
pm90 9y ago
Apple designed its own ARM chips and they design some pretty damn good ones. More competition and choice in the semiconductor space is universally a good thing.
- yndoendo 9y ago
  Can you actually purchase an Apple ARM SOC or make your own SOC with their processor for product development? If not, then I wouldn't classify them has a competitor since they only allow for internal usage.

opencl 9y ago

The entire reason that AMD hardware isn't considered a viable alternative to NVIDIA is that it doesn't run CUDA, which is going to be true of these custom chips too. The chips FB are working on are probably far more similar to Google's TPUs than anything NVIDIA makes though.

flamedoge 9y ago
So the problem is programmability
Symmetry 9y ago
For researchers, yes. At the scale Facebook and Google operate at the lack of CUDA isn't much of a problem. The big difference is that AMD's GPUs only work on vectors whereas Google's TPUs (and NVidia's latest card) can work on matrices directly for a large increase in throughput.
- p1esk 9y ago
  Most DL researchers don't care about CUDA, they use higher level frameworks (Tensorflow, Pytorch, Caffe, etc). The frameworks can be extended to support other backends (e.g. OpenCL). If someone builds their own hardware, provides integrated support for that hardware in Tensorflow, and demonstrates speed advantages, people will use it.
danielh96 9y ago
Why do you say that? Because they already have a large code base that would be too much work to move over? Or superior features in CUDA? This is actually related to some ongoing research for me, be very interested in any insight.
- p1esk 9y ago
  Because there's no compelling reason not to use CUDA.
tsomctl 9y ago
This is something that I've wondered about. There is nothing about Tensorflow (for example) that is specific to CUDA. Why doesn't AMD dedicate a couple of engineers to port it? They don't even have to do a complete job. Only about half of the Tensorflow operations are used for most models.

arbuge 9y ago

The other factor at play is that Nvidia chips were originally designed for graphics processing tasks, which may not be the best architecture for AI.

twtw 9y ago
... and were completely redesigned starting in the mid-2000s to be good general purpose parallel processors. Modern GPUs have very little specialized graphics hardware. This doesn't mean there aren't better architectures, but this idea that it's bad cause it's a graphics accelerator is horribly out of date.
deepnotderp 9y ago
A widely propagated myth. GPUs were largely programmable parallel processors by the time they became in vogue for deep learning. Bill Dally (nVidia chief scientist) has said that the hard wired graphics takes up so little die area that it is essentially negligible in terms of cost for them. Plus, the latest Volta chips have "tensor cores" the equivalent of little TPUs.

jacksmith21006 9y ago

What? Google has their TPUs to do their AI work. On the device they now have their PVC. https://www.wired.com/2017/04/building-ai-chip-saved-google-building-dozen-new-data-centers/ https://www.wired.com/2017/04/building-ai-chip-saved-google-... Google's TPU Chip Helped It Avoid Building Dozens of New Data ...

justicezyx 9y ago

I am not sure many still argue core compute is not gf's core competency. If there are still many, I'd say they did a good job of hiding core competency.

shmerl 9y ago

They can ditch CUDA, and start using AMD in addition. It's still not a huge increase in choice, but better than one.

twtw 9y ago
Ditching CUDA would mean accepting at least 10-30% performance hit on NVIDIA cards and losing the ability to use the newest NVIDIA improvements. OpenCL doesn't know about HMMA (aka "Tensor Cores"), so it would be about an order of magnitude slower for DL on Volta. I think you'd be hard pressed to find people willing trade significant performance for the option of running applications on AMD's less power efficient processors.
- shmerl 9y ago
  Nvidia should provide Vulkan extensions to access those I suppose. So they should still be accessible. There is some plan to converge Vulkan and OpenCL into one coherent combination.