6 ms·

Apparently tensorflow is quite close to supporting opencl, and now that nvidia put a 10x multiple on their cards for ML use you can be sure that there will be a

by tensor 9y ago

Apparently tensorflow is quite close to supporting opencl, and now that nvidia put a 10x multiple on their cards for ML use you can be sure that there will be an army of people making sure that opencl becomes the standard.

andars 9y ago
Just curious, what cards are you comparing when referring to the 10x multiple?
agibsonccc 9y ago
Just going to say, just because tensorflow supports it won't mean much. The ecosystem as a whole is still largely cuda. This includes the database tech(kinetica,mapd), the resource managers (mesos/yarn), not to mention HPC. There's also just not much incentive for people to move to AMD here. There's a reason NVIDIA gives out these graphics cards like candy to academia. It's because they want to make their margins in data center while keeping the broader developer community locked in to cuda. People on HN make broad and sweeping comments about "if you open it they will come and everything will be magically better". It's a lot more complicated than that. The market incentives just aren't quite there yet. Could it happen one day? Yes. Will it happen today? No. It's going to take a lot more than this for other vendors to start supporting AMD. What I will say: There will be competition and the space is heating up. AMD is one player. Now let me put my commercial hat on here: What would it take for me as a deep learning vendor to support/care about AMD? 1. Show me the money. I need a clear revenue stream. AMD has some hope here. Customers don't care about which gpu they use as long as it fulfills a use case they care about. 2. Code: Show me something that's actually a robust. A 1 off fork of any deep learning framework (see the random 1 off caffe forks that aren't actually caffe by nvidia, intel,amd) 3. Share the burden. Put the code out there and support the broader community. Actually maintain and follow up with the latest innovations (hint: this is hard. throwing code over the wall once doesn't mean anything) 4. Amazon will be key here. Google cloud (look google is great but they aren't the leading cloud player by a long shot and likely won't be anytime soon) - get them on board with some AMD cards. 5. (Disclaimer I know the founders) - make opencl not suck. https://vertex.ai/ https://vertex.ai/ is an interesting player in this space. So look, I won't say it's impossible. Let's just not ignore the actual commercial market forces that are also at play here.
- notyourday 9y ago
  There used to be this gigantic company that ruled networking world. Its name was Cisco. No one could touch it. Costs for a basic series XX with 4 slots was N, they charged X, for 6 slots, it was N+20, they charged 4X, for 16 slots it was N+200, they charged 30X. What is Cisco is the current question.
  - agibsonccc 9y ago
    Nowhere did I say it's never going to happen. These things take time to evolve. I even talked about the market forces and what's currently plausible. There's always big companies that get disrupted because they kill good will. That's the start to something. It takes more than 1 move. Will NVIDIA make this mistake over time? Likely. Will it be AMD and Tensorflow support that does it? Not by a long shot. It will be by a series of players providing the right incentives. My response (nor any response to this) should not be treated as binary. Ultimately in order for a company to be "disrupted" you need to actually analyze and exploit the market forces involved here. just writing this off as "open ecosystem is all we need" is dangerous.
    - jamesblonde 9y ago
      I have to say I disagree with you on the AMD/TensorFlow point here, Adam. Your previous points are completely valid. Nothing of note will happen in 2018, I expect. But by early 2019, maybe - if AMD (or Intel) gets their act together on the software side of things. I don't think the "community" will do it for them. Buy maybe a big player like Amazon will have had enough of Nvidia and support an open alternative. Although we support Cuda in our version of YARN (Hops Hadoop), I expect we'd add ROCm or OpenCL or whatever - if it was a serious alternative. That would happen quickly. The problem, of course, is that we would want great support in TensorFlow first before we do that. Data scientists need a seamless transition - including from a performance perspective. For us, that also means support for distributed deep learning (Ring AllReduce over infiniband). I don't expect that will happen in 2018, and could take until 2020, if i'm being realistic. That means when AMD finally get some good DL libraries, Nvidia will still have one-up on them with distributed DL (reduce training time linearly with more GPUs). The other wildcard i haven't seen people mention here is the Neural Network Processor (NNP) from Intel Nervana. The hardware has potential. As long as the software doesn't force us to use BigDL, it has potential.

by tensor 9y ago

andars 9y ago

Just curious, what cards are you comparing when referring to the 10x multiple?

agibsonccc 9y ago

Just going to say, just because tensorflow supports it won't mean much. The ecosystem as a whole is still largely cuda. This includes the database tech(kinetica,mapd), the resource managers (mesos/yarn), not to mention HPC. There's also just not much incentive for people to move to AMD here. There's a reason NVIDIA gives out these graphics cards like candy to academia. It's because they want to make their margins in data center while keeping the broader developer community locked in to cuda. People on HN make broad and sweeping comments about "if you open it they will come and everything will be magically better". It's a lot more complicated than that. The market incentives just aren't quite there yet. Could it happen one day? Yes. Will it happen today? No. It's going to take a lot more than this for other vendors to start supporting AMD. What I will say: There will be competition and the space is heating up. AMD is one player. Now let me put my commercial hat on here: What would it take for me as a deep learning vendor to support/care about AMD? 1. Show me the money. I need a clear revenue stream. AMD has some hope here. Customers don't care about which gpu they use as long as it fulfills a use case they care about. 2. Code: Show me something that's actually a robust. A 1 off fork of any deep learning framework (see the random 1 off caffe forks that aren't actually caffe by nvidia, intel,amd) 3. Share the burden. Put the code out there and support the broader community. Actually maintain and follow up with the latest innovations (hint: this is hard. throwing code over the wall once doesn't mean anything) 4. Amazon will be key here. Google cloud (look google is great but they aren't the leading cloud player by a long shot and likely won't be anytime soon) - get them on board with some AMD cards. 5. (Disclaimer I know the founders) - make opencl not suck. https://vertex.ai/ https://vertex.ai/ is an interesting player in this space. So look, I won't say it's impossible. Let's just not ignore the actual commercial market forces that are also at play here.

notyourday 9y ago
There used to be this gigantic company that ruled networking world. Its name was Cisco. No one could touch it. Costs for a basic series XX with 4 slots was N, they charged X, for 6 slots, it was N+20, they charged 4X, for 16 slots it was N+200, they charged 30X. What is Cisco is the current question.
- agibsonccc 9y ago
  Nowhere did I say it's never going to happen. These things take time to evolve. I even talked about the market forces and what's currently plausible. There's always big companies that get disrupted because they kill good will. That's the start to something. It takes more than 1 move. Will NVIDIA make this mistake over time? Likely. Will it be AMD and Tensorflow support that does it? Not by a long shot. It will be by a series of players providing the right incentives. My response (nor any response to this) should not be treated as binary. Ultimately in order for a company to be "disrupted" you need to actually analyze and exploit the market forces involved here. just writing this off as "open ecosystem is all we need" is dangerous.
  - jamesblonde 9y ago
    I have to say I disagree with you on the AMD/TensorFlow point here, Adam. Your previous points are completely valid. Nothing of note will happen in 2018, I expect. But by early 2019, maybe - if AMD (or Intel) gets their act together on the software side of things. I don't think the "community" will do it for them. Buy maybe a big player like Amazon will have had enough of Nvidia and support an open alternative. Although we support Cuda in our version of YARN (Hops Hadoop), I expect we'd add ROCm or OpenCL or whatever - if it was a serious alternative. That would happen quickly. The problem, of course, is that we would want great support in TensorFlow first before we do that. Data scientists need a seamless transition - including from a performance perspective. For us, that also means support for distributed deep learning (Ring AllReduce over infiniband). I don't expect that will happen in 2018, and could take until 2020, if i'm being realistic. That means when AMD finally get some good DL libraries, Nvidia will still have one-up on them with distributed DL (reduce training time linearly with more GPUs). The other wildcard i haven't seen people mention here is the Neural Network Processor (NNP) from Intel Nervana. The hardware has potential. As long as the software doesn't force us to use BigDL, it has potential.