4 ms·
There are zero DL frameworks that support AMD cards as a primary target. Most have some kind of branch or patchset with OpenCL support. The problem is that the
by nl 9y ago
There are zero DL frameworks that support AMD cards as a primary target.
Most have some kind of branch or patchset with OpenCL support. The problem is that they aren't great. If you need any new layers there is no support. There is nothing like CuDNN so you don't get the high speed convolutional kernels.
It's great to blame developers for supporting NVidia, but the thing is NVidia are great to work with. They dedicate large teams to deep learning support (not like the 2 or 3 part time Devs AMD does), and they publish good research and tutorials. AMD does nothing like this.
- tgtweak 9y agoAmd had always had first class support for openCL, cuda is Nvidia proprietary and although Nvidia "supports" OpenCL, it's quite bad. Issue #22 on tensorflow is regarding openCL support. AMD has done some interesting work on HCC (a more proper gpgpu compiler approach with llvm base) and that is showing promise. See here: https://instinct.radeon.com/wp-content/uploads/sites/4/2017/11/1-848x700.jpg https://instinct.radeon.com/wp-content/uploads/sites/4/2017/... Additionally, they support cuda decompiling to HIP which is an intermediary that can be built to target Nvidia (via nvcc) or amd (via HCC). Nvidia has built a lot of tooling for DL, such as cudnn, and the new Tesla cards have dedicated silicon for tensor calculation. Amd does have a cudnn equivalent called MlOpen. They have also ported caffe via HIP and it works well. Work is being done by amd right now to torch, mxnet and tensorflow to add support for amd hardware with minimal burden to the maintainers of these projects. You can read about some of the DL toolkits available here: https://instinct.radeon.com/en/6-deep-learning-projects-amd-radeon-instinct/ https://instinct.radeon.com/en/6-deep-learning-projects-amd-... I think it's particularly bad form on behalf of everyone in the DL framework and library world to cator only to Nvidia and cuda, and that they very much walked into this shake down with open arms. The original comment is correct in that contributing support for OpenCL (which works on mobile too) will alleviate this to a fair degree. It's one of those things where the more momentum is behind it, the more device manufactures will focus on ensuring their opencl compiler is building properly optimized kernels for their hardware. Start contributing to OpenCL or adding hip support to existing projects and we'll see some viable alternatives pop up from not only AMD, but players like Qualcomm and Samsung.
- nl 9y agoOpenCL doesn't have an equivalent to CuDNN. I'm not saying it's a great situation, I'm saying that NVidia has always had better libraries, tools and performance, and it isn't surprising that developers use them. Deep learning is hard and slow enough without using second class tools.
- tgtweak 9y ago> Amd does have a cudnn equivalent called MlOpen.
- nl 9y agoMIOpen could be great one day. ATM you still get random problems like this: https://github.com/ROCmSoftwarePlatform/MIOpen/issues/19 https://github.com/ROCmSoftwarePlatform/MIOpen/issues/19 That's an order of magnitude worse performance than NVidia on ResNet 52 for 2 months with no real reason. Great idea, but no one can use it reliably yet.
- hedgehog 9y agoMuch respect to NVIDIA and their software team but the situation is changing. PlaidML is like cuDNN for every GPU. Fully open source, faster in many cases than TF+cuDNN on NVIDIA, beats vendor tools on other architectures, Linux/Mac/Win. Supports Keras currently but more frameworks are not difficult (patches welcome).
- dnautics 9y agoapparently Caffe has support for AMD cards through ROCm https://rocm.github.io/dl.html https://rocm.github.io/dl.html
- nl 9y agoThat's exactly what I mean. It's a patchset (note that it isn't upstreamed in their chart), and it's caffe. That was great in 2015.