3 ms·
- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could ha
by fizixer 7y ago
- Or AMD could change their policy of 'never miss an opportunity to miss an opportunity' and offer high-performance OpenCL GPGPU offerings. Then nVidia could have all the stroke they wanted.
- Or Tensorflow/Pytorch could've crapped on OpenCL a little less by releasing a fully functional OpenCL version everytime they released a fully functional Cuda version, instead of worshipping Cuda year in and year out.
- Or Google could start selling their TPUv2, if not TPUv3, while they're on the verge of releasing TPUv4.
- Or one of the other big-tech's Facebook/Microsoft/Intel could make and start selling a TPU-equivalent device.
- Or I could finish school and get funded to do all/most of the above ;)
edit: On a more serious note, a cloud/on-prem hybrid is absolutely the right way to go. You should have a 4x 2080 ti rig available 24x7 for every ML engineer. It costs about $6k-8k a piece [0]. Prototype the hell out of your models on on-prem hardware. Then when your setup is in working condition and starts producing good results on small problems, you're ready to do a big computation for final model training. Then you send it to the cloud, for final production run. (Guess what, on a majority of your projects, you might realize, the final production run could be carried out on on-prem itself; you just have to keep it running 24 hours-a-day for a few days or up to a couple weeks.)
[0]: https://l7.curtisnorthcutt.com/the-best-4-gpu-deep-learning-rig https://l7.curtisnorthcutt.com/the-best-4-gpu-deep-learning-...
- m0zg 7y agoAs someone who has actually worked on this stuff soup to nuts, it's not as easy as people imagine, because you can't just support some subset of available ops and call it a day. If you want to make OpenCL pie from scratch, you must first make the universe, and support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same". This is hard and time consuming, and this field is hard enough as it is. What makes it even harder is that only NVIDIA has decent, mature tooling. There is some work on ROCM though, so AMD is not _totally_ dead in the water. I'd say they're about 90% dead in the water.
- simonebrunozzi 7y agoAre you in the Bay area? Would love to chat. Thinking of an idea where your expertise could be very handy. $my-hn-username at gmail.
- derefr 7y ago> support every single stupid thing (among thousands) and even mimic some of the bugs so that models work "the same". Do you need to do the stupid things performantly, though? Because that sounds like a case for skipping microcode shims, and going straight to instructions that trap into a software implementation. Or just running the whole compute-kernel in a driver-side software emulator that then compiles real sub-kernels for each stretch of non-stupid GPGPU instructions, uploads those, and then calls into them from the software emulation at the right moments. Like a profile-guided JIT, but one that can't actually JIT everything, only some things.
- fizixer 7y agoI know Tensorflow decided to be cuda-exclusive for the silly reason that the matrix library they were using (eigen) only supported cuda. I have never recovered from that.
- dnautics 7y agoThat article is mostly right, but there's one part that got skimped on that will mess you up big time with about an 20% chance if you run for long enough.