3 ms·
I was around the space before Alexnet came out. Without NVidia the last 15 years of AI would not have happened. Deep learning was only possible because you cou
by llm_trw 2y ago
I was around the space before Alexnet came out. Without NVidia the last 15 years of AI would not have happened.
Deep learning was only possible because you could do it on NVidia cards with Cuda without having to use the big machines in the basement.
Trying to convince anyone that neural networks could be useful in 2009 was impossible - I got a grant declined and my PhD supervisor told me to drop the useless tech and focus on something better like support vector machines.
- dagmx 2y agoI was around then too and people forget that AMD also had GPU compute APIs. The difference is AMD killed theirs to use OpenCL and NVIDIA kept CUDA around as well.
- llm_trw 2y agoJust like how ROCm is supposed to be competitive today and it isn't unless you have an army of grad students to look after your data center cards. I tried using AMD Stream and it lacked documentation, debugging information and most of the tools needed to get anything done without a large team of experts. NVidia by comparison could - and was - used by single grad students on their franken-stations which we build out of gaming GPUs. The less we talk about the disaster that the move to opencl was the better.
- dagmx 2y agoI agree ROCm was a mess and only recently is even really usable. That said ROCm is quite a recent thing borne out of acknowledging that OpenCL 2 was a disaster. OpenCL 1 had a reasonable shot and was gaining adoption but 2 scuppered it.
- llm_trw 2y agoDid you ever train anything past mnist on AMD? I try every five years or so and promise myself never again each and every time.
- dagmx 2y agoNot with Rocm since I’ve moved my personal stack to NVIDIA (for rendering) and Macs for day to day use. I did write quite a bit of OpenCL prior to that on Intel/AMd/NVIDIA, both for training and for general rendering though, and did some work with Stream before then.
- llm_trw 2y agoWas it OpenCL1? That's the only one I hadn't tried out for AMD GPUs. Everything else I have and can say with absolute certainty that you spend more time fighting the hardware than you did writing code. Cuda by comparison JustWorks^tm.
- dagmx 2y agoBoth 1 and 2. I haven’t done much with 3 as OpenCL is effectively a dead api at this point. 1 was definitely a lot easier to work with than 2. CUDA is easier than both but I don’t think I hit anything I could do in CUDA that I couldn’t do in OpenCL, though CUDA of course had a larger ecosystem of existing libraries.
- llm_trw 2y agoDamn, I missed the best one. Thanks for the trip down memory lane.
- aledalgrande 2y agoI was not a PhD, but I studied both around that period in university and can confirm that neural networks were only seen as a kind of theoretical plaything. All the libraries were pushing SVMs.