3 ms·
Fun fact: Long before the dawn of the GPU deep learning hype (and even before CUDA was a thing), a bunch of CS nerds from Korea managed to train a neural networ
by sigmoid10 1y ago
Fun fact: Long before the dawn of the GPU deep learning hype (and even before CUDA was a thing), a bunch of CS nerds from Korea managed to train a neural network on an ATI (now AMD) Radeon 9700 Pro using nothing but shaders [1]. They saw an even bigger performance improvement than Hinton and his group did for AlexNet 8 years later using CUDA.
[1] https://ui.adsabs.harvard.edu/abs/2004PatRe..37.1311O/abstract https://ui.adsabs.harvard.edu/abs/2004PatRe..37.1311O/abstra...
- math_dandy 1y agoCool, I had not heard about this. Adding this paper to my machine learning teaching bibliography. Even though the start of the deep learning renaissance is typically dated to 2012 with Alexnet, things were in motion week before that. As you point out, GPU training was validated at least 8 years previously. Concurrently, some very prescient researchers like Li were working hard to generate large scale datasets like ImageNet (CVPR 2009, https://www.image-net.org/static_files/papers/imagenet_cvpr09.pdf https://www.image-net.org/static_files/papers/imagenet_cvpr0...). And in 2012 it all came together.
- sigmoid10 1y agoSince shaders were designed for, well, shading, this early experiment was more of an academic playground exercise than useful research. But AlexNet still wasn't the first deep neural network trained using CUDA. It had already been done three years earlier: https://dl.acm.org/doi/10.1145/1553374.1553486 https://dl.acm.org/doi/10.1145/1553374.1553486 The ImageNet competition had also been around since 2010. So the ingredients were actually all there before.