4 ms·
I wish someone could take this simple example and add GPU support (and how that can be accomplished best OpenCL/CUDA etc.) just so one could understand how the
by bullen 6y ago
I wish someone could take this simple example and add GPU support (and how that can be accomplished best OpenCL/CUDA etc.) just so one could understand how the GPU actually helps accelerate things when the amount of neurons/layers grows?
Edit: A bit sad that OpenCL does not work on Jetson Nano and I can't find anything recent that allows Java to run CUDA!? So we're back to AMD?
Also "In the next post we will see if adding another layer to our neural network can help in improving the predictions ;)" but there is no next post even after 4 years!
- suyash 6y agoLook up project - JCUDA and this article if you're interested in learning more about that https://blogs.oracle.com/javamagazine/programming-the-gpu-in-java https://blogs.oracle.com/javamagazine/programming-the-gpu-in...
- bullen 6y agoThx, after some more investigating I decided to write my own CPU stuff first then maybe add OpenGL compute shaders if I need them...
- suyash 6y agoOk, but these are open source projects that companies like Oracle and NVIDIA are workin on which is awesome.
- bullen 6y agoCUDA is not so open-source... and the glue between Java and CUDA seems tricky!
- suyash 6y agoI mentioned JCUDA (Java Bindings for CUDA) which is an open source project https://github.com/jcuda https://github.com/jcuda Here is the tutorial : http://www.jcuda.org/tutorial/TutorialIndex.html http://www.jcuda.org/tutorial/TutorialIndex.html
- OkayPhysicist 6y agoThe math involved turns out to be really simple linear algebra. Matrix multiplication is an embarrassingly parallel task.
- banachtarski 6y ago> Matrix multiplication is an embarrassingly parallel task. (GPU programmer take). This is a common misconception. In reality, matrix multiplication is _extremely tricky_ to implement a fast parallel algorithm for because the number of useful instructions you can perform per byte is limited and you end up getting memory bound very quickly, so you start needing to think about properly employing LDS, managing bank conflicts, etc. For many applications that actually need to run fast, matrices are great for sorting out the theory, but then it's a good time to come up with something that will actually run fast after the paper problem is sorted out.
- stingraycharles 6y agoIs this why my memory consumption explodes when I increase the batch size of my inputs? Sorry I’m a relative GPU noob, but I find it fascinating to observe some higher level interactions with eg pytorch; it seems like the GPU is a very leaky distraction, and you quickly find yourself battling GPU memory management, truncating input sizes and whatnot in order to deal with the insane memory requirements.
- banachtarski 6y agoBatch size is an orthogonal concept. The reason your memory increases so much is because with larger batches, you need to store more intermediate results before running your optimizer. This would be true with a pure CPU implementation of an SGD (stochastic gradient descent) optimizer as well.
- 37ef_ced3 6y agoIt's not very relevant, but here is an open source code generator for "from scratch" x86-64 SIMD convolutional neural nets in C99: https://nn-512.com/ https://nn-512.com/ The C99 code is stand-alone, with no external dependencies, and there are numerous example blocks to look at. The algorithms are state-of-the-art (multi-tile Winograd-Cook-Toom-Lavin, multi-tile strided Fourier, arbitrary dilated/strided convolutions with no im2col redundancy, etc.)