3 ms·

You're making a lot of lousy assumptions. As a few points of reference: * My list comprehensions run over gigabytes of data (but sometimes 3 orders of magnitud

by wegs 6y ago

You're making a lot of lousy assumptions. As a few points of reference: * My list comprehensions run over gigabytes of data (but sometimes 3 orders of magnitude bigger or smaller). Stream processing of big data. It's not deep learning, but it's slow and potentially deeply parallel. It would move to MIMD trivially, and SIMD with just a little bit of work. * There are programming languages which support models almost exactly like this for data processing. Sun Labs Fortress comes to mind as an early example. This would generalize to a lot of contexts -- much smaller than you're giving credit for. * Most of the issues, like startup times, are implementation-specific, rather than fundamental, and could be mitigated for much smaller data too. You do need to wrap your head around changes in programming paradigms to make that work. There is some overhead for latency (you'll probably never do well moving a list with 10 items to a GPU), but most of those aren't where programs are performance-bound. * Many database operations map very well to MIMD. * You're making deep assumptions that you're talking to an idiot. That doesn't make you look smart or right, or lead to a constructive discussion.

solidasparagus 6y ago
You're right about my assumptions, I apologize. Without knowing specifically what you do, I can't say if GPUs would make sense. But I don't agree that many workloads could be moved to GPUs cost effectively. It's very hard to feed work in fast enough to keep the GPU busy enough to be cost effective given the limited amount of GPU memory you have to work with.
- wegs 6y ago
  Well, there's a question of whether the GPU has to be fed fast enough. My CPU is nominally rated at around 150-200 gigaflops. My GPU is rated at about 5 teraflops. That's about a 30x speed difference (and there are obviously faster GPUs out there). That's enough to move me from compute-bound to IO-bound and make things a lot faster. Once I'm IO-bound, I'll obviously see no more performance increase, but I figure I'll get a good 5-10x before I get there. Right now, code runs anywhere from a few seconds to overnight, depending on what I'm doing. I'll also mention I'm working on many projects, so that's not overhead I incur every day, just once in a while. Moving to GPU would move that to running anywhere from more-or-less instantly to an hour, I figure, based on similar very back-of-the-envelope benchmarks and guesstimates. That's totally worth dropping $1000 on a new GPU, if that's all it took and things worked out-of-the-box. It'd pay for itself in a few programmer hours. On the other hand, that's totally not worth weeks of programmer / dev-ops time for switching to a proprietary tool chain. An alternative there is to wait for my computation, or to optimize my code. Both of those seem cheaper than maintaining a GPGPU workflow, where GPGPU is right now. If GPGPU came batteries-included in Ubuntu+numpy, it'd be an entirely different story.
nl 6y ago
If this is the workload you are looking at, then you really should look at CuPy. I know you discard this because you don't like the NVidia dependency, but it's not much different to switching a BLAS library or using Intel's numerically optimised Numpy distribution. Your code remains the same, you just change the import and get magic speed. If you still refuse to look at it, then perhaps consider cuBLAS[1], which you can switch out for any other BLAS library (eg [2]). It's one thing that AMD actually has bothered to do and they have version available for CPUs[3] and GPUs[4]. [1] https://developer.nvidia.com/cublas https://developer.nvidia.com/cublas [2] https://towardsdatascience.com/is-your-numpy-optimized-for-speed-c1d2b2ba515 https://towardsdatascience.com/is-your-numpy-optimized-for-s... [3] https://developer.amd.com/amd-aocl/blas-library/ https://developer.amd.com/amd-aocl/blas-library/ [4] https://rocmdocs.amd.com/en/latest/ROCm_Tools/rocblas.html https://rocmdocs.amd.com/en/latest/ROCm_Tools/rocblas.html

by wegs 6y ago

solidasparagus 6y ago

You're right about my assumptions, I apologize. Without knowing specifically what you do, I can't say if GPUs would make sense. But I don't agree that many workloads could be moved to GPUs cost effectively. It's very hard to feed work in fast enough to keep the GPU busy enough to be cost effective given the limited amount of GPU memory you have to work with.

wegs 6y ago
Well, there's a question of whether the GPU has to be fed fast enough. My CPU is nominally rated at around 150-200 gigaflops. My GPU is rated at about 5 teraflops. That's about a 30x speed difference (and there are obviously faster GPUs out there). That's enough to move me from compute-bound to IO-bound and make things a lot faster. Once I'm IO-bound, I'll obviously see no more performance increase, but I figure I'll get a good 5-10x before I get there. Right now, code runs anywhere from a few seconds to overnight, depending on what I'm doing. I'll also mention I'm working on many projects, so that's not overhead I incur every day, just once in a while. Moving to GPU would move that to running anywhere from more-or-less instantly to an hour, I figure, based on similar very back-of-the-envelope benchmarks and guesstimates. That's totally worth dropping $1000 on a new GPU, if that's all it took and things worked out-of-the-box. It'd pay for itself in a few programmer hours. On the other hand, that's totally not worth weeks of programmer / dev-ops time for switching to a proprietary tool chain. An alternative there is to wait for my computation, or to optimize my code. Both of those seem cheaper than maintaining a GPGPU workflow, where GPGPU is right now. If GPGPU came batteries-included in Ubuntu+numpy, it'd be an entirely different story.

nl 6y ago

If this is the workload you are looking at, then you really should look at CuPy. I know you discard this because you don't like the NVidia dependency, but it's not much different to switching a BLAS library or using Intel's numerically optimised Numpy distribution. Your code remains the same, you just change the import and get magic speed. If you still refuse to look at it, then perhaps consider cuBLAS[1], which you can switch out for any other BLAS library (eg [2]). It's one thing that AMD actually has bothered to do and they have version available for CPUs[3] and GPUs[4]. [1] https://developer.nvidia.com/cublas https://developer.nvidia.com/cublas [2] https://towardsdatascience.com/is-your-numpy-optimized-for-speed-c1d2b2ba515 https://towardsdatascience.com/is-your-numpy-optimized-for-s... [3] https://developer.amd.com/amd-aocl/blas-library/ https://developer.amd.com/amd-aocl/blas-library/ [4] https://rocmdocs.amd.com/en/latest/ROCm_Tools/rocblas.html https://rocmdocs.amd.com/en/latest/ROCm_Tools/rocblas.html