3 ms·

hmmm. this is a little unfair. i've got around a factor of 30 speedup using a cheap GPU (some old nvidia card i had in my machine) compared to a fairly meaty

hmmm. this is a little unfair. i've got around a factor of 30 speedup using a cheap GPU (some old nvidia card i had in my machine) compared to a fairly meaty xeon at work. the important difference is that the kernel (written in opencl) was much more than just "taking the sine". once the data are on the gpu you don't want to move them back until you've done as much work as possible. and there are different "kinds" of memory on the gpu. simply taking the sine of every value in an array is not using that memory efficiently (you're not using the equivalent of the gpu's L2 cache (and is cpu cache explaining the increased speed on the cpu?)). so this would be io limited - all the test is doing is comparing memory transfer to the cpu with memory transfer to the gpu, and coming up with similar numbers. i guess the fairest thing you can say is that if you're not going to put the effort into understanding how best to use gpus, leave them alone. (incidentally, unless you need 64bit fp, or are really limited by pci slots, it makes much more sense to buy commodity graphics cards rather than the new top-end fermi made-for-compute cards - for us switching from an "old" graphics card to a c2050 gave an extra 20% speedup, iirc, which is peanuts compared to the factor of 30 from moving off the cpu; also, nvidia support for the expensive cards sucks, so opencl is worth the extra effort for the peace of mind that you can try something else)

StavrosK 15y ago
Not only that, but, Matlab? Seriously? I don't know the internals, but I'm fairly sure that there's going to be some switching between the CPU and GPU for Matlab to be able to run code there. If you want something to run fast on the GPU, you code it in a language that can run entirely on the GPU. Not Matlab.
- andrewcooke 15y ago
  the code i ported was originally matlab. if you can write the operations without explicit loops (using matlab's builtins) then it should be able to translate things fairly well to multiple processors. i don't know much about matlab's gpu support (when i looked, you needed third party libs - looks like that has changed?) but again, if it's standard transforms and matrix operations they should be able to make it fairly efficient. in our case the matlab code had explicit loops (in a sense i was lucky - the inner kernel operations were messy enough to need explicit coding, but regular enough to be handled efficiently on a gpu) and that's what really kills you (because you're bouncing out of optimised c code into their interpreter). just moving from matlab to c (on the cpu) also gave us a significant speedup. what i'm saying, in a slow rambling way, is that calculating a bunch of sines should be as fast in matlab as in hand-coded c or opencl (on gpu and cpu respectively). because that can be expressed as a vector operation and matlab will have invested a lot of time and effort into making that code fast.
  - StavrosK 15y ago
    Interesting. I checked the manual after my post above and they do execute sines entirely on the GPU, however you do still need to jump in and out for most coding. It does invalidate my point for his example, though (but it doesn't invalidate any of the other points).
- melonakos 15y ago
  If in MATLAB, use Jacket. It's better than what MathWorks offers, http://accelereyes.com/compare http://accelereyes.com/compare If in C, C++, Fortran, Python, use LibJacket. Learn more, http://accelereyes.com/libjacket_tour http://accelereyes.com/libjacket_tour Disclaimer: yes, i work on both of those :)
  - andrewcooke 15y ago
    so the matlab solution ships data back and forth for each line of the code? ouch. i looked at your package back then and got the impression you mapped variables to the GPU main memory, which i agree makes more sense.
malbs 15y ago
A consumer level GTX 590 is almost twice as powerful as a Tesla card (It's 2x gtx 580s in on-card sli), and the retail price? For a 590 you're looking at about $800 aud, a tesla is upwards of $4000.

StavrosK 15y ago

Not only that, but, Matlab? Seriously? I don't know the internals, but I'm fairly sure that there's going to be some switching between the CPU and GPU for Matlab to be able to run code there. If you want something to run fast on the GPU, you code it in a language that can run entirely on the GPU. Not Matlab.

andrewcooke 15y ago
the code i ported was originally matlab. if you can write the operations without explicit loops (using matlab's builtins) then it should be able to translate things fairly well to multiple processors. i don't know much about matlab's gpu support (when i looked, you needed third party libs - looks like that has changed?) but again, if it's standard transforms and matrix operations they should be able to make it fairly efficient. in our case the matlab code had explicit loops (in a sense i was lucky - the inner kernel operations were messy enough to need explicit coding, but regular enough to be handled efficiently on a gpu) and that's what really kills you (because you're bouncing out of optimised c code into their interpreter). just moving from matlab to c (on the cpu) also gave us a significant speedup. what i'm saying, in a slow rambling way, is that calculating a bunch of sines should be as fast in matlab as in hand-coded c or opencl (on gpu and cpu respectively). because that can be expressed as a vector operation and matlab will have invested a lot of time and effort into making that code fast.
- StavrosK 15y ago
  Interesting. I checked the manual after my post above and they do execute sines entirely on the GPU, however you do still need to jump in and out for most coding. It does invalidate my point for his example, though (but it doesn't invalidate any of the other points).
melonakos 15y ago
If in MATLAB, use Jacket. It's better than what MathWorks offers, http://accelereyes.com/compare http://accelereyes.com/compare If in C, C++, Fortran, Python, use LibJacket. Learn more, http://accelereyes.com/libjacket_tour http://accelereyes.com/libjacket_tour Disclaimer: yes, i work on both of those :)
- andrewcooke 15y ago
  so the matlab solution ships data back and forth for each line of the code? ouch. i looked at your package back then and got the impression you mapped variables to the GPU main memory, which i agree makes more sense.

malbs 15y ago

A consumer level GTX 590 is almost twice as powerful as a Tesla card (It's 2x gtx 580s in on-card sli), and the retail price? For a 590 you're looking at about $800 aud, a tesla is upwards of $4000.