5 ms·

I'm really sorry I will try to dig up the source myself, but I've read basically the opposite argument in a few technical papers -- that GPU is NOT as fast as c

by fat0wl 12y ago

I'm really sorry I will try to dig up the source myself, but I've read basically the opposite argument in a few technical papers -- that GPU is NOT as fast as claimed for many classic test algorithms (actual speed-up is more like factor of 2 than 10) and that the performance gap between traditional CPUs and GPUs is actually lessening. I'm going to read this article anyway to hear their take & for the learning experience, but does anyone remember any of the counter-arg articles?

fat0wl 12y ago
hah i dont understand why this was (even temporarily) downvoted. You do realize that the CPU & GPU industries are at war with each other, right? I read the article now, cool technical overview -- but basically all of these processor arch articles have a slant in all the paragraphs where they wax poetic (abstract, analysis/conclusion). I think it would be helpful for people to be aware of this... AFAIK Nvidia (their company name is at the top of this paper btw in case u weren't paying attention) are trying to generalize their chips to the point where they can enter the CPU market, and Intel chips can render 3D graphics well enough to handle most games that are ~5 years old (since Haswell or maybe one or 2 gens before). So this isn't a particularly slanted article but there is a fair amount of propaganda / contrived performance studies in this market... NVIDIA & Intel are vying for each other's core customer bases. Anyone interested in the field should dig up the articles that try to debunk perfomance myths as well as studying architecture overview. (Some of the sentences in the last few paragraphs, for example, made me sorta queasy & would get shredded on Wikipedia.)
tmurray 12y ago
disclaimer: I work in this space and have done so for a while, including previously on CUDA and on Titan. GPUs for general purpose computation were never 100x faster than CPUs like people claimed in 2008 or so. They're just not. That was basically NV marketing mixed with a lot of people publishing some pretty bad early work on GPUs. Lots of early papers that fanned GPU hype followed the same basic form: "We have this standard algorithm, we tested it on a single CPU core with minimal optimizations and no SIMD (or maybe some terrible MATLAB code with zero optimization), we tested a heavily optimized GPU version, and look the GPU version is faster! By the way, we didn't port any of those optimizations back to the CPU version or measure PCIe transfer time to/from the GPU." It was utterly trivial to get any paper into a conference by porting anything to the GPU and reporting a speedup. Most of the GPU related papers from this time were awful. I remember one in particular that claimed a 1000x speedup by timing just the amount of time it took for the kernel launch to the GPU instead of the actual kernel runtime, and somehow nobody (either the authors or the reviewers) realized that this was utterly impossible. GPUs have more FLOPs and more memory bandwidth in exchange for requiring PCIe and lots of parallel work. if your algorithm needs those more than anything else (like cache), can minimize PCIe transfer time, and handles the whole massive parallelism thing well, then GPUs are a pretty good bet. If you can't, then they're not going to work particularly well. (now, if you need to do 2D interpolation and can use the texture fetch hardware on the GPU to do it instead of a bunch of arbitrary math... yeah, that's a _huge_ performance increase because you get that interpolation for free from special-purpose hardware. but that's incredibly rare in practice)
- fat0wl 12y ago
  ah, yes. :) very nice detailed summary of some of the issues in this sect of "academia" (I put that in quotes only because all the research seems to be co-written by corps). I am into audio DSP & am planning to port a couple of audio algorithms (lots of FFT & linear algebra) to run on GPU but haven't even gotten to it because I considered it a pre-mature optimization to this point. I'm sure it would improve performance, but nowhere near what GPU advocates would claim. My biggest reason? "PCIe transfer time to/from GPU", plus it would be unoptimized GPU code. Once you read a few of these papers it becomes painfully obvious that a lot of tuning goes into the GPU algorithms that offer anything more than a low single-digit factor of speedup. It's still very significant (cutting a 3 hour algorithm down to 1 would be huge) but if you're in an early stage of research it may be a toss-up over whether its better to just tune the algorithm itself / run computations overnight rather than going through the trouble of writing a GPU-based POC. Maybe if you have 1 or 2 under your belt its not such a big deal but for most of the researchers I know GPU algorithm rewrites would not be trivial. (I've been doing enterprise Java coding for about 2 years now so the idea isn't so intimidating now, but in a past life of mucking around with Matlab scripts I'm sure it would have been daunting).
_ihaque 12y ago
tmurray is basically right: most of the really big reported gains are artifacts of unoptimized CPU code. Except for hardware special functions on GPUs, you shouldn't be able to exceed the theoretical perf ratios between GPU and CPU, which are roughly ~30x in FLOPs and ~10x in bandwidth. Depending on whether arithmetic or memory is the algorithmic bottleneck, you'll hit one of those limits. I wrote a paper [1] on this in one particular domain (computational chemistry) more or less as a rebuttal to a paper that claimed enormous GPU speedups; it was a consequence of slow CPU code, not especially fast GPU code. [1] http://cs.stanford.edu/people/ihaque/papers/2dtanimoto.pdf http://cs.stanford.edu/people/ihaque/papers/2dtanimoto.pdf
- sklogic 12y ago
  One should not forget another important thing: 1GFLOP in CPU is more expensive in terms of power than 1GFLOP in a GPU. So it's not only about chasing GFLOPs, in the mobile and embedded world it's also all about power.
  - brigade 12y ago
    But normalizing for power reduces the GPU advantage even more! Haswell for instance achieves about 5.3-5.8 GFLOP/W, compared to 24-28 GFLOP/W of Maxwell. That's less than a 5x theoretical computational gain.
    - sklogic 12y ago
      Take a look at, say, Raspberry Pi: 24GFLOP for 1/2W. You won't get this for any number of mobile CPU cores.
      - tmurray 12y ago
        but that's GLES 2.0, which is significantly less flexible than the kinds of GPUs we're discussing here and is not even in the same ballpark as a CPU (and almost certainly significantly less strict in terms of floating point precision than a GLES 3 device).

by fat0wl 12y ago

fat0wl 12y ago

hah i dont understand why this was (even temporarily) downvoted. You do realize that the CPU & GPU industries are at war with each other, right? I read the article now, cool technical overview -- but basically all of these processor arch articles have a slant in all the paragraphs where they wax poetic (abstract, analysis/conclusion). I think it would be helpful for people to be aware of this... AFAIK Nvidia (their company name is at the top of this paper btw in case u weren't paying attention) are trying to generalize their chips to the point where they can enter the CPU market, and Intel chips can render 3D graphics well enough to handle most games that are ~5 years old (since Haswell or maybe one or 2 gens before). So this isn't a particularly slanted article but there is a fair amount of propaganda / contrived performance studies in this market... NVIDIA & Intel are vying for each other's core customer bases. Anyone interested in the field should dig up the articles that try to debunk perfomance myths as well as studying architecture overview. (Some of the sentences in the last few paragraphs, for example, made me sorta queasy & would get shredded on Wikipedia.)

tmurray 12y ago

disclaimer: I work in this space and have done so for a while, including previously on CUDA and on Titan. GPUs for general purpose computation were never 100x faster than CPUs like people claimed in 2008 or so. They're just not. That was basically NV marketing mixed with a lot of people publishing some pretty bad early work on GPUs. Lots of early papers that fanned GPU hype followed the same basic form: "We have this standard algorithm, we tested it on a single CPU core with minimal optimizations and no SIMD (or maybe some terrible MATLAB code with zero optimization), we tested a heavily optimized GPU version, and look the GPU version is faster! By the way, we didn't port any of those optimizations back to the CPU version or measure PCIe transfer time to/from the GPU." It was utterly trivial to get any paper into a conference by porting anything to the GPU and reporting a speedup. Most of the GPU related papers from this time were awful. I remember one in particular that claimed a 1000x speedup by timing just the amount of time it took for the kernel launch to the GPU instead of the actual kernel runtime, and somehow nobody (either the authors or the reviewers) realized that this was utterly impossible. GPUs have more FLOPs and more memory bandwidth in exchange for requiring PCIe and lots of parallel work. if your algorithm needs those more than anything else (like cache), can minimize PCIe transfer time, and handles the whole massive parallelism thing well, then GPUs are a pretty good bet. If you can't, then they're not going to work particularly well. (now, if you need to do 2D interpolation and can use the texture fetch hardware on the GPU to do it instead of a bunch of arbitrary math... yeah, that's a _huge_ performance increase because you get that interpolation for free from special-purpose hardware. but that's incredibly rare in practice)

fat0wl 12y ago
ah, yes. :) very nice detailed summary of some of the issues in this sect of "academia" (I put that in quotes only because all the research seems to be co-written by corps). I am into audio DSP & am planning to port a couple of audio algorithms (lots of FFT & linear algebra) to run on GPU but haven't even gotten to it because I considered it a pre-mature optimization to this point. I'm sure it would improve performance, but nowhere near what GPU advocates would claim. My biggest reason? "PCIe transfer time to/from GPU", plus it would be unoptimized GPU code. Once you read a few of these papers it becomes painfully obvious that a lot of tuning goes into the GPU algorithms that offer anything more than a low single-digit factor of speedup. It's still very significant (cutting a 3 hour algorithm down to 1 would be huge) but if you're in an early stage of research it may be a toss-up over whether its better to just tune the algorithm itself / run computations overnight rather than going through the trouble of writing a GPU-based POC. Maybe if you have 1 or 2 under your belt its not such a big deal but for most of the researchers I know GPU algorithm rewrites would not be trivial. (I've been doing enterprise Java coding for about 2 years now so the idea isn't so intimidating now, but in a past life of mucking around with Matlab scripts I'm sure it would have been daunting).

_ihaque 12y ago

tmurray is basically right: most of the really big reported gains are artifacts of unoptimized CPU code. Except for hardware special functions on GPUs, you shouldn't be able to exceed the theoretical perf ratios between GPU and CPU, which are roughly ~30x in FLOPs and ~10x in bandwidth. Depending on whether arithmetic or memory is the algorithmic bottleneck, you'll hit one of those limits. I wrote a paper [1] on this in one particular domain (computational chemistry) more or less as a rebuttal to a paper that claimed enormous GPU speedups; it was a consequence of slow CPU code, not especially fast GPU code. [1] http://cs.stanford.edu/people/ihaque/papers/2dtanimoto.pdf http://cs.stanford.edu/people/ihaque/papers/2dtanimoto.pdf

sklogic 12y ago
One should not forget another important thing: 1GFLOP in CPU is more expensive in terms of power than 1GFLOP in a GPU. So it's not only about chasing GFLOPs, in the mobile and embedded world it's also all about power.
- brigade 12y ago
  But normalizing for power reduces the GPU advantage even more! Haswell for instance achieves about 5.3-5.8 GFLOP/W, compared to 24-28 GFLOP/W of Maxwell. That's less than a 5x theoretical computational gain.
  - sklogic 12y ago
    Take a look at, say, Raspberry Pi: 24GFLOP for 1/2W. You won't get this for any number of mobile CPU cores.
    - tmurray 12y ago
      but that's GLES 2.0, which is significantly less flexible than the kinds of GPUs we're discussing here and is not even in the same ballpark as a CPU (and almost certainly significantly less strict in terms of floating point precision than a GLES 3 device).