3 ms·
I'm actually curious, given the massive speed difference between CPU and GPU: can anyone here think of any situation where a fallback to CPU speed would still b
by crazygringo 6y ago
I'm actually curious, given the massive speed difference between CPU and GPU: can anyone here think of any situation where a fallback to CPU speed would still be acceptable for a real-life application?
The only thing I can think of is simple image operations like a 5x speedup for a Gaussian blur filter... but then I also wonder if you'd even bother writing code to do it on the GPU in the first place.
- soylentgraham 6y agoWith OpenCL, one of the big (massive) benefits was being able to choose to run a kernel on CPU or GPU. The main case where CPU ran a lot faster was random memory access.
- rewq4321 6y ago> a 5x speedup [...] I also wonder if you'd even bother writing code to do it on the GPU in the first place A 5x speed-up is very significant in many applications! I've spent a bunch of energy on several projects just aiming for a 2x speed-up on GPUs - battling against SIMD and other optimisations on the CPU side. As for real-world examples: The tensforflow.js CPU back-end runs something like 10x slower than the WebGL back-end, and I've got some great mileage out of the CPU back-end in simple node.js apps that don't have a GPU attached to their server.
- dahart 6y ago> can anyone here think of any situation where a fallback to CPU speed would still be acceptable for a real-life application? Sure, there are valid scenarios where the slower speed fallback is acceptable. Note that I agree with the GP comment; fallback should be explicit not automatic. - Developer adoption. Not everyone has a GPU, but with a fallback path everyone can try it out and write code that will work on the fast path. - Application development. Along the same lines as developer adoption, not every customer has a GPU either, so if I write an app I may want to let the user try a new feature (with a warning that it’s slow) rather than tell them they can’t use the feature at all until they upgrade their hardware. - Heterogeneous compute farms. I’m thinking in particular of render farms for VFX and CG movies, as an example, how a render farm is large and may not be full of GPU nodes due to cost. A fallback path allows time/perf to be balanced against whatever the farm’s budget for GPU nodes is. - Test machines. Running automated end-to-end integration tests to check for correctness doesn’t depend on speed, and if you’re renting your test machines, it might be cheaper to fallback to CPU than to rent GPU nodes. Think Selenium tests running on a bunch of AWS nodes, for example.
- Jasper_ 6y agoThe GPU isn't a "make code run fast" button, there are all sorts of algorithms that run poorly on a GPU because of its heavy SIMD structure and its programming model requires a programmer to cross their ts and dot their is, and the overhead of uploading/downloading data is not free, and scheduling tasks is not free. Look at how ISPC had to change the C language to make it parallelizable for actual problems. I've attempted to write compute frustum culling in GPU.js and it came out slower than my CPU approach. Hand-written shader performed much better than either.
- ShinTakuya 6y agoI mean, a GPU is only faster if it's doing many operations in parallel. So you can definitely have code that is faster on a CPU than on a GPU. And there'd be borderline cases where it's faster on a GPU but only by a bit.
- jlokier 6y agoThe CPU isn't always too slow. For example rendering a spreadsheet, text editor, web browser or CAD screen might prefer GPU to make it visibly smoother and more satisfying (especially scrolling, compositing and high density text updates), but will still be useful on CPU. And current CPUs are faster than old GPUs for some things, especially if vector instructions are used. And GPUs are sometimes slow due to technical issues such as bus traffic, even if the GPU itself is fast. So it's not as simple as "never use the CPU".
- dwheeler 6y agoYes! In fact, in practically ALL applications it's better to work than not work at all. The only exception I can think of is directly-interactive software like video games and VR imaging, where minimum fps really does matter. Many situations for various customers aren't massive datasets but modest sets, so you might be on the left of the J-curve. Requiring 2 seconds, instead of 0.2 seconds, for a result is simply not important in most cases. I think it's fair to warn the user that "GPU acceleration not working" if the performance is expected to be much better with a GPU.
- wtracy 6y ago> I also wonder if you'd even bother writing code to do it on the GPU in the first place. Some of your customers have large datasets that are only manageable with hardware acceleration, and others don't? Maybe you don't want to maintain two separate codebases?