4 ms·

“CPUs excel at performing multiple tasks, including control, logic, and device-management operations, but their ability to perform fast arithmetic calculations

“CPUs excel at performing multiple tasks, including control, logic, and device-management operations, but their ability to perform fast arithmetic calculations is limited. GPUs are the opposite. Originally designed to render 3D video games, GPUs are capable of fewer tasks than CPUs, but they are specially designed to perform mathematical calculations very rapidly.” I thought the advantage of the GPU is not speed but parallelism or are modern power saving processors also slow compared to GPUs on non-parallel tasks? Also, is it common to use Registered Trademarks (FastEddy) in a paper these days. I know a lot of universities try to commercialize research, is this the reason for the trademark?

dahart 5y ago
> I thought the advantage of the GPU is not speed but parallelism or are modern power saving processors also slow compared to GPUs on non-parallel tasks? Fast and parallel are two ways to say the same thing, or in other words, the wide parallelism of the GPU's many math cores is what makes it faster than CPU's relatively small capability to do math in parallel. (The x-factor is so large, the power saving features don't really make a meaningful difference.) The tradeoff is you need a highly parallel workload to be efficient and that much faster on the GPU.
- foota 5y ago
  I think they're actually asking, if you have a single "thread" of a GPU, is that one thread quicker at math than a single thread of a CPU? The most fair comparison imo might be a single "warp" in nvidia terms against a single CPU thread, which is roughly similar to what a CPU might be doing for AVX.
  - stephen999 5y ago
    It’s much slower. GPU single thread performance is very slow. Unless you are doing some specialized operation that has a special instruction on GPU, it will be slower. GPU clock speeds are lower in general and the CPU has a much wider super scalar pipeline. Can be 4-8 wide vs single or double wide for most GPUs. It also has many latency-hiding techniques for a single thread. The GPU has none, it uses multiple threads to hide latency.
wongarsu 5y ago
It's useful to think of GPUs as CPUs with low core frequency but incredibly wide SIMD instructions. They are very good at doing the exact same instructions in parallel (apply this formula to 4 million pixels), they are fairly bad at anything that includes branches or loops (doesn't fit the SIMD model), or random memory access (only so much memory bandwidth to go around between all the cores). That makes GPUs generally good at arithmetic tasks, but horrible at control or logic tasks since those usually involve lots of branching.
- sampo 5y ago
  > They are very good at doing the exact same instructions in parallel (apply this formula to 4 million pixels) In numerical fluid mechanics, you rather want to apply the same formula to the nearest 27 (3x3x3) or 125 (5x5x5) pixels in the 3-dimensional array. And then store the result to another 3-dimensional array. Or maybe for calculating a value in a 3d array A, you need to apply a formula that looks at the nearest 27 values in both array A itself and also in another 3d array B. Maybe also C.
  - dahart 5y ago
    This is true for pixels too; think about a median or blur operation, or a 2d fluid sim, you might gather and combine the nearest NxN pixels. 2d and 3d are nearly identical, the main difference is just indexing. Parent comment was talking about the single formula that produces the output value written to the cell, which as you say might well involve reading inputs from many nearby cells. Fluid mechanics can also be thought of as a single formula per output cell. (And normally it’s most natural to organize GPU threads so that they are 1:1 with output cells.)
seoaeu 5y ago
The architecture of GPUs is less exotic than people would have you believe. A given graphics card will have a couple dozen relatively slow cores (arranged in a NUMA hierarchy) each with 10 logical threads and 32 or 64-wide SIMD. The 10 logical threads/core enables many concurrent memory operations to be in flight at the same time, while the wide SIMD enables massive parallelism. There's some cleverness in the programming model however: the code the programmer writes is executed on a single SIMD lane so 32 or 64 copies of it can be run in lockstep. In total, to keep every lane of every logical thread of every core busy requires thousands of concurrent threads. (There is also some special purpose hardware for graphics related tasks, but that is less relevant to GPGPU workloads)

dahart 5y ago

> I thought the advantage of the GPU is not speed but parallelism or are modern power saving processors also slow compared to GPUs on non-parallel tasks? Fast and parallel are two ways to say the same thing, or in other words, the wide parallelism of the GPU's many math cores is what makes it faster than CPU's relatively small capability to do math in parallel. (The x-factor is so large, the power saving features don't really make a meaningful difference.) The tradeoff is you need a highly parallel workload to be efficient and that much faster on the GPU.

foota 5y ago
I think they're actually asking, if you have a single "thread" of a GPU, is that one thread quicker at math than a single thread of a CPU? The most fair comparison imo might be a single "warp" in nvidia terms against a single CPU thread, which is roughly similar to what a CPU might be doing for AVX.
- stephen999 5y ago
  It’s much slower. GPU single thread performance is very slow. Unless you are doing some specialized operation that has a special instruction on GPU, it will be slower. GPU clock speeds are lower in general and the CPU has a much wider super scalar pipeline. Can be 4-8 wide vs single or double wide for most GPUs. It also has many latency-hiding techniques for a single thread. The GPU has none, it uses multiple threads to hide latency.

wongarsu 5y ago

It's useful to think of GPUs as CPUs with low core frequency but incredibly wide SIMD instructions. They are very good at doing the exact same instructions in parallel (apply this formula to 4 million pixels), they are fairly bad at anything that includes branches or loops (doesn't fit the SIMD model), or random memory access (only so much memory bandwidth to go around between all the cores). That makes GPUs generally good at arithmetic tasks, but horrible at control or logic tasks since those usually involve lots of branching.

sampo 5y ago
> They are very good at doing the exact same instructions in parallel (apply this formula to 4 million pixels) In numerical fluid mechanics, you rather want to apply the same formula to the nearest 27 (3x3x3) or 125 (5x5x5) pixels in the 3-dimensional array. And then store the result to another 3-dimensional array. Or maybe for calculating a value in a 3d array A, you need to apply a formula that looks at the nearest 27 values in both array A itself and also in another 3d array B. Maybe also C.
- dahart 5y ago
  This is true for pixels too; think about a median or blur operation, or a 2d fluid sim, you might gather and combine the nearest NxN pixels. 2d and 3d are nearly identical, the main difference is just indexing. Parent comment was talking about the single formula that produces the output value written to the cell, which as you say might well involve reading inputs from many nearby cells. Fluid mechanics can also be thought of as a single formula per output cell. (And normally it’s most natural to organize GPU threads so that they are 1:1 with output cells.)

seoaeu 5y ago

The architecture of GPUs is less exotic than people would have you believe. A given graphics card will have a couple dozen relatively slow cores (arranged in a NUMA hierarchy) each with 10 logical threads and 32 or 64-wide SIMD. The 10 logical threads/core enables many concurrent memory operations to be in flight at the same time, while the wide SIMD enables massive parallelism. There's some cleverness in the programming model however: the code the programmer writes is executed on a single SIMD lane so 32 or 64 copies of it can be run in lockstep. In total, to keep every lane of every logical thread of every core busy requires thousands of concurrent threads. (There is also some special purpose hardware for graphics related tasks, but that is less relevant to GPGPU workloads)