5 ms·

A curious result: Using four-wide SIMD instructions gives a mere 40% speed-up compared to the scaler case. I wonder if this is actually a net loss. - CPU's ar

A curious result: Using four-wide SIMD instructions gives a mere 40% speed-up compared to the scaler case. I wonder if this is actually a net loss. - CPU's are power limited today (high-confidence) - Using the SIMD units vice scalar units increases power consumption (medium confidence - certainly some arithmetic-limited algorithms consume much more power with SSE) Conclusion - the incremental impact on total system throughput is negative for this speed boost? I wonder how much of a raw throughput improvement is needed to justfiy using the SSE units' increased power consuption.

pcwalton 11y ago
You have to also take into account that getting your work done 40% faster lets the system go to sleep 40% sooner. Getting the CPU into an idle state tends to dominate everything else. In my experience, it's very rarely worth choosing a slower algorithm out of power consumption concerns. Use all the silicon available to you to get the work done and go to sleep.
- brandmeyer 11y ago
  The reference provided by nkurz measured clearly that it is a net win in most cases. However, that fact isn't inherantly obvious. CPU's aggressively use clock gating to shut down idle parts of the processor at a fine-grained level. So avoiding the use of several entire compute units does have some savings as well. The difference is that I was assuming something more like 2x power usage, where the actual difference is in the small single digits.
  - Sanddancer 11y ago
    Also, keep in mind how little of the modern CPU's die space is actually dedicated to the traditional cores. A quad-core skylake processor dedicates over half its space to things other than CPU cores [1], and that each core itself has a lot of space needed for paths that are always going to be used, like instruction decoding and storing the registers. So gating in a situation like this is only ever going to turn off a very small bit of the processor. [1] http://www.techpowerup.com/img/15-08-18/77a.jpg http://www.techpowerup.com/img/15-08-18/77a.jpg
nkurz 11y ago
Using the SIMD units vs scalar units increases power consumption True, but only slightly. If SIMD works for the problem, the overall efficiency can be much greater. This paper has some good numbers: http://kentcz.com/downloads/P149-ISCA14-Preprint.pdf http://kentcz.com/downloads/P149-ISCA14-Preprint.pdf I wonder if this is actually a net loss. No, on modern processors SIMD is usually a big win. The basic summary is that "fixed costs dominate variable costs". Increasing vector width (if the problem allows it) translates almost directly to increased efficiency: "Doubling the SIMD width with AVX instructions doubles performance and only increases total power by 4.3%, which results in a 1.9x improvement in energy efficiency." (quotes from the paper) More generally, higher Instructions Per Cycle (IPC) means higher efficiency. The overhead of the processor is a big factor. For computationally intense problems, working faster and finishing in fewer cycles makes for higher energy efficiency.
- brandmeyer 11y ago
  Good reference. This is a sign that the power consumed by instruction decoding and OoO processing is much greater than the arithmetic done by each instruction in general-purpose processors.
  - wtallis 11y ago
    This is something that the Mill processor folks refer to frequently, by comparing DSP power usage against general-purpose CPU power usage. The power consumption of your ALUs is nothing compared to the bag of tricks CPUs use to keep them busy: register renaming, instruction reordering, multiple kinds of speculation, etc.

pcwalton 11y ago

You have to also take into account that getting your work done 40% faster lets the system go to sleep 40% sooner. Getting the CPU into an idle state tends to dominate everything else. In my experience, it's very rarely worth choosing a slower algorithm out of power consumption concerns. Use all the silicon available to you to get the work done and go to sleep.

brandmeyer 11y ago
The reference provided by nkurz measured clearly that it is a net win in most cases. However, that fact isn't inherantly obvious. CPU's aggressively use clock gating to shut down idle parts of the processor at a fine-grained level. So avoiding the use of several entire compute units does have some savings as well. The difference is that I was assuming something more like 2x power usage, where the actual difference is in the small single digits.
- Sanddancer 11y ago
  Also, keep in mind how little of the modern CPU's die space is actually dedicated to the traditional cores. A quad-core skylake processor dedicates over half its space to things other than CPU cores [1], and that each core itself has a lot of space needed for paths that are always going to be used, like instruction decoding and storing the registers. So gating in a situation like this is only ever going to turn off a very small bit of the processor. [1] http://www.techpowerup.com/img/15-08-18/77a.jpg http://www.techpowerup.com/img/15-08-18/77a.jpg

nkurz 11y ago

Using the SIMD units vs scalar units increases power consumption True, but only slightly. If SIMD works for the problem, the overall efficiency can be much greater. This paper has some good numbers: http://kentcz.com/downloads/P149-ISCA14-Preprint.pdf http://kentcz.com/downloads/P149-ISCA14-Preprint.pdf I wonder if this is actually a net loss. No, on modern processors SIMD is usually a big win. The basic summary is that "fixed costs dominate variable costs". Increasing vector width (if the problem allows it) translates almost directly to increased efficiency: "Doubling the SIMD width with AVX instructions doubles performance and only increases total power by 4.3%, which results in a 1.9x improvement in energy efficiency." (quotes from the paper) More generally, higher Instructions Per Cycle (IPC) means higher efficiency. The overhead of the processor is a big factor. For computationally intense problems, working faster and finishing in fewer cycles makes for higher energy efficiency.

brandmeyer 11y ago
Good reference. This is a sign that the power consumed by instruction decoding and OoO processing is much greater than the arithmetic done by each instruction in general-purpose processors.
- wtallis 11y ago
  This is something that the Mill processor folks refer to frequently, by comparing DSP power usage against general-purpose CPU power usage. The power consumption of your ALUs is nothing compared to the bag of tricks CPUs use to keep them busy: register renaming, instruction reordering, multiple kinds of speculation, etc.