8 ms·

As someone excited by the improvements that are possible moving from AVX to AVX2, I was recently surprised to learn that Intel has chips released this year that

by nkurz 12y ago

As someone excited by the improvements that are possible moving from AVX to AVX2, I was recently surprised to learn that Intel has chips released this year that do not support AVX. AVX-512 is still supposedly scheduled for release next year with Skylake. How long do you suppose until it's reasonable to presume that most processors will support it? SSE 4.2 came out in 2008 and appears to be the baseline now, so 5 years seems like a reasonable guess. It's hard to believe this degree of ISA segmentation is going to benefit Intel in the end.

wtallis 12y ago
I think that the SIMD instructions are well past the point of diminishing returns, both for the kinds of instructions provided and the width of the vectors. I'm more interested in how VT-d and TSX allow different software architectures, if and only if you can design your OS around the assumption that they're available. By only including them on select models, Intel's making them much less useful even for the customers who want to pay extra to have those features.
- thechao 12y ago
  I used to write software rasterizers in a past life. AVX-512 is straight-up an observed 8x performance gain over SSE: a well-written software rasterizer is a very clever thread-scheduling algorithm wrapped around long lists of FMA-load/store sequences. I know that ray-tracers, databases, and other 'big data' all gain equally-well from AVX-512. The IO-request depth on these new parts is such that, for ALU-heavy work, with good IO-spread, memory latency is completely hidden---fetch time is still ~250-300c, but you just no longer see it. The only thing I miss is 'addsets', which was dropped from LRBni, as this instruction was 'the rasterizer function': it now requires a fairly involved sequence to replicate in AVX-512. Pining for other lost things: if we had the LRBni up- and down- convert instructions, a software texture sampler would be a lot more feasible.
  - coherentpony 12y ago
    Does gcc support these new wider registers yet?
    - valarauca1 12y ago
      Yes and no. If you write the ASM you can use them, but it's code generation likely won't (but then I haven't checked for 2 months). Most things that can be vectorizes will be placed in SSE rather then AVX. Also the GCC generally sucks at optimizing for SSE, or determining when code should use SSE as opposed to standard registers. Generally speaking the LLVM backend does better SSE and vectorization code generation. But some think it used SSE to much/incorrectly. So TLDR no New wide registers are VERY new. The are barely supported, my processor has AVX2.0 and I have (still have to) one of these days set up perf properly because all of its fault codes aren't properly baked into the kernel yet (as of 3.17). (Sorry for the lack of references) Also the biggest draw back of AVX is they don't hold their state between context changes :/
  - wtallis 12y ago
    Wider SIMD units may provide a linear performance increase for many workloads, but they have a super-linear impact on the up front cost of the chip, especially when you take into account the opportunity cost of those transistors: they could have been dedicated to something that might have also helped non-numerical workloads. Wide SIMD is great to have, but it doesn't come free, or else we wouldn't have GPUs.
    - deleted 12y ago
      [deleted]
  - sounds 12y ago
    It may just be that Intel has abandoned this use case to the GPU. Which obviously is bad news for those of us who still rely on CPU SIMD. But wtallis' argument to the contrary is quite compelling: https://news.ycombinator.com/item?id=8801808 https://news.ycombinator.com/item?id=8801808
  - berkut 12y ago
    You only get those performance gains if you've got the cache and memory bandwidth - which is often very lacking. E.g. AVX (8-wide) was added for SandyBridge, but it wasn't always that usable until Intel doubled the cache bandwidth with Ivy Bridge. With ray tracing the increase in performance going from SSE (4-wide) to AVX (8-wide) for BVH intersection was only ~25-30% - instead of the theoretical 100% increase). You're generally limited by memory bandwidth.
- vardump 12y ago
  Image processing, etc. stuff I sometimes write seems to benefit pretty linearly from wider SIMD registers, like in AVX2. It'd be useful to get up to a cache line wide SIMD registers. Cache line is the granularity memory subsystem works at anyways. I wonder when cache line size will be increased from 64 to 128 bytes. Hopefully that wouldn't affect total number of lines, though. Mere 512 entries in L1D cache is already such a pain.
- TheLoneWolfling 12y ago
  CISC is rapidly becoming a requirement, however. Power requirements scale as the square of the frequency, and also, perhaps even more importantly, leakage power, and hence, heat generated per unit area, increases as the process size decreases. As such, as process sizes shrink, less and less actual area of the chip can be active at any one time. ("Dark silicon"). In other words, you might as well have areas of the chip that aren't used most of the time, because most of the chip can't be active at once anyways.
- varelse 12y ago
  I think that the multiple 32-way SIMD units on GPUs demonstrates there's plenty of room for widening on CPUs before we hit diminishing returns. The question in my mind is how much more like a GPU should CPUs become and Xeon Phi demonstrated how easy it is to get the mix totally wrong.
  - wtallis 12y ago
    There are clearly plenty of common workloads that can make use of arbitrarily wide SIMD units. But nothing about the GPU market provides a clear indication that those SIMD units are better grafted onto the CPU than delegated to a coprocessor. The programming model is certainly simpler if they're part of the CPU's normal instruction pipeline, but on the other hand most CPUs don't have the memory bandwidth to feed more than a handful of GPU-style SIMD units and certainly don't have the power delivery or dissipation capacity to handle the compute resources of a mid-range GPU in addition to the normal CPU. Having the big SIMD units on a coprocessor card seems to be working just fine for the kinds of workloads that really need a lot of vector processing power. Those workloads also seem to not benefit much from things like speculative out of order execution, which frees up a lot of die space and power draw budget for more SIMD units. To some extent it's inevitable that SIMD code can't be making too much use of CPU design features that benefit branchy code, or else the code couldn't be vectorized in the first place. The workloads that are making heavy use of SIMD on the CPU are mainly using it for bursts of number crunching where the problem is too small to be worth the latency penalty of moving it to the GPU, but that penalty is getting smaller with each generation. The small integrated graphics processors common on non-server CPUs can all but eliminate that penalty, further reducing the demand for extra-wide SIMD in the CPU cores.
vardump 12y ago
That's why I feel dynamic code generation is the future. JIT. It's madness to write high performance code to take CPU [1] and DRAM [2] specifics into account. It's easy to write something that performs well on one system. [1]: 32-bit or 64-bit -> how many registers are available. SSE2, SSE 4.1, AVX2, etc. Different prefetchers, cache configurations. Different instruction level limitations and performance. Differing amounts of cache with rather variable latencies. [2]: Different number of memory channels, different distance between DRAM page changes.
- iopq 12y ago
  Compilers generate static code that will branch to the correct code path based on your architecture. You don't need to JIT until there are so many paths your executable becomes too big.
  - vardump 12y ago
    Any references about this compiler?
    - nitrogen 12y ago
      icc does this, as an example. It got Intel into trouble because they changed the compiler at one point to use the slowest path on AMD processors.
    - tmurray 12y ago
      https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gcc/Function-Attributes.html https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gcc/Function-Attrib... and http://pasky.or.cz/dev/glibc/ifunc.c http://pasky.or.cz/dev/glibc/ifunc.c look up the target attribute and the ifunc attribute--it's basically a way to compile multiple versions of a function for different targets in a single source file and then use the dynamic linker to determine which one to resolve at runtime. obvious use is for things like optimized memcpy implementations.
      - vardump 12y ago

by nkurz 12y ago

wtallis 12y ago

I think that the SIMD instructions are well past the point of diminishing returns, both for the kinds of instructions provided and the width of the vectors. I'm more interested in how VT-d and TSX allow different software architectures, if and only if you can design your OS around the assumption that they're available. By only including them on select models, Intel's making them much less useful even for the customers who want to pay extra to have those features.

thechao 12y ago
I used to write software rasterizers in a past life. AVX-512 is straight-up an observed 8x performance gain over SSE: a well-written software rasterizer is a very clever thread-scheduling algorithm wrapped around long lists of FMA-load/store sequences. I know that ray-tracers, databases, and other 'big data' all gain equally-well from AVX-512. The IO-request depth on these new parts is such that, for ALU-heavy work, with good IO-spread, memory latency is completely hidden---fetch time is still ~250-300c, but you just no longer see it. The only thing I miss is 'addsets', which was dropped from LRBni, as this instruction was 'the rasterizer function': it now requires a fairly involved sequence to replicate in AVX-512. Pining for other lost things: if we had the LRBni up- and down- convert instructions, a software texture sampler would be a lot more feasible.
- coherentpony 12y ago
  Does gcc support these new wider registers yet?
  - valarauca1 12y ago
    Yes and no. If you write the ASM you can use them, but it's code generation likely won't (but then I haven't checked for 2 months). Most things that can be vectorizes will be placed in SSE rather then AVX. Also the GCC generally sucks at optimizing for SSE, or determining when code should use SSE as opposed to standard registers. Generally speaking the LLVM backend does better SSE and vectorization code generation. But some think it used SSE to much/incorrectly. So TLDR no New wide registers are VERY new. The are barely supported, my processor has AVX2.0 and I have (still have to) one of these days set up perf properly because all of its fault codes aren't properly baked into the kernel yet (as of 3.17). (Sorry for the lack of references) Also the biggest draw back of AVX is they don't hold their state between context changes :/
- wtallis 12y ago
  Wider SIMD units may provide a linear performance increase for many workloads, but they have a super-linear impact on the up front cost of the chip, especially when you take into account the opportunity cost of those transistors: they could have been dedicated to something that might have also helped non-numerical workloads. Wide SIMD is great to have, but it doesn't come free, or else we wouldn't have GPUs.
  - deleted 12y ago
    [deleted]
- sounds 12y ago
  It may just be that Intel has abandoned this use case to the GPU. Which obviously is bad news for those of us who still rely on CPU SIMD. But wtallis' argument to the contrary is quite compelling: https://news.ycombinator.com/item?id=8801808 https://news.ycombinator.com/item?id=8801808
- berkut 12y ago
  You only get those performance gains if you've got the cache and memory bandwidth - which is often very lacking. E.g. AVX (8-wide) was added for SandyBridge, but it wasn't always that usable until Intel doubled the cache bandwidth with Ivy Bridge. With ray tracing the increase in performance going from SSE (4-wide) to AVX (8-wide) for BVH intersection was only ~25-30% - instead of the theoretical 100% increase). You're generally limited by memory bandwidth.
vardump 12y ago
Image processing, etc. stuff I sometimes write seems to benefit pretty linearly from wider SIMD registers, like in AVX2. It'd be useful to get up to a cache line wide SIMD registers. Cache line is the granularity memory subsystem works at anyways. I wonder when cache line size will be increased from 64 to 128 bytes. Hopefully that wouldn't affect total number of lines, though. Mere 512 entries in L1D cache is already such a pain.
TheLoneWolfling 12y ago
CISC is rapidly becoming a requirement, however. Power requirements scale as the square of the frequency, and also, perhaps even more importantly, leakage power, and hence, heat generated per unit area, increases as the process size decreases. As such, as process sizes shrink, less and less actual area of the chip can be active at any one time. ("Dark silicon"). In other words, you might as well have areas of the chip that aren't used most of the time, because most of the chip can't be active at once anyways.
varelse 12y ago
I think that the multiple 32-way SIMD units on GPUs demonstrates there's plenty of room for widening on CPUs before we hit diminishing returns. The question in my mind is how much more like a GPU should CPUs become and Xeon Phi demonstrated how easy it is to get the mix totally wrong.
- wtallis 12y ago
  There are clearly plenty of common workloads that can make use of arbitrarily wide SIMD units. But nothing about the GPU market provides a clear indication that those SIMD units are better grafted onto the CPU than delegated to a coprocessor. The programming model is certainly simpler if they're part of the CPU's normal instruction pipeline, but on the other hand most CPUs don't have the memory bandwidth to feed more than a handful of GPU-style SIMD units and certainly don't have the power delivery or dissipation capacity to handle the compute resources of a mid-range GPU in addition to the normal CPU. Having the big SIMD units on a coprocessor card seems to be working just fine for the kinds of workloads that really need a lot of vector processing power. Those workloads also seem to not benefit much from things like speculative out of order execution, which frees up a lot of die space and power draw budget for more SIMD units. To some extent it's inevitable that SIMD code can't be making too much use of CPU design features that benefit branchy code, or else the code couldn't be vectorized in the first place. The workloads that are making heavy use of SIMD on the CPU are mainly using it for bursts of number crunching where the problem is too small to be worth the latency penalty of moving it to the GPU, but that penalty is getting smaller with each generation. The small integrated graphics processors common on non-server CPUs can all but eliminate that penalty, further reducing the demand for extra-wide SIMD in the CPU cores.

vardump 12y ago

That's why I feel dynamic code generation is the future. JIT. It's madness to write high performance code to take CPU [1] and DRAM [2] specifics into account. It's easy to write something that performs well on one system. [1]: 32-bit or 64-bit -> how many registers are available. SSE2, SSE 4.1, AVX2, etc. Different prefetchers, cache configurations. Different instruction level limitations and performance. Differing amounts of cache with rather variable latencies. [2]: Different number of memory channels, different distance between DRAM page changes.

iopq 12y ago
Compilers generate static code that will branch to the correct code path based on your architecture. You don't need to JIT until there are so many paths your executable becomes too big.
- vardump 12y ago
  Any references about this compiler?
  - nitrogen 12y ago
    icc does this, as an example. It got Intel into trouble because they changed the compiler at one point to use the slowest path on AMD processors.
  - tmurray 12y ago
    https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gcc/Function-Attributes.html https://gcc.gnu.org/onlinedocs/gcc-4.7.4/gcc/Function-Attrib... and http://pasky.or.cz/dev/glibc/ifunc.c http://pasky.or.cz/dev/glibc/ifunc.c look up the target attribute and the ifunc attribute--it's basically a way to compile multiple versions of a function for different targets in a single source file and then use the dynamic linker to determine which one to resolve at runtime. obvious use is for things like optimized memcpy implementations.
    - vardump 12y ago