4 ms·

The megahertz-scaling "Free Lunch" was declared dead 15 years ago [http://www.gotw.ca/publications/concurrency-ddj.htm http://www.gotw.ca/publications/concurren

The megahertz-scaling "Free Lunch" was declared dead 15 years ago [http://www.gotw.ca/publications/concurrency-ddj.htm http://www.gotw.ca/publications/concurrency-ddj.htm] and it's been only getting deader. People are finally, grudgingly accepting that they must go parallel unless we want to see software performance stagnate permanently. For most people here, the issue has been obvious since before they learned to program. But, still they are putting off learning how to deal with it. The first, obvious answer to that is threading. But, in my experience, SIMD is a bigger bang for the buck for two reasons: 1) No synchronization problems. 2) Better cache utilization. It's not just that SIMD forces you to work in large, contiguous blocks. Fun fact: When you aren't using SIMD you are only using a fraction of your L1 cache bandwidth! A big challenge is that SIMD intrinsic-function APIs are weird. They have inscrutable function names and sometimes difficult semantics. What helped me greatly was going through the effort of writing #define wrappers for myself that just gave each function in SSE1-3 names that made sense to me. I don't expect many people to put in that effort. And, unfortunately, I don't have go-to recommendations for pre-existing libraries. Best I can do is: https://github.com/VcDevel/Vc https://github.com/VcDevel/Vc is working on being standardized into C++. It's great for processing medium-to-large arrays. https://ispc.github.io/ https://ispc.github.io/ is great for writing large, complicated SIMD features. https://github.com/microsoft/DirectXMath https://github.com/microsoft/DirectXMath is not actually tied to DirectX. It's has a huge library of small-vector linear algebra (3D graphics math) function. It used to be pretty tied to MS's compiler. But, I believe they've been cleaning it up to be cross compiler lately.

CyberDildonics 7y ago
Can you say more about non SIMD instructions not making full use of the L1 bandwidth? Is it just that even keeping all the integer units busy still doesn't equate to using all the bandwidth? I suppose that makes sense when adding up the numbers for clock cycles and bytes. I'm guessing this not common to point out since being limited to L1 cache bandwidth is so unlikely to be a program's main bottleneck.
- corysama 7y ago
  Intel's scalar pipelining does do an amazing job of keeping pipes busy. And, well-pipelined code can approximate SIMD performance. But, in practice to solidly get that kind of pipelining you need to pretty much write your scalar code as if you were emulating SIMD. But, the point is that a 4-byte load instruction leaves 12 bytes of bandwidth on the table for many architectures -even with a perfect L1 cache hit. I point it out because I usually get rebuttals that everything is memory bound (true) and that using the cache well is more important (true, but it turns out...).

CyberDildonics 7y ago

Can you say more about non SIMD instructions not making full use of the L1 bandwidth? Is it just that even keeping all the integer units busy still doesn't equate to using all the bandwidth? I suppose that makes sense when adding up the numbers for clock cycles and bytes. I'm guessing this not common to point out since being limited to L1 cache bandwidth is so unlikely to be a program's main bottleneck.

corysama 7y ago
Intel's scalar pipelining does do an amazing job of keeping pipes busy. And, well-pipelined code can approximate SIMD performance. But, in practice to solidly get that kind of pipelining you need to pretty much write your scalar code as if you were emulating SIMD. But, the point is that a 4-byte load instruction leaves 12 bytes of bandwidth on the table for many architectures -even with a perfect L1 cache hit. I point it out because I usually get rebuttals that everything is memory bound (true) and that using the cache well is more important (true, but it turns out...).