3 ms·

The hatred is probably not that "irrational". We live in an era where specialized hardware for specialized problems is required because new manufacturing proces

by sprash 4y ago

The hatred is probably not that "irrational". We live in an era where specialized hardware for specialized problems is required because new manufacturing processes may give us a (seemingly slowing down) increase of transistor budget but not really better switching frequencies. We will have units for matrix multiplication, video codecs, AI cores or full blown GPUs. All those units can only be fast in specialized hardware because of predictable memory access patterns and arranging memory/cache topology accordingly "solving" the problem of low switching frequency with high bandwidth. A general purpose CPU however should specialize on unpredictable memory access. This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks.

janwas 4y ago
The hatred is specifically directed at "FP" which I understand to be floating-point. Makes sense inside an OS kernel but a large majority of HPC would indeed consider this irrational. I understand that dark silicon is helpful, but am not so sure that fixed-function HW is the way to go. Perhaps video _de_coding is the most convincing from your list; codec generations are 5+ years, so enough time to benefit from HW. Encoding, on the other hand, tends not to be impressive unless perhaps there is also a software component. For the rest, programmability and deployability (can we rely on it?) is a major issue. Software has often been the limiting factor. Another big concern is the 'hardware lottery'. The algorithms we develop and get are selected for, and tuned to, the current hardware. Perhaps this gets us 5x energy efficiency vs CPU/SIMD. But by painting ourselves ever further into the corner of dense linear algebra, which is definitely not the way that nature implements intelligence, we are missing out on far larger opportunities. For example: spiking nets or memristors have the potential to be 2 or 3 orders of magnitude better. Or actual sparsity, not the fixed-pattern thing (now that is a prime example of an irrelevent benchmark, because AFAIK algorithms haven't yet been able to use them well). > A general purpose CPU however should specialize on unpredictable memory access. Should it really? I think rather we should avoid such accesses whenever possible, because their energy cost now dwarfs that of computation. > This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks. I have difficulty understanding how a reasonable person can come to such a conclusion. Lemire (the author linked here) has a long series of results showing nice speedups from AVX-512. I personally have seen gains in image compression, string processing, cryptography, linear algebra, integer coding, hash tables, databases, sorting, and compression. [Opinions are my own.]
- sprash 4y ago
  > I have difficulty understanding how a reasonable person can come to such a conclusion. The applications are very niche. Compilers are usually not smart enough to utilize SIMD, it is a hit or miss. And in order to implement properly efficient SIMD algorithms you need experts that are rare. Furthermore many algorithms that work great with SIMD work even better as compute shader on your run of the mill cheap iGPU. The application of this article is the best example how irrelevant SIMD really is: How many Terabytes of UTF8 are you converting to UTF16 per day? probably zero.
  - janwas 4y ago
    What leads you to think the list of applications I enumerated is 'niche'? > in order to implement properly efficient SIMD algorithms you need experts that are rare Some truth to this, but many algorithms can be implemented once and then reused, like a standard library. > many algorithms that work great with SIMD work even better as compute shader on your run of the mill cheap iGPU Also agree to some extent, except that you'd have more concerns about availability, vendor lock-in, and performance portability. > best example how irrelevant SIMD really is: How many Terabytes of UTF8 are you converting to UTF16 per day? probably zero. First, how does one example of a SIMD-enabled algorithm show that SIMD itself is irrelevant? Second, have you considered that some databases store UTF-16 and want to convert it for interoperability (or vice versa)? IBM apparently has dedicated instructions for this. Would they have been added if there was no demand?

by sprash 4y ago

janwas 4y ago

The hatred is specifically directed at "FP" which I understand to be floating-point. Makes sense inside an OS kernel but a large majority of HPC would indeed consider this irrational. I understand that dark silicon is helpful, but am not so sure that fixed-function HW is the way to go. Perhaps video _de_coding is the most convincing from your list; codec generations are 5+ years, so enough time to benefit from HW. Encoding, on the other hand, tends not to be impressive unless perhaps there is also a software component. For the rest, programmability and deployability (can we rely on it?) is a major issue. Software has often been the limiting factor. Another big concern is the 'hardware lottery'. The algorithms we develop and get are selected for, and tuned to, the current hardware. Perhaps this gets us 5x energy efficiency vs CPU/SIMD. But by painting ourselves ever further into the corner of dense linear algebra, which is definitely not the way that nature implements intelligence, we are missing out on far larger opportunities. For example: spiking nets or memristors have the potential to be 2 or 3 orders of magnitude better. Or actual sparsity, not the fixed-pattern thing (now that is a prime example of an irrelevent benchmark, because AFAIK algorithms haven't yet been able to use them well). > A general purpose CPU however should specialize on unpredictable memory access. Should it really? I think rather we should avoid such accesses whenever possible, because their energy cost now dwarfs that of computation. > This means AVX-512 is somewhat misplaced on a CPU and probably only exists because it served Intel to create nice numbers in irrelevant benchmarks. I have difficulty understanding how a reasonable person can come to such a conclusion. Lemire (the author linked here) has a long series of results showing nice speedups from AVX-512. I personally have seen gains in image compression, string processing, cryptography, linear algebra, integer coding, hash tables, databases, sorting, and compression. [Opinions are my own.]

sprash 4y ago
> I have difficulty understanding how a reasonable person can come to such a conclusion. The applications are very niche. Compilers are usually not smart enough to utilize SIMD, it is a hit or miss. And in order to implement properly efficient SIMD algorithms you need experts that are rare. Furthermore many algorithms that work great with SIMD work even better as compute shader on your run of the mill cheap iGPU. The application of this article is the best example how irrelevant SIMD really is: How many Terabytes of UTF8 are you converting to UTF16 per day? probably zero.
- janwas 4y ago
  What leads you to think the list of applications I enumerated is 'niche'? > in order to implement properly efficient SIMD algorithms you need experts that are rare Some truth to this, but many algorithms can be implemented once and then reused, like a standard library. > many algorithms that work great with SIMD work even better as compute shader on your run of the mill cheap iGPU Also agree to some extent, except that you'd have more concerns about availability, vendor lock-in, and performance portability. > best example how irrelevant SIMD really is: How many Terabytes of UTF8 are you converting to UTF16 per day? probably zero. First, how does one example of a SIMD-enabled algorithm show that SIMD itself is irrelevant? Second, have you considered that some databases store UTF-16 and want to convert it for interoperability (or vice versa)? IBM apparently has dedicated instructions for this. Would they have been added if there was no demand?