5 ms·

This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong. Sometimes it's really difficult to tell the

This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong. Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more. He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter. Of course I'm sure an expert in the field could tell instantly. I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome? He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that. I would love to know what industry people here think, and then I might study the video / findings with more attention.

glangdale 6y ago
Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish. Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great). The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten. I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ). AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-a-bit-bashers-perspective/ https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints). You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).
- glangdale 6y ago
  Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.
  - pantalaimon 6y ago
    RISC-V is also adding Vector extensions. The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one. https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html
    - glangdale 6y ago
      I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.
    - PaulHoule 6y ago
      The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.
    - Const-me 6y ago
      > so code can take advantage of newer hardware / degrade gracefully on less capable one I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff. 1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such. 2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.
PaulHoule 6y ago
A curious aspect of this is that Intel has been doing very well financially despite having been failing technologically for a long time (if you buy the old story that Intel's process lead was at the center of it's market leadership.) High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance. --- Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc. Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes. Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles. Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs. Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.
- TomVDB 6y ago
  I don’t understand your point that Nvidia (or ATI) would have gone under if it weren’t for deep learning or crypto. Crypto on GPU was only a relatively short lived flash in the pan, and deep learning has only been a meaningful revenue generator in the past 5 years or so. GPUs were doing just fine without that.
valuearb 6y ago
There are no good equity analysts. If they were good they’d run their own funds.
ksec 6y ago
This is the same guy who loudly proclaims Apple moved away from Intel because of bugs in Skylake and subsequent (x)lake. That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts. At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver? It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC. But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.

glangdale 6y ago

Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish. Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great). The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten. I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ). AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-a-bit-bashers-perspective/ https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints). You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).

glangdale 6y ago
Oh, I should note that an interest in large SIMD is not confined to Intel - the ARM world is (slowly) heading towards SVE (or so my friends from that quarter assure me). So it's not just a partisan pro-Intel or pro-x86 thing on my part.
- pantalaimon 6y ago
  RISC-V is also adding Vector extensions. The cool thing about those is that they are variable length, so code can take advantage of newer hardware / degrade gracefully on less capable one. https://gms.tf/riscv-vector.html https://gms.tf/riscv-vector.html
  - glangdale 6y ago
    I think going for variable length is a Bold Move, and am not sure I like it for an ISA that otherwise is trying to be conservative. I'll be more optimistic about it when I see SVE (a) on more than one HPC machine and (b) implemented in a variety of widths. I'll be impressed if it works, but I'm not going to be impressed ahead of time with it while it's an aspirational design because it has aspirations.
  - PaulHoule 6y ago
    The small, medium and large sizes built into arms neon accomplishes some scalability without going back to what vector units were like in the big hair 1980s.
  - Const-me 6y ago
    > so code can take advantage of newer hardware / degrade gracefully on less capable one I never user variable-length SIMD on CPUs, but I have substantial experience with current SIMD up to AVX2, and GPGPUs. I’m not sure I like that variable-length stuff. 1. Variable-length SIMD is only suitable for vertical operations. Many useful algorithms are only possible to vectorize with shuffles and such. 2. With registers size unknown to compilers, it’s impossible to create a calling convention to pass data in registers, as opposed to RAM. RAM is slow on modern hardware, and is becoming slower over time (in relation to compute speed; nanoseconds are fine). GPUs are OK with that because they are massively multithreaded, they switch threads instead of waiting for data from memory. CPUs can’t do that, they only have 2 hardware threads per core.

PaulHoule 6y ago

A curious aspect of this is that Intel has been doing very well financially despite having been failing technologically for a long time (if you buy the old story that Intel's process lead was at the center of it's market leadership.) High-volume production helped Intel crush SPARC, Alpha and most of other other high-end CPU architectures in the 1990s. Today ARM is the volume leader that threatens to overtake Intel in performance. --- Intel has segmented the market in ways that have been long-term harmful. The idea is that they look at every little feature and show that hyperscalers can get $5000 of value from feature X on the chip, so they fuse it off unless they pay $4999 for the special edition. Everybody else pays in die area, development costs, etc. Intel's I/O choices have long been about controlling the PC platform and when you see all of the "Game of Thrones" mergers involving companies like Marvel and Mellanox so much of it has to do with that. Intel has long been stingy with PCIe lanes -- on paper it might look like you have a lot of lanes but when you deal with the motherboard and (usually less than i9) CPU in from of you will find there are capricious restrictions about how you can use those lanes. Microsoft switched to Wayland (well, WDDM, Microsoft's Wayland-like framework) with Windows Vista and that was based on having a GPU that could do bulk operations on pixel rectangles. Intel was mortified that a ATI or NVIDIA GPU would become necessary to the PC experience so they created Intel Integrated "Graphics" that approached the minimum performance required for Vista. If it hadn't been for deep learning and crypto mining, NVIDIA might have gone under. If it hadn't been for the merger with AMD and the SoC(s) for the Xbox and Playstation, we might not AMD GPUs. Intel did not want to see a transition to PCI4 or an increase in memory bandwidth because they didn't sell products that could take advantage of it. NVIDIA did, so they pushed NVLink. Be it GPU, FPGA, or ASIC, you can't have a revolution in performance unless your workload is crazy compute-heavy or you have a revolution in memory and I/O bandwidth.

TomVDB 6y ago
I don’t understand your point that Nvidia (or ATI) would have gone under if it weren’t for deep learning or crypto. Crypto on GPU was only a relatively short lived flash in the pan, and deep learning has only been a meaningful revenue generator in the past 5 years or so. GPUs were doing just fine without that.

valuearb 6y ago

There are no good equity analysts. If they were good they’d run their own funds.

ksec 6y ago

This is the same guy who loudly proclaims Apple moved away from Intel because of bugs in Skylake and subsequent (x)lake. That is um..... a very CPU designer preoperative to say the least. And has been rebutted by other industry experts. At the end of the day, none of this matters. Would better SMT performance, Single Thread Performance, Higher Clock Speed, AVX512, etc etc matter if Intel cant deliver? It is also strange that he mentions die area concern. That is the least concern of Intel because they are vertically integrated. The cost accounting is entirely different to TSMC. But yes, MBAs were the problem. It started when Andy Bryant became the Chairman and the Patrick Gelsinger was pushed out.