22 ms·

What's wrong with Intel, and how to fix it: Former principal engineer unloads

175 points by em3rgent0rdr 6y ago

Jonnax 6y ago
So Linus, and this guy is saying AVX512 was a bad idea. "“The state of software out there is really not favoring going larger vectors,” Piednoel said in the video. “In fact, you can see clearly in Cinebench for example—that is not one of my favorite benchmarks, especially for a laptop where it doesn’t make any sense—but you can see that AMD is winning the battle of throughput. It’s because they have more cores and they can afford to have more cores.”" But in other places I've seen a lot of hype for it but without much discussion on actual use cases. With now people saying that it isn't useful except to give intel leadership in benchmarks. Is anyone here actually using it and seeing a benefit?
- BeeOnRope 6y ago
  AVX-512 is Intel's best vector ISA to date and it is definitely useful beyond benchmarks. In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts). It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability. The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512. So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.
  - FullyFunctional 6y ago
    I'm sure AVX-512 is wonderful and you can easily produce examples where it makes loops go N times faster. That unfortunately is missing the point. Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it. The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral. Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
    - BeeOnRope 6y ago
      I'm not missing the point as I was responding to whether AVX-512 is useful in practice. It is. It is some badly designed extensions where the earlier ones were great: it's arguably better than the earlier ones. Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is. The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
      - FullyFunctional 6y ago
        Sorry if I misunderstood your point. > It's not like Intel (or any other vendor) is rolling back progress Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
      - tremon 6y ago
        To me it looks like you and the GP just have different definitions of "in practice". Your definition of "in practice" seems to exclude the (general) availability of these extensions, whereas the GP's definition specifically includes them: if, as a developer, you can't rely on certain features being available, you're forced to optimize for the non-AVX512 case anyway, and any time spent exclusively on AVX512 is additional cost for a small audience. So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
- atq2119 6y ago
  AVX-512 really has two parts. It widens vectors to 512 bits, but it also makes the instruction set much more orthogonal and clean. It also adds masking of lanes. These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.
  - dman 6y ago
    Unfortunately there is a clock penalty to using AVX-512 and it is only available on a small part of the chips that Intel ships (For the longest time it was only available on Xeons). AMD does not support AVX-512 yet. As a result no one really had the incentive to add support for it outside HPC where people have control of both the workload and the hardware it runs on. To further complicate matters there are multiple variants of AVX-512 and different chip families ship with different subset of the variants. In short Intel could not have done more to discourage use of AVX-512 if they tried.
    - atq2119 6y ago
      Right, that's kind of the point. They should have made the cleaned up and extended instruction set available everywhere at least for 256-bit vectors. That would have made the good, non-trade-off parts that people actually liked pervasive.
    - lagadu 6y ago
      The chip family variants problem can't be overstated, it's ridiculous. For example the laptop I bought last year with a flagship latest gen cpu has far more support than the flagship latest gen desktop cpu that I bought two months ago. For reference: https://twitter.com/InstLatX64/status/1131489307473174528/photo/1 https://twitter.com/InstLatX64/status/1131489307473174528/ph...
    - me_me_me 6y ago
      Its not magic, everything has trade-offs. The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).
- glangdale 6y ago
  I've used it a fair bit, but it's too exotic to show up on enough machines to really focus on. It's not just a trick to make benchmarks run faster; that's an idiotic conspiracy theory. It's just that the release of AVX-512 happens to coincide with the 10nm disaster. Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core. Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).
- me_me_me 6y ago
  > and this guy is saying AVX512 was a bad idea. He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly. Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.
deleted 6y ago
[deleted]
supernova87a 6y ago
This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong. Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more. He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter. Of course I'm sure an expert in the field could tell instantly. I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome? He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that. I would love to know what industry people here think, and then I might study the video / findings with more attention.
- glangdale 6y ago
  Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish. Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great). The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten. I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ). AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-a-bit-bashers-perspective/ https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints). You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).
aPoCoMiLogin 6y ago
> Ryzen’s “Hyper-Threading” looked good because of poor single-threaded performance So if my CPU is one of the worst in single-threaded performance, then multi-threaded performance will be the best? That make no sense and this guy doesn't seem like he knows what he is talking about..
- FullyFunctional 6y ago
  He's completely right. Hyperthreading is all about putting idle functional units to work when you can't extract instruction level parallelism from single threaded code. His claim is that Intel processors did better and thus only got a small relative boost from HT (IIRC it used to be 10%, maybe it's 20% now). If your processor does poorer at ILP extraction, you get relatively more out of HT. This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. For a more extreme case look to older POWER processors which were 4-way threaded, presumably because they were even worse on single threaded code.
  - aPoCoMiLogin 6y ago
    > This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. citation needed. benchmarks that I've seen [0][1] there is not that big of difference between the two. [0] https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/5 https://www.anandtech.com/show/15578/cloud-clash-amazon-grav... ZEN1 [1] https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 ZEN2
    - FullyFunctional 6y ago
      https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/processor-benchmarks Even normalizing for the frequency isn't enough to explain the difference
      - kasabali 6y ago
tails4e 6y ago
I've always wondered, if I compile an application and don't specify the architecture as explicitly having AVX, just say x86 64bit, will alternate code paths be in the binary for processors with AVX? Or alternatively, if I do specify AVX support on the command line, then what happens if the binary runs on a CPU without AVX? The reason I ask is I use commercial tools for circuit simulation, and they run on a wide range of CPU architectures, but it would be a shame if thst CPU compatability comes at a performance hit when run on a more capable CPU architecture
- mojuba 6y ago
  Wondered the same. I'm also curious if Apple's Accelerate framework uses AVX underneath when available. Sounds like an OK compromise if it's the case: maybe the compiler will miss a few opportunities to optimize some of your loops, but if you are aware of Accelerate you will use it in your audio/video processing anyway, which should be a win.
  - saagarjha 6y ago
    It does indeed. And NEON on ARM, of course.
- pabs3 6y ago
  I depends on the compiler you use and the approaches it has to optional CPU features and the options you pass to the compiler. For example, with GCC you can set the baseline CPU features. You can also use function multi-versioning to provide faster functions for CPUs with newer features. https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
- glangdale 6y ago
  Answer is generally no; you have to compile something with specific flags for a given architecture, and you will normally only get one path. There may be exceptions for some libraries which may do clever stuff, but generally you pick your level of machine support (usually low) and live with it. Note that there are many new machines (Atoms and low-brand stuff like "Pentium" branded CPUs - even recent ones based on Comet Lake) which don't have anything beyond SSE4.x). IIRC they don't even have BMI/BMI2 which is a huge pain.
bosswipe 6y ago
None of that matters as long as Intel can't ship their 10nm and 7nm nodes. This guy doesn't seem to have an opinion on what's wrong at the fabs but that is the big life or death question for Intel right now.
- PaulHoule 6y ago
  I am glad to hear Intel is dabbling at TMSC (they have a plan B if not an actual business plan) but it is painful to see Intel talking as if nothing was wrong while congressmen are waking up to the reality that we might not be making chips in the us anymore. And that's what is scary. Intel is failing left and right but the rap track from Intel sounds like an ad for Disneyland.
  - justinclift 6y ago
    Wonder if the fabs being developed in China are or will be competitive to Intel in the ~near future?
ENOTTY 6y ago
Link to the video that the article is summarizing https://www.youtube.com/watch?v=fiKjzeLco6c https://www.youtube.com/watch?v=fiKjzeLco6c The creator's Twitter can be found here https://twitter.com/FPiednoel https://twitter.com/FPiednoel
- Jonnax 6y ago
  https://twitter.com/FPiednoel/status/1290874265705697280?s=19 https://twitter.com/FPiednoel/status/1290874265705697280?s=1... Looks like his video is him promoting himself to investors to try to get himself on the Intel board of directors.
  - Google234 6y ago
    That seems a bit wacky to be fair.
  - markus_zhang 6y ago
    That's fair. I mean if he wants to change that he has to be in the board. But I think investors like less risky ways to make revenue (i.e. by cutting costs)
PhantomGremlin 6y ago
From the article: Piednoel didn’t spare words for Intel’s culture, which he said has changed drastically and promotes MBAs over those with technical prowess. The current CEO is a finance / MBA type, so it's no surprise that the current culture favors that. In Intel's defense, the previous CEO was a process engineer. But, unfortunately, he couldn't keep his weiner in his pants. He also was probably the wrong choice as CEO since, under his watch, Intel process development apparently went to shit. Oh, well. Intel was a great company for many years. Perhaps it will reinvent itself.
- BeetleB 6y ago
  The previous CEO, who was a process engineer, was the one who brought the current CEO to Intel. It is widely believed the reasons for his firing were merely a pretext - they felt he was performing poorly and wanted to find an excuse to get rid of him. His focus definitely wasn't on CPUs - just look at all the acquisitions under his watch, and the focus away from CPUs into everything else (AI, autonomous vehicles, etc). He also sold all the stock he conceivably could during his tenure: https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-shares-after-the-company-already-knew-about-massive-security-flaws.html https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-sh...
  - dijit 6y ago
    The previous CEO was trying to diversify the company because CPU dominance has diminishing returns. Obviously this has backfired tremendously, as we now see the rise of ARM in the processor space and even AMD to conquer x86. But intel still make the best consumer networking cards! For whatever that is worth.
    - duhast 6y ago
      > But intel still make the best consumer networking cards! For whatever that is worth. Could you be more specific?
- nine_k 6y ago
  As somebody noted, when you have cornered a large part of the market, you don't feel as much technical pressure, and success in selling becomes key. So sales and MBAs start to run the show, because it makes the business sense, and while doing so risk to lose the sight of the technological advances that made the dominant position possible. Intel's founder, Andy Grove, used to say: "Only the paranoid survive". (He personally survived under a false identity in Nazi-occupied Hungary.) I'm afraid Intel had stopped being paranoid enough before it started to stumble more and more seriously.
ENOTTY 6y ago
I disagree that Intel has "lost focus" to the detriment of its core competencies, after all, Intel is a large company that spends billions on R&D every quarter. Rather I call it Intel expanding its focus to expand its addressable market to include 5G, AI, autonomous driving, advanced memory, software-defined networking, etc. I think Francois is focused on Intel being a CPU manufacturing company, when Intel sees itself as a data-processing company. It's important to note that these aren't totally unrelated pushes (a la Google X), but rather all of the new areas leverage Intel's core competency in silicon manufacturing. (Though, many of these products are due to be fabbed at TSMC, according to rumors.) "Xeon should dump unused core space" - Intel already does this. Intel has two CPU microarchitectures - Sunny Cove[1] (used for desktops, laptops, workstations, and beefy servers) and Tremont[2] (used for low-power clients and specialized servers). Tremont ditches the AVX instructions. Notably, Tremont is used in specialized server SoCs that target the storage, networking, and IoT/embedded markets. Could Intel differentiate its product lineup even more? At the core level, one should ask, Are there IPs that should be added to Tremont or subtracted from Sunny Cove? Can some IPs be added at the SoC level? [1]: https://en.wikichip.org/wiki/intel/microarchitectures/sunny_cove https://en.wikichip.org/wiki/intel/microarchitectures/sunny_... [2]: https://en.wikichip.org/wiki/intel/microarchitectures/tremont https://en.wikichip.org/wiki/intel/microarchitectures/tremon...
LordHeini 6y ago
It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them. As a consumer i care way more about my CPU actually working as advertised and not some Specre mitigation killing the performance. The price performance ratio of AMD is just better, nobody cares about some weird extension if i can have a Threadripper with a bazillion cores for less money doing the same stuff. Even only for marketing reasons. And GPUs stole the vector cake years ago anyway.
- Veedrac 6y ago
  > It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them. I recall him mentioning exactly this.
gok 6y ago
Whining about "MBAs" is almost always technical person speak for general discontent about company direction, often from the non-business parts. You don't see complaints about CEOs of Apple, Google, or Microsoft much, though they all have MBAs.
- lars 6y ago
  The problem isn't having an MBA though is it, it's (allegedly) not being competent enough in engineering. All the CEOs you mentioned have engineering degrees, while Intel's CEO does not.
- me_me_me 6y ago
  MBAs improve numbers, engineers create new products. The mentality of MBA crowd is to get the numbers right eat the cake and when things go sideways move on to a new job. Intel is a hardware tech business, its a complicated space that needs deep knowledge to understand it before you shape the future of it. Technical people might not be the only ones that should run business but they should be the core of it. I have same view as the video, engineers need more power and decision making not less to win. Intel will not fight back by cutting down cost of production by 5%. But by creating next gen chip.
  - deleted 6y ago
    [deleted]
  - mobilio 6y ago
    Same happens to Boeing. When they have engineers - we get 737, 747 and many more. when they have MBAs - we get 737-MAX, 787.
    - me_me_me 6y ago
      Thats a great direct example of the thought process. We will save x% by doing workarounds and call it MAX instead of biting the bullet spending money and creating next iconic plane design.
dathinab 6y ago
I'm not knowing how much of this things are true, but isn't it ironic that AMD has more benefits from SMT because they have less pipeline optimizations (and similar), but exactly this pipeline optimizations (and similar) seem to be at least partially at fault for some of the Specter style attacks and make it harder to put more cores into the system (as this additional optimizations are likely need more silicon space...). Sometimes I'm wondering if the best (but software side maybe impractical) design for (non Threadripper level) consumer PC's would be something along the line of. - 1 single non SMT core with most advanced pipeline optimizations and highest clock speed which only always runs one thread pinned to it (many consumers tend to only run one "costy" applications with need for highest performance at the same time, e.g. a game main event loop). - around this 4+ SMT enabled cores which are (mostly/reasonably) specter save and have less max. clock speed (other open applications, other threads from the current game, should be good enough for most applications) - 1+ very low power cores which might have another arch (for the background maintenance tasks of the OS and programs like e.g. update jobs, slow long polling, always-on/connected features etc.) - 1+ crypto core with integrated hardware security module, used to thinks like signing AES keys and similar (e.g. setup but not run TLS) Through it's probably unrealistic as e.g. thermal regulations for high speed threads is optimized by "bouncing" the logical thread between cores and software tends to not want to do things like write the background maintenance code for a different arch then the rest of the software, well phone OS probably could force this. Microsoft + Intel kinda can't.
- Scaless 6y ago
  You are more or less describing the architecture of the PlayStation 3: https://en.wikipedia.org/wiki/PlayStation_3_technical_specifications#Central_processing_unit https://en.wikipedia.org/wiki/PlayStation_3_technical_specif... The problem with this type of system is both the software and OS have to be aware of what is or isn't a "main thread" job. You have to architect your whole program around job systems to spread your data between the main thread and sub-processes which was a major complaint of gamedevs for the PS3. There's also the fact that many consumer software developers just don't care and will flag all of their processes as high priority jobs. It worked out decently well on PS3 because it was such a locked down system, but for a general purpose desktop it would be utter chaos.
kingcobraninja 6y ago
Just a heads up, this site is impossible to read on mobile.
ilaksh 6y ago
I am honestly not enthusiastic about x86 anymore. It seems like basically an entrenched legacy architecture at this point. What are Intel's plans for HSA or RISC-5? How good are their AI products? RealSense seems pretty exciting to me. Is there going to be a successor to the PC design? My crazy hope is that we will get some type of module system where you don't have to open the case and can just plug in something like M.2 modules but with a superfast nextgen bus. Is there an upstart chipmaker that can help push Intel out (I wish)? I have heard good things about Nuvia. When I hear about a company that is not run by engineers making bad decisions and pissing off engineers, I am actually eager to see them die. Just like I wish Boeing would die for similar reasons. But I know that both are very unlikely to go away. One can always dream.
SomeoneFromCA 6y ago
Speaking of AVX-512: I find it very odd, that they still produce deaktop CPU's without AVX (not even AVX2) at all (10th gen Celerons), yet they are pushing AVX-512 into laptops at the same time.
- jdsully 6y ago
  They produced a few low volume 10nm laptop parts so they could say 10nm has finally shipped. They can’t produce enough volume to move their whole product line over.
crb002 6y ago
Except AWS and Apple can and do recompile the ocean, even over to ARM. AVX512 free chips and gutting legacy X86 are worth it, perhaps an x86_64_2020 target?

175 points by em3rgent0rdr 6y ago

So Linus, and this guy is saying AVX512 was a bad idea. "“The state of software out there is really not favoring going larger vectors,” Piednoel said in the video. “In fact, you can see clearly in Cinebench for example—that is not one of my favorite benchmarks, especially for a laptop where it doesn’t make any sense—but you can see that AMD is winning the battle of throughput. It’s because they have more cores and they can afford to have more cores.”" But in other places I've seen a lot of hype for it but without much discussion on actual use cases. With now people saying that it isn't useful except to give intel leadership in benchmarks. Is anyone here actually using it and seeing a benefit?

BeeOnRope 6y ago
AVX-512 is Intel's best vector ISA to date and it is definitely useful beyond benchmarks. In fact, too low penetration of AVX-512 has been a problem, rather than too much: for a long time it has only been available on server chips, not laptop (a small fraction of laptops have gotten it recently) or destkop (outside some low-volume "extreme" parts, which were really just rebadged server parts). It would also be quite unusual that some instructions could be very useful in benchmarks that are based on real-life, heavily used applications, but not in real life. Outside of small, easily gamed benchmark that doesn't seem plausible: if the CPU is good a video encoding benchmark, it will be good at video encoding, with high probability. The main problem with AVX-512 is lack of software exploitation. Unlike with frequency boosts, increased cache sizes, better branch predictors, etc: this speedup doesn't come for free. Either compilers have to use the new instructions, or people have to use them by hand. The former has been very limited because these instructions cause a frequency drop (so-called "license based downclocking"), so compilers have mostly disabled their use by default: otherwise, a single AVX-512 instruction could cause a large impact on surrounding code which doesn't use AVX-512. So by-hand exploitation remains, and penetration has just been too low and the people with the skills to do this are limited.
- FullyFunctional 6y ago
  I'm sure AVX-512 is wonderful and you can easily produce examples where it makes loops go N times faster. That unfortunately is missing the point. Microprocessors implementation is an extremely careful balance and effectively everything is a trade off. The area, power, and complexity of features comes at a non-zero cost to code that doesn't use it. The fact that AVX-512 isn't on every processor AND the performance benefits are FAR from obvious (as the core clocks down when using it) really hinders its adoption, which in turn lowers the value; a downward spiral. Intel is in a fine mess of their own making. I'm no fan of either company and AMD gets perhaps a little too much credit as things would have looked a lot different had 10 nm succeeded on the original schedule, but I bought my first AMD processor since Athlon 64 because it's better.
  - BeeOnRope 6y ago
    I'm not missing the point as I was responding to whether AVX-512 is useful in practice. It is. It is some badly designed extensions where the earlier ones were great: it's arguably better than the earlier ones. Now adoption has been poor and there is a chicken-and-egg effect as you point out, but that's separate from the question of whether AVX-512 is useful in practice. It is. The point about tradeoffs is well taken. It's not like Intel (or any other vendor) is rolling back progress in other areas to add larger vector units: rather they try to progress along all viable axes at once.
    - FullyFunctional 6y ago
      Sorry if I misunderstood your point. > It's not like Intel (or any other vendor) is rolling back progress Performance doesn't stand still so even if they are improving, they might have improved more having spend the AVX-512 on something else. And most definitely, larger vector unit does cost you somewhere else.
    - tremon 6y ago
      To me it looks like you and the GP just have different definitions of "in practice". Your definition of "in practice" seems to exclude the (general) availability of these extensions, whereas the GP's definition specifically includes them: if, as a developer, you can't rely on certain features being available, you're forced to optimize for the non-AVX512 case anyway, and any time spent exclusively on AVX512 is additional cost for a small audience. So, whether the instructions themselves are any good then is of secondary concern because of the other problems of Intel's own making, and you both seem to be in agreement about those.
atq2119 6y ago
AVX-512 really has two parts. It widens vectors to 512 bits, but it also makes the instruction set much more orthogonal and clean. It also adds masking of lanes. These latter parts are what's really useful about AVX-512. The widening of the vectors is just an area tradeoff that depends a lot on the particular workload.
- dman 6y ago
  Unfortunately there is a clock penalty to using AVX-512 and it is only available on a small part of the chips that Intel ships (For the longest time it was only available on Xeons). AMD does not support AVX-512 yet. As a result no one really had the incentive to add support for it outside HPC where people have control of both the workload and the hardware it runs on. To further complicate matters there are multiple variants of AVX-512 and different chip families ship with different subset of the variants. In short Intel could not have done more to discourage use of AVX-512 if they tried.
  - atq2119 6y ago
    Right, that's kind of the point. They should have made the cleaned up and extended instruction set available everywhere at least for 256-bit vectors. That would have made the good, non-trade-off parts that people actually liked pervasive.
  - lagadu 6y ago
    The chip family variants problem can't be overstated, it's ridiculous. For example the laptop I bought last year with a flagship latest gen cpu has far more support than the flagship latest gen desktop cpu that I bought two months ago. For reference: https://twitter.com/InstLatX64/status/1131489307473174528/photo/1 https://twitter.com/InstLatX64/status/1131489307473174528/ph...
  - me_me_me 6y ago
    Its not magic, everything has trade-offs. The bigger and more powerful AVX instruction the more power it needs - more power results in more heat, heat that modern cpus cannot handle (reason why we dont see higher cloked cpus).
glangdale 6y ago
I've used it a fair bit, but it's too exotic to show up on enough machines to really focus on. It's not just a trick to make benchmarks run faster; that's an idiotic conspiracy theory. It's just that the release of AVX-512 happens to coincide with the 10nm disaster. Probably the biggest problem is that new Intel ISAs often are associated with downclocking (this isn't new with AVX-512 - AVX2 had it as well). Usually these problems were short-lived, but with 10nm getting stalled, they've stuck around longer. This is pure poison if like me, you generally write libraries. It's scary to look at some 60% speedup you might get from some neat rewrite for your bit of the task, but then wonder whether you're going to make the rest of the task several percent slower due to a downclock in what might only be 20% of the time. So it's hard to make the call to use AVX-512 in its early stages unless you control the whole core. Linus thinks that everything is either a kernel or a gcc execution, and happily disregards anything that doesn't fit his own workloads. AVX-512 can do Good Things for a disparate range of stuff - in-memory databases, regex/string matching, bioinformatics, CPU intensive network workloads, parsing and lexing as well as the usual scientific computing and AI workloads (many of the latter are also pretty good on GPGPU).
me_me_me 6y ago
> and this guy is saying AVX512 was a bad idea. He said that its a bad idea to put it in every xeon chip. AVX is great tool for specific high throughput oncore calculations, when offloading to gpu would be too costly. Saying AVX512 is universally bad is like saying that welding machines are bad tools since carpenters never make use of it.

deleted 6y ago

[deleted]

supernova87a 6y ago

This is purely a total outsider's opinion, so I caveat that and please don't get all offended if I'm totally wrong. Sometimes it's really difficult to tell the difference between a crackpot theorist and a good equities analyst. Watching this guy's video, it's hard to tell which he comes off as more. He's just coherent and convincing enough that as an outsider I could believe his collection of observations. But the presentation is also just disorganized enough, fixated on certain details enough, and a scattered collection enough that I can't tell whether these factors matter. Of course I'm sure an expert in the field could tell instantly. I've seen enough interviews of former engineers (usually retired guys) of whatever <xyz> company who know about a very specific fault of the product/tool/etc and can claim convincingly up and down that it will cause catastrophic failure, fundamental strategy misstep, etc. But will it really be a major factor for overall outcome? He proceeds from analysis of the smallest die area detail to the opposite end of the spectrum, the problem with MBAs + Marketing, within 1 minute. With typos. Also, the guy doesn't have a long history of posting analysis of this type, and he's honest about his info being 8 years old. Not sure what his motivation for making the video is. So it's hard to put too much weight on it, as if were like a Ming-Chi Kuo predicting Apple's next iPhone form factor or something like that. I would love to know what industry people here think, and then I might study the video / findings with more attention.

glangdale 6y ago
Francois is a smart guy, but there's a range of opinions on the matters he raises. We argue a good deal on Twitter ("constructive confrontation" between former Intel Principal Engineers :-) ). He put together the video pretty quickly so don't dismiss it just for lack of polish. Personally, I argue with him a lot on AVX-512. I think AVX-512 is a Good Thing (or will be shortly - the first instantiation in Skylake Server - SKX - isn't great). The biggest meta-problem with AVX-512 is that due to process issues, the pause button got hit just as it appeared in a server-only, 'first cut' form. AVX2 had downclocking issues when it first appeared too - but these were rapidly mitigated and forgotten. I personally feel that SIMD is underutilized in general purpose workloads (partly due to Intel's fecklessness in promoting SIMD - including gratuitous fragmentation - e.g. a new Comet Lake machine with Pentium branding doesn't even suport AVX2 and has it fused off). Daniel Lemire and I built simdjson (https://github.com/lemire/simdjson https://github.com/lemire/simdjson) to show the power of SIMD in a non-traditional domain, and a lot of the benefits of Hyperscan (https://github.com/intel/hyperscan https://github.com/intel/hyperscan) were also due to SIMD. Notably, both use SIMD in a range of ways that aren't necessarily all that amenable to the whole idea of "just do it all on an accelerator/GPU" - choppy mixes of SIMD and GPR-based computing. Not everything looks like matrix multplication, or AI (but I repeat myself :-) ). AVX-512 is a combination of fascinating and frustrating. I enthuse about recent developments on my blog (https://branchfree.org/2019/05/29/why-ice-lake-is-important-a-bit-bashers-perspective/ https://branchfree.org/2019/05/29/why-ice-lake-is-important-...) so I won't repeat why it's so interesting, but the learning curve is steep and there's plenty of frustrations and hidden traps. Intel collectively tends to be more interested in Magic Fixes (autovec) and Special Libraries for Important Benchmarks rather than building first-rate material to improve the general level of expertise with SIMD programming (one of my perpetual complaints). You more or less have to teach yourself and feel your way through the process of getting good at SIMD. That being said, the benefits are often huge - not only because SIMD is fast, but because you can do fundamentally different things that you can't do quickly in GPR ("General Purpose Register") land (look, for example, at everyone's favorite instruction PSHUFB - you can't do the equivalent on a GPR without going to memory).

aPoCoMiLogin 6y ago

> Ryzen’s “Hyper-Threading” looked good because of poor single-threaded performance So if my CPU is one of the worst in single-threaded performance, then multi-threaded performance will be the best? That make no sense and this guy doesn't seem like he knows what he is talking about..

FullyFunctional 6y ago
He's completely right. Hyperthreading is all about putting idle functional units to work when you can't extract instruction level parallelism from single threaded code. His claim is that Intel processors did better and thus only got a small relative boost from HT (IIRC it used to be 10%, maybe it's 20% now). If your processor does poorer at ILP extraction, you get relatively more out of HT. This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. For a more extreme case look to older POWER processors which were 4-way threaded, presumably because they were even worse on single threaded code.
- aPoCoMiLogin 6y ago
  > This all agree with the benchmarks in which Intel still generally is ahead on single thread performance. citation needed. benchmarks that I've seen [0][1] there is not that big of difference between the two. [0] https://www.anandtech.com/show/15578/cloud-clash-amazon-graviton2-arm-against-intel-and-amd/5 https://www.anandtech.com/show/15578/cloud-clash-amazon-grav... ZEN1 [1] https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 https://www.anandtech.com/show/14694/amd-rome-epyc-2nd-gen/9 ZEN2
  - FullyFunctional 6y ago
    https://browser.geekbench.com/processor-benchmarks https://browser.geekbench.com/processor-benchmarks Even normalizing for the frequency isn't enough to explain the difference
    - kasabali 6y ago

tails4e 6y ago

I've always wondered, if I compile an application and don't specify the architecture as explicitly having AVX, just say x86 64bit, will alternate code paths be in the binary for processors with AVX? Or alternatively, if I do specify AVX support on the command line, then what happens if the binary runs on a CPU without AVX? The reason I ask is I use commercial tools for circuit simulation, and they run on a wide range of CPU architectures, but it would be a shame if thst CPU compatability comes at a performance hit when run on a more capable CPU architecture

mojuba 6y ago
Wondered the same. I'm also curious if Apple's Accelerate framework uses AVX underneath when available. Sounds like an OK compromise if it's the case: maybe the compiler will miss a few opportunities to optimize some of your loops, but if you are aware of Accelerate you will use it in your audio/video processing anyway, which should be a win.
- saagarjha 6y ago
  It does indeed. And NEON on ARM, of course.
pabs3 6y ago
I depends on the compiler you use and the approaches it has to optional CPU features and the options you pass to the compiler. For example, with GCC you can set the baseline CPU features. You can also use function multi-versioning to provide faster functions for CPUs with newer features. https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
glangdale 6y ago
Answer is generally no; you have to compile something with specific flags for a given architecture, and you will normally only get one path. There may be exceptions for some libraries which may do clever stuff, but generally you pick your level of machine support (usually low) and live with it. Note that there are many new machines (Atoms and low-brand stuff like "Pentium" branded CPUs - even recent ones based on Comet Lake) which don't have anything beyond SSE4.x). IIRC they don't even have BMI/BMI2 which is a huge pain.

bosswipe 6y ago

None of that matters as long as Intel can't ship their 10nm and 7nm nodes. This guy doesn't seem to have an opinion on what's wrong at the fabs but that is the big life or death question for Intel right now.

PaulHoule 6y ago
I am glad to hear Intel is dabbling at TMSC (they have a plan B if not an actual business plan) but it is painful to see Intel talking as if nothing was wrong while congressmen are waking up to the reality that we might not be making chips in the us anymore. And that's what is scary. Intel is failing left and right but the rap track from Intel sounds like an ad for Disneyland.
- justinclift 6y ago
  Wonder if the fabs being developed in China are or will be competitive to Intel in the ~near future?

ENOTTY 6y ago

Link to the video that the article is summarizing https://www.youtube.com/watch?v=fiKjzeLco6c https://www.youtube.com/watch?v=fiKjzeLco6c The creator's Twitter can be found here https://twitter.com/FPiednoel https://twitter.com/FPiednoel

Jonnax 6y ago
https://twitter.com/FPiednoel/status/1290874265705697280?s=19 https://twitter.com/FPiednoel/status/1290874265705697280?s=1... Looks like his video is him promoting himself to investors to try to get himself on the Intel board of directors.
- Google234 6y ago
  That seems a bit wacky to be fair.
- markus_zhang 6y ago
  That's fair. I mean if he wants to change that he has to be in the board. But I think investors like less risky ways to make revenue (i.e. by cutting costs)

PhantomGremlin 6y ago

From the article: Piednoel didn’t spare words for Intel’s culture, which he said has changed drastically and promotes MBAs over those with technical prowess. The current CEO is a finance / MBA type, so it's no surprise that the current culture favors that. In Intel's defense, the previous CEO was a process engineer. But, unfortunately, he couldn't keep his weiner in his pants. He also was probably the wrong choice as CEO since, under his watch, Intel process development apparently went to shit. Oh, well. Intel was a great company for many years. Perhaps it will reinvent itself.

BeetleB 6y ago
The previous CEO, who was a process engineer, was the one who brought the current CEO to Intel. It is widely believed the reasons for his firing were merely a pretext - they felt he was performing poorly and wanted to find an excuse to get rid of him. His focus definitely wasn't on CPUs - just look at all the acquisitions under his watch, and the focus away from CPUs into everything else (AI, autonomous vehicles, etc). He also sold all the stock he conceivably could during his tenure: https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-shares-after-the-company-already-knew-about-massive-security-flaws.html https://www.cnbc.com/2018/01/04/intel-ceo-reportedly-sold-sh...
- dijit 6y ago
  The previous CEO was trying to diversify the company because CPU dominance has diminishing returns. Obviously this has backfired tremendously, as we now see the rise of ARM in the processor space and even AMD to conquer x86. But intel still make the best consumer networking cards! For whatever that is worth.
  - duhast 6y ago
    > But intel still make the best consumer networking cards! For whatever that is worth. Could you be more specific?
nine_k 6y ago
As somebody noted, when you have cornered a large part of the market, you don't feel as much technical pressure, and success in selling becomes key. So sales and MBAs start to run the show, because it makes the business sense, and while doing so risk to lose the sight of the technological advances that made the dominant position possible. Intel's founder, Andy Grove, used to say: "Only the paranoid survive". (He personally survived under a false identity in Nazi-occupied Hungary.) I'm afraid Intel had stopped being paranoid enough before it started to stumble more and more seriously.

ENOTTY 6y ago

I disagree that Intel has "lost focus" to the detriment of its core competencies, after all, Intel is a large company that spends billions on R&D every quarter. Rather I call it Intel expanding its focus to expand its addressable market to include 5G, AI, autonomous driving, advanced memory, software-defined networking, etc. I think Francois is focused on Intel being a CPU manufacturing company, when Intel sees itself as a data-processing company. It's important to note that these aren't totally unrelated pushes (a la Google X), but rather all of the new areas leverage Intel's core competency in silicon manufacturing. (Though, many of these products are due to be fabbed at TSMC, according to rumors.) "Xeon should dump unused core space" - Intel already does this. Intel has two CPU microarchitectures - Sunny Cove[1] (used for desktops, laptops, workstations, and beefy servers) and Tremont[2] (used for low-power clients and specialized servers). Tremont ditches the AVX instructions. Notably, Tremont is used in specialized server SoCs that target the storage, networking, and IoT/embedded markets. Could Intel differentiate its product lineup even more? At the core level, one should ask, Are there IPs that should be added to Tremont or subtracted from Sunny Cove? Can some IPs be added at the SoC level? [1]: https://en.wikichip.org/wiki/intel/microarchitectures/sunny_cove https://en.wikichip.org/wiki/intel/microarchitectures/sunny_... [2]: https://en.wikichip.org/wiki/intel/microarchitectures/tremont https://en.wikichip.org/wiki/intel/microarchitectures/tremon...

LordHeini 6y ago

It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them. As a consumer i care way more about my CPU actually working as advertised and not some Specre mitigation killing the performance. The price performance ratio of AMD is just better, nobody cares about some weird extension if i can have a Threadripper with a bazillion cores for less money doing the same stuff. Even only for marketing reasons. And GPUs stole the vector cake years ago anyway.

Veedrac 6y ago
> It is a bit odd that there is no mention of the awful security problems like Spectre and Meltdown which mostly hit Intel and was very poorly handled by them. I recall him mentioning exactly this.

gok 6y ago

Whining about "MBAs" is almost always technical person speak for general discontent about company direction, often from the non-business parts. You don't see complaints about CEOs of Apple, Google, or Microsoft much, though they all have MBAs.

lars 6y ago
The problem isn't having an MBA though is it, it's (allegedly) not being competent enough in engineering. All the CEOs you mentioned have engineering degrees, while Intel's CEO does not.
me_me_me 6y ago
MBAs improve numbers, engineers create new products. The mentality of MBA crowd is to get the numbers right eat the cake and when things go sideways move on to a new job. Intel is a hardware tech business, its a complicated space that needs deep knowledge to understand it before you shape the future of it. Technical people might not be the only ones that should run business but they should be the core of it. I have same view as the video, engineers need more power and decision making not less to win. Intel will not fight back by cutting down cost of production by 5%. But by creating next gen chip.
- deleted 6y ago
  [deleted]
- mobilio 6y ago
  Same happens to Boeing. When they have engineers - we get 737, 747 and many more. when they have MBAs - we get 737-MAX, 787.
  - me_me_me 6y ago
    Thats a great direct example of the thought process. We will save x% by doing workarounds and call it MAX instead of biting the bullet spending money and creating next iconic plane design.

dathinab 6y ago

I'm not knowing how much of this things are true, but isn't it ironic that AMD has more benefits from SMT because they have less pipeline optimizations (and similar), but exactly this pipeline optimizations (and similar) seem to be at least partially at fault for some of the Specter style attacks and make it harder to put more cores into the system (as this additional optimizations are likely need more silicon space...). Sometimes I'm wondering if the best (but software side maybe impractical) design for (non Threadripper level) consumer PC's would be something along the line of. - 1 single non SMT core with most advanced pipeline optimizations and highest clock speed which only always runs one thread pinned to it (many consumers tend to only run one "costy" applications with need for highest performance at the same time, e.g. a game main event loop). - around this 4+ SMT enabled cores which are (mostly/reasonably) specter save and have less max. clock speed (other open applications, other threads from the current game, should be good enough for most applications) - 1+ very low power cores which might have another arch (for the background maintenance tasks of the OS and programs like e.g. update jobs, slow long polling, always-on/connected features etc.) - 1+ crypto core with integrated hardware security module, used to thinks like signing AES keys and similar (e.g. setup but not run TLS) Through it's probably unrealistic as e.g. thermal regulations for high speed threads is optimized by "bouncing" the logical thread between cores and software tends to not want to do things like write the background maintenance code for a different arch then the rest of the software, well phone OS probably could force this. Microsoft + Intel kinda can't.

Scaless 6y ago
You are more or less describing the architecture of the PlayStation 3: https://en.wikipedia.org/wiki/PlayStation_3_technical_specifications#Central_processing_unit https://en.wikipedia.org/wiki/PlayStation_3_technical_specif... The problem with this type of system is both the software and OS have to be aware of what is or isn't a "main thread" job. You have to architect your whole program around job systems to spread your data between the main thread and sub-processes which was a major complaint of gamedevs for the PS3. There's also the fact that many consumer software developers just don't care and will flag all of their processes as high priority jobs. It worked out decently well on PS3 because it was such a locked down system, but for a general purpose desktop it would be utter chaos.

kingcobraninja 6y ago

Just a heads up, this site is impossible to read on mobile.

ilaksh 6y ago

I am honestly not enthusiastic about x86 anymore. It seems like basically an entrenched legacy architecture at this point. What are Intel's plans for HSA or RISC-5? How good are their AI products? RealSense seems pretty exciting to me. Is there going to be a successor to the PC design? My crazy hope is that we will get some type of module system where you don't have to open the case and can just plug in something like M.2 modules but with a superfast nextgen bus. Is there an upstart chipmaker that can help push Intel out (I wish)? I have heard good things about Nuvia. When I hear about a company that is not run by engineers making bad decisions and pissing off engineers, I am actually eager to see them die. Just like I wish Boeing would die for similar reasons. But I know that both are very unlikely to go away. One can always dream.

SomeoneFromCA 6y ago

Speaking of AVX-512: I find it very odd, that they still produce deaktop CPU's without AVX (not even AVX2) at all (10th gen Celerons), yet they are pushing AVX-512 into laptops at the same time.

jdsully 6y ago
They produced a few low volume 10nm laptop parts so they could say 10nm has finally shipped. They can’t produce enough volume to move their whole product line over.

crb002 6y ago

Except AWS and Apple can and do recompile the ocean, even over to ARM. AVX512 free chips and gutting legacy X86 are worth it, perhaps an x86_64_2020 target?