15 ms·
Reducing the Performance Gap of Intel's MKL on AMD Threadripper
- RL_Quine 7y agoThat's some seriously scummy stuff from Intel.
- dekken_ 7y agorelated https://www.youtube.com/watch?v=eINDw7zH5dk https://www.youtube.com/watch?v=eINDw7zH5dk
- faeyanpiraat 7y agoMaybe, but please note this part of the article: "It is also possible that the resulting code path has some precision loss or other problems on AMD hardware. I have not tested for that!"
- RL_Quine 7y agoI'm a lot more willing to believe that Intel is attempting to cook benchmarks, than it being a decision made out of caution to save face for AMD if the results are incorrect. They sat for an excruciatingly long time making no obvious improvements to their processors, and then suddenly more than halved the price of everything the moment there was competition.
- catalogia 7y agoYeah, some companies might deserve the benefit of the doubt but Intel isn't one of them. They shouldn't be judged in a vacuum; Intel is a company with a long and nasty history of underhanded bullshit.
- creato 7y ago> than it being a decision made out of caution to save face for AMD if the results are incorrect. It's not about "saving face" for anyone, it's about the possibility of bug reports that need to be investigated, and possibly new code paths need to be added if a fix is necessary. > and then suddenly more than halved the price of everything the moment there was competition. This would have happened regardless of how much or little their products improved in the prior years. The price is what enough people are willing to pay, and that goes down if there's an alternative.
- Maxious 7y agoeg. "AMD shipped Ryzen 3000 with a serious microcode bug in its random number generator." "Windows users couldn't successfully launch Destiny 2, and Linux users in many cases couldn't even get their system to boot." https://arstechnica.com/gadgets/2019/10/how-a-months-old-amd-microcode-bug-destroyed-my-weekend/ https://arstechnica.com/gadgets/2019/10/how-a-months-old-amd...
- djmips 7y agoThe article currently states the Destiny 2 issue was unrelated to the bug in the random number generator.
- creato 7y agoSee the caveat at the end: > It is also possible that the resulting code path has some precision loss or other problems on AMD hardware. I have not tested for that! Intel doesn't test it either, which is why they disable it. You could argue that they should test and enable on AMD CPUs, but this isn't clearly scummy behavior (to me), it's not a zero cost decision they made solely to screw over AMD. Buying the hardware and testing it has a cost. It may not even be possible to test it to the same degree they test their own hardware. There may be modes, subtle differences between architecture versions, etc. that are hard to know about to test for.
- nemetroid 7y agoUntil the opposite has been shown, the assumption should be that a CPU that indicates that it supports AVX2 does support AVX2.
- roenxi 7y agoThat is not the case, this is at the boundary of software and hardware and real-world concerns dominate. It is quite possible that a library highly optimised to a specific hardware set could have ghastly bugs on hardware it isn't tested on and testing has a cost. Intel is being reasonable and in a sense prudent by only supporting hardware they understand very well. It isn't like they are sending the lawyers out to sue anyone writing high performance code on AMD chips - that would be unreasonable.
- the8472 7y agoThe usual approach (e.g. taken in the linux kernel, llvm, hotspot jvm) is to look at the cpuid capability flags and rely on them by default and only blacklist specific CPU models when they don't implement advertised features correctly, not the other way around.
- yxhuvud 7y agoThen don't test it. It is perfectly ok (and even expected) for Intel to not test on AMD. Just don't intentionally cripple it.
- jandrewrogers 7y agoThe caveat at the end of the article is grounded in reality. I've personally experienced material precision bugs when running Intel-targeted code on AMD microarchitectures. These edge cases are very difficult to test for. The only reason we ever detected instances of these bugs is because we ran numerical codes at massive scales (finding the edge cases by brute force) and customers occasionally ran identical test workloads across both Intel and AMD clusters and compared numerical results. Every so often, something would surface. It is not Intel's responsibility to ensure that there is no obscure edge case in AMD's microarchitecture that may cause the MKL algorithms to give incorrect results on AMD silicon. MKL's design is based in part on Intel knowing the internal details of their own silicon.
- Dylan16807 7y agoCorrect, it's not their responsibility to avoid AMD bugs. So if switching from fast possibly-buggy instructions to slow possibly-buggy instructions is some lazy attempt at avoiding bugs, they should stop doing it.
- jchw 7y ago> MKL's design is based in part on Intel knowing the internal details of their own silicon. This is just conjecture unless there’s a citation to support it. Besides, the microarchitecture design is bound to change dramatically over time just as a matter of course. Relying on things that they don’t document publicly is probably not a wise strategy. How would the teams coordinate not breaking each-other? I can imagine such a position would be incredibly unpopular amongst those working on silicon designs (and maybe not popular amongst those working on software, either.) Also, nobody has really presented a reasoning that justifies this behavior, since if it broke on AMD hardware, it still wouldn’t be Intel’s problem, and therefore perhaps they should just use the instruction sets as intended and documented instead.
- jandrewrogers 7y agoThis is literally discussed in the MKL documentation, which spends considerable time on numerical reproducibility across microarchitectures. You have a rather strong opinion for someone who can't be bothered to educate themselves on the subject matter. I'm not an Intel fanboi but casually dismissing this specific issue is just ignorance. Intel's algorithms, for better and worse, are very aware of the limitations of their silicon implementations.
- deleted 7y ago[deleted]
- zepearl 7y agoIs the "MKL" mentioned by the article referring to https://docs.anaconda.com/mkl-optimizations/ https://docs.anaconda.com/mkl-optimizations/ (or something similar)? If yes, then there is apparently an active engagement by Intel (therefore involving people&money) and I don't see anything wrong if they optimize their code for their CPUs and then ignore other manufacturers (AMD) - mooost probably they just don't have a reason about why they should improve their competition + trying to do that anyway (without having full insight about the "foreign" multiple CPU architectures) might generate even worse performance which in turn would backfire (e.g. "aaahh Intel is sabotaging AMD CPUs"). So, personally I would have no problem if when using iMKL ("intel MKL") the code would run faster on Intel CPUs by default and being conservative on other CPUs - but if I would own an AMD CPU then I would expect AMD to implement some similar optimization for their own CPUs (aMKL?).
- gameswithgo 7y agowhat you are talking about isn’t the situation. they are intentionally detecting cpu features wrong so as to not use features amd cpus have. nobody is expecting special amd optimization attention.
- zepearl 7y agoOk, if it's like that ("intentionally detecting cpu features wrong") then I would agree. Can a "CPU feature" (e.g. sse2, avx, aes, etc...) be implemented the same way between different architectures?
- gallexme 7y ago(Until someone has a more detailed answer) Both Intel and AMD are on the same X86 architecture, and sse2 etc are just extensions of the X86 architecture, u can check which feature flags a CPU has And the call/use of these features is the same for AMD or Intel
- BeeOnRope 7y agoThey explicitly look for the "GenuineIntel" product string and then basically disable all the efficient code paths if it isn't found, totally against their own advice for ISA extension detection [1]. https://www.agner.org/optimize/blog/read.php?i=49 https://www.agner.org/optimize/blog/read.php?i=49
- seriesf 7y agoThe real “big caveat” here is to make sure that this actually improves your real workload in real life with real inputs. Zen and Zen+ have a lot of edge cases, for example their pdep/pext support is just crazy slow and how slow it is depends on the input values (it gets slower if more bits are set, which is bananas). But this also goes for genuine intel hardware. MKL is not always optimal. Even -march=native does not always produce the best code. Your program may be faster with the latest instructions disabled.
- partingshots 7y agoIf you go to the MKL library, it specifically says it speeds up performance on non-Intel processors as well. That they would state this and still sabotage the performance for AMD processors by directly disabling performance enhancements for them shows to me that this was a calculated decision that was willfully made.
- seriesf 7y agoOr it shows that the mkl developers are aware of some obscene edge case that kills the performance of a major customer’s application. This article tested ONE thing.
- lhl 7y agoIt's certainly not an "obscene edge case" - MKL cripples AMD on every single matrix operation tested here: https://github.com/flame/blis/blob/master/docs/graphs/large/l3_perf_epyc_nt1.pdf https://github.com/flame/blis/blob/master/docs/graphs/large/...
- aloknnikhil 7y agoWe honestly don't know how this impacts the results as was noted at the end. I'd like to see the numbers from the same run w/ the env variable set on an Intel CPU. Perhaps this debug code does something drastically different?
- moonbug 7y agoThe MKL is, and ways has been, a marketing tool for Intel. They develop it to make their CPUs look good. Why should they help the other team? This really isn't news to anyone who cares about linear algebra library performance, it's just that it's passed from general consciousness in the decade since AMD was last relevant. For performance portability to non-Intel CPUs ooenblas and its ilk have always been the way to go. You might think it's a sharp practice, in which case, you're free to vote with your wallet.
- yarg 7y agoAt very least, because they're legally obliged to. The anti monopoly laws prohibit companies in Intel's position from doing what they're doing (and they also agreed not to in the settlement for the AMD lawsuit).
- greglindahl 7y agoWas that part of the settlement made public? I got subpoenaed by both sides (compiler vendor) but no one told me much about the settlement other than it was a small amount of money to Intel that was a big windfall for AMD.
- kbumsik 7y ago> At very least, because they're legally obliged to. Are you sure about that Intel MKL is also applied to that law? Intel MKL is a propriety software by Intel and its system requirements clearly states that it is Intel 64 and IA-32 only. Do you mean that all propriety software made by the hardware manufacturer should support competitor's hardware as well?
- lawrenceyan 7y agoThat's false. Intel's MKL library has been repeatedly marketed as a tool for all processors including non-Intel ones. Combined with this fact, Intel's choice to purposefully remove performance enhancements for AMD processor is very clearly in violation of antitrust law.
- m0zg 7y agoAMD needs to do something like https://github.com/intel/mkl-dnn https://github.com/intel/mkl-dnn and https://github.com/opencv/dldt https://github.com/opencv/dldt, either by talking to Intel to get them to accept contributions (vastly preferable), or, failing that, by forking the libs and implementing their own support. With recent work on efficient NN architectures the CPUs are pretty viable for most _inference_ (not training) tasks. DLDT/OpenVINO perf is particularly impressive. It does, however, extensively use JIT kernel generation through https://github.com/herumi/xbyak https://github.com/herumi/xbyak depending on the supported features for a particular chip (search for "mayiuse()" in the code to see examples). It's anyone's guess what it will detect and generate on an AMD chip however. Now that AMD is raking in more cash, it'd be a great time to invest some of that back into their dev ecosystem.
- gnufx 7y agoThe relevant free low-level library in this area is libxsmm, which drove MKL to improve for small matrix work by being faster a while ago. I don't know how it performs on AMD CPUs, or how much a priori detailed micro-architecture knowledge it requires, but I doubt the maintainer would refuse changes for AMD. It was referenced with no interest in https://news.ycombinator.com/item?id=16600347 https://news.ycombinator.com/item?id=16600347 How specific to Intel CPUs is MKL-DNN (now renamed)? When I looked at it, the CPU code seemed fairly generic SIMD.
- m0zg 7y agoMKL-DNN is not MKL though. I think MKL-DNN is fairly specific to Intel CPUs though if you want maximum performance. It'll necessarily make certain assumptions about instruction throughput and latency, something that's pretty much certain to be different on AMD in some cases. In super tight kernels such minute differences often boil down to much lower performance. So if AMD wants to do a good job, they'll need, at least in some cases, to detect AMD CPUs and JIT their own specialized kernels that perform well on AMD.
- gnufx 7y agoSure, but it wasn't immediately obvious it was that specific when I looked, hence the question.
- shaklee3 7y agoHas anyone seen similar benchmarks, but with BLIS? OpenBLAS is interesting, but BLIS would possibly perform better.
- edraferi 7y agoThe article says they’d like to include BLIS in the benchmark, but nobody has written a good Numpy Conda package with BLIS hooks yet
- fock 7y agobecause all of the people "benchmarking" cpus nowadays are just clueless self-advertisers...
- lhl 7y agoThe article was specific to AMD's BLIS fork, but just as an FYI, using standard BLIS w/ numpy (which I believe would still outperform OpenBLAS) is pretty straightforward: conda create -c conda-forge -n numpy-blis numpy "blas=*=blis"
- wjnc 7y agoAre there any reads you can recommend on how to recover from (possibly) failed experiments like these under pip / conda / linux in general? I've been doing R and Linux for two decades, but still am perfectly capable of totally breaking my python/jupyter-workflow by messing up dependencies without knowing how to recover. Would love to learn to remedy that gap and reducing the risk.
- paol 7y agoWhen dealing with purely python packages: always always work in python virtual envs. Conduct these kinds of experiments in separate venvs. If worse comes to worse, a reset consists simply in deleting the venv directory and installing from scratch. If an experiment involves system packages it's a little trickier. You can learn about Docker (which does for the whole system what venv does for python packages), but in most cases you may get away with just taking note of what packages you have installed/uninstalled, so you can revert the process later if needed.
- Abishek_Muthian 7y ago10th gen intel (Ice Lake) has more instructions under AVX-512 and are not limited to just Xeon processors. So wouldn't comparing with latest Ice Lake CPU using intel MKL instead of Xeon-W 2175 yield even larger performance gap for workloads taking advantage of the new instruction Subset? i.e. AVX-512 F, CD, VL, DQ, BW, IFMA, VBMI, VBMI2, VPOPCNTDQ, BITALG, VNNI, VPCLMULQDQ, GFNI, VAES.
- wmf 7y agoThe server/workstation version of Ice Lake is not available yet and I wouldn't recommend buying a quad-core Ice Lake laptop for high performance computing.
- Abishek_Muthian 7y agoBtw, AMD Ryzen 3900X used in the comparison isn't a server CPU as well.
- Teknoman117 7y agoIt's an unfortunate trend. I got a PR merged which removed an "Is GenuineIntel" check in the ZFS on Linux crypto layer. It checked for GenuineIntel, and only if found did it check cpuid. Dunno who originally checked that in, but it seemed pretty ridiculous to me. AMD has had an AES-NI implementation since Bulldozer. I feel it would have been acceptable to say if you had a mutant x86 CPU which set the AES-NI bit in CPUID and then had a non-compliant implementation l, that was on you.
- PedroBatista 7y agoProbably some guy trying to clean his ToDo list, but in recent times as we scratch more than the surface, there’s a real possibility that guy’s paycheck was “Genuine Intel” too.
- robin_reala 7y agoThat feels like an easy string to search for across open source projects. Are there any other low-hanging fruit in core libraries?
- eberkund 7y agoWould there be any merit in the OS hiding what CPU brand? Keep exposing what specific CPU features/instruction sets are supported but don't display whether it is AMD/Intel/Via. What's the benefit? I find it hard to believe there are many "optimizations" out there that despite the instruction sets being the same do different things based on the specific CPU model or brand. If I'm wrong and there is a substantial amount of software that requires that then perhaps it can be hidden behind some kind of request access popup.
- paol 7y agoFirst, the OS can't hide that because it's not a middleman in the execution of program code. When a program is running on a CPU it has access to the instructions that tell it what CPU it is. Second, there are already many ways for programs to get granular CPU feature information. It's entirely the software's fault if it decides to ignore that information and instead use a big "if" switch keyed solely on the manufacturer.
- pmjordan 7y agoFirst, the OS can't hide that because it's not a middleman in the execution of program code. When a program is running on a CPU it has access to the instructions that tell it what CPU it is. Technically, it could, using virtualisation extensions to x86. CPUID can be trapped by a hypervisor, and the hypervisor can supply whatever result it likes. This is how Qemu/KVM simulates different CPU models. Mainstream OSes don’t use virtualisation extensions for regular processes of course, and hypercalls would be even more expensive than regular syscall based context switches, so this is probably not something that is going to change in the short to medium term.
- gnufx 7y agoAs for a previous article like this -- it's pointless. Just use BLIS, like AMD do, and which is infinitely faster than MKL on non-x86 systems. https://github.com/flame/blis https://github.com/flame/blis