5 ms·
An Intel Programmer Jumps over Wall: First Impressions of ARM SIMD Programming
- glangdale 8y agoAuthor here. Happy to answer questions, endure abuse, or, best yet, be put in my place by someone with a Really Nice Table of ARM Latencies and Throughputs that Could Have Been Found If I Wasn't Such an Idiot. [ Note the title of the article was "Jumps over The Wall", in keeping with the dish. ]
- sam_lowry_ 8y agoHow do you feel working on code that is (will be) used by governments and organizations of all kinds to censor the Internet?
- glangdale 8y agoOne of the uncomfortable aspects of open source is that you don't get to choose who uses it. The majority of customers we were aware of were security customers looking to stop attacks on machines behind next-gen firewalls. We weren't aware of any "democracy suppression" use cases, but it would be naive to assume it's impossible. (edit - also worth noting that this question seems to refer to my old job; I don't currently work on Hyperscan, although I may go back to regex one day. I'm sympathetic with the line of questioning, though, so please stop downvoting the parent)
- 0xfaded 8y agoThe best you can do for an ARM latency table is ddging "aXX optimization guide" a57: http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Cortex_A57_Software_Optimization_Guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Co... a72: http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/cortex_a72_software_optimization_guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/co... Notably not public is the a53 version of this document. I did some work getting ORB_SLAM to run on the raspberry pi a couple years ago. My conclusion was that the instruction timings for NEON instructions are more or less the same as the a57, but just remember there are only two instructions issued per cycle and everything is in order. Edit: replace google with ddg
- glangdale 8y agoI've seen those. We also can get A55, A75 but notably not A76 either. And what's there is... terse. But generally OK (not a fan of the blank spot in latency/throughput tables, regardless of who is doing it). Sadly, ARM is the exception - Ampere, Marvell/Cavium and Apple don't have a _whiff_ of information like this out there.
- inetknght 8y agoI see your simdjson code appears to be C but has a dependency for jsoncpp. There's also a graph which compares several different json libraries except jsoncpp. I've used jsoncpp for some other projects and I'd be curious about your opinion and any comparisons you might have.
- glangdale 8y agoThat could be clearer, sorry. We compare against RapidJSON and sajson for the bulk of the paper and for the 'splash' graphics as these are the fastest two libraries that validate to a similar extent (gason is a little faster than RapidJSON but accepts some crazy stuff). In the paper, we do also include measurements against 7 other libraries including jsoncpp. It is an order of magnitude slower than we are on the measurements we took. I am not familiar enough with it to know whether that's meaningful; it may have a lot of other attributes that make it more useful for what you're doing than what we have. We're pretty barebones, especially at this stage.
- gok 8y agoYou're using C intrinsics instead of assembly, right? Are you sure the compiler isn't doing the scheduling well enough on its own?
- glangdale 8y agoIf you're think about algorithms in terms of how they might embed into a processor, you need to know the throughput and latency numbers even if the compiler is getting things right. For example, there are some nice possibilities with the TBL instruction (or PSHUFB, or VERMB, etc) for doing character class membership tests. Which version you use would have a lot to do with whether you think TBL is going to issue 2/cycle, 1/cycle or 0.5/cycle (just fr'instance) - you might design the algorithm quite differently. It's not just a case of picking the one design choice and then having the compiler schedule that - if you're not thinking about this during design, you're not going to be remotely near peak performance.
- gok 8y agoSure but the compiler can compile your intrinsics to different-but-equivalent instructions, and modern compilers actually do that pretty aggressively. Are you sure the code output was actually using the instructions you expected?
- burntsushi 8y agoOften times you wind up needing to change the entire algorithm to use different instructions with different latency/throughput numbers. The classic case off the top of my head (for Intel at least) is PCMPESTRI versus completely different approaches for substring search.
- glangdale 8y agoYes. I look at the asm all the time, having not fallen off the turnip truck yesterday.
- Rebelgecko 8y agoI'm not an expert assembly programmer and I'm definitely not an expert at using SIMD instructions, but the last time I created a NEON implementation of an algorithm I was able to beat GCC's output (from compiling C intrinsics) after an afternoon of tweaking things by hand. From looking at GCC's output it didn't seem like it was doing a great job interleaving ARM and NEON ops to take advantage of that pipeline, although maybe it's gotten better since GCC 5.
- snvzz 8y agoI'm curious about what you think about RISC-V's draft V extension.
- dragontamer 8y agoBTW: I would suggest you look into the closely related art of GPGPU programming. OpenCL, CUDA, or whatever. The architecture of GPUs is SIMD on steroids. You'll likely be very happy with the metrics associated with GPU-shared memory (NVidia) or Local-memory (AMD). NVidia publishes throughput metrics in their PTX page: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#maximize-instruction-throughput https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... AMD's information is harder to find, but it does exist: http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings -------- I haven't been doing too much SIMD programming myself, but I know the general intrinsics. I think ARM's biggest problem is documentation. Intel's intrinsics page is great: https://software.intel.com/sites/landingpage/IntrinsicsGuide/ https://software.intel.com/sites/landingpage/IntrinsicsGuide... ARM's intrinsics page is... not as good. But I guess still workable. As you've noted, ARM's intrinsics documentation is missing latency / throughput metrics. https://developer.arm.com/technologies/neon/intrinsics https://developer.arm.com/technologies/neon/intrinsics POWER9 by the way has latency/throughput metrics published. I'm most happy with POWER9 as an alternative CPU, too bad its so expensive though. ---------- Intel + Agner Fog is definitely the best documentation for how things work and optimization information. NVidia CUDA and POWER9 seem to be #2 and #3 with documentation. Since GPUs are available for cheap these days (under $300 and they plug into any motherboard), GPUs seem to be the best alternative computer these days to learn. Since they're decently documented, its good to explore IMO.
- glangdale 8y agoI got my start on performance programming doing GPGPU regex (a terrible idea in our context - networked IPS - but that's for another time). This was in 2006, so this meant doing GPGPU using "Cg", including such delights as using textures as arrays and floats for everything. CUDA 0.9 and a engineering sample G80 card were a revelation. I keep meaning to go back and do some GPGPU again. I am very fond of architectures that actually exist. GPU is a very different bet than SIMD but a lot of fun. Incidentally PTX is a fiction - more of an IR than something with true throughput and latency numbers. I don't know if anyone programs GPUs directly but that would be interesting.
- 8y ago
- Iwan-Zotow 8y agoScalar Vector Extensions?! Used to be Scalable VE
- glangdale 8y agoFixed, thanks. I type the word 'scalar' so much, apparently, I didn't notice.
- devit 8y agoLooks like the author is going to enjoy RISC-V SIMD (once it is finalized).
- glangdale 8y agoI'm not sure... I like the look of the bit manipulation stuff, but I'm not really a fan of the variable-length vector approach. I think these systems are built to make the world safe for matrix multiply - and simple math workloads - but the short-vector approach (e.g. the typical x86 SIMD style of doing things) is sometimes exactly what you want. I keep meaning to write more about this. I have found 3 categories of SIMD use in my own work: 1) Doing one thing a gazillion times (e.g. conventional SIMD). This works well on vector machines, of course. 2) Using SIMD registers to do "more stuff". So in a couple string matchers and regex matchers I've designed, you are really use using a SIMD register because it's bigger than a GPR register (duh). But this might be because you want to simulate a 512-bit NFA rather than a 64-bit NFA. 3) Using SIMD operations to do weird, irregular stuff where what you're really getting is a substitute for branchy code. I blogged about an example of this called "SMH" https://branchfree.org/2018/05/30/smh-the-swiss-army-chainsaw-of-shuffle-based-matching-sequences/ https://branchfree.org/2018/05/30/smh-the-swiss-army-chainsa...
- mastax 8y agoI'll grant that a lot of code can't be readily adapted to large vectors, but this doesn't seem like much of a problem for (proposed) RISC-V. If you're creating an algorithm that only works on a specific width, you can just `setvl` and assert that you have enough space available. You're likely targeting a specific CPU or class of CPU where you know there will be support for 256-bit vectors or whatever. If someone tries to run it on a cheap embedded CPU, they'll be disappointed that your code won't run on their 64-bit vectors, but this is no different from trying to run AVX code on an Intel Atom. I suppose it's hard to know until we can actually write code for it, but I haven't imagined a scenario where the RV model is significantly worse than packed SIMD, other than a tiny bit of vector configuration bookkeeping. I think that's a worthwhile tradeoff for getting simple, portable, fast code for elementwise operations and implementation flexibility. I'd love to be convinced otherwise, though.
- phaedrus 8y agoThis is by the same person who authored Hyperscan, which is also mentioned on this blog & worth a read.
- Const-me 8y agoAbout documentation, Intel’s is not great either. Otherwise I wouldn’t bother making this: https://github.com/Const-me/IntelIntrinsics https://github.com/Const-me/IntelIntrinsics
- jcranmer 8y agohttps://software.intel.com/sites/landingpage/IntrinsicsGuide/ https://software.intel.com/sites/landingpage/IntrinsicsGuide... is kept in my browser history. It gives pretty good documentation of most of the details of intrinsics (including timing information on some processors for some instructions), although it is missing the enum definitions.
- wmu 8y agoI did similar thing, but the output are unix man pages: https://github.com/WojciechMula/man-intrinsics https://github.com/WojciechMula/man-intrinsics
- Solar19 8y agoNice work. By the way, on the x86 simdjson did you use any of the string compare instructions in SSE 4.2? Or was AVX faster?
- glangdale 8y agoSSE4.2 is bollocks. It was DOA. There are better ways of doing most of it using PSHUFB and what's more, those ways were better when SSE4.2 arrived. It's only gotten slower, relatively speaking, as it is exiled to the edge of the die and has not been promoted to wider regs (AVX2, AVX512).
- blu42 8y agoProps to the author for the objective article. Re CA53 throughput and latencies, no -- I don't have the tables, but I've done my fair share of tbl/tbx mesurements: https://www.cnx-software.com/2017/08/07/how-arm-nerfed-neon-permute-instructions-in-armv8/ https://www.cnx-software.com/2017/08/07/how-arm-nerfed-neon-...