6 ms·

Author here. Happy to answer questions, endure abuse, or, best yet, be put in my place by someone with a Really Nice Table of ARM Latencies and Throughputs that

Author here. Happy to answer questions, endure abuse, or, best yet, be put in my place by someone with a Really Nice Table of ARM Latencies and Throughputs that Could Have Been Found If I Wasn't Such an Idiot. [ Note the title of the article was "Jumps over The Wall", in keeping with the dish. ]

sam_lowry_ 8y ago
How do you feel working on code that is (will be) used by governments and organizations of all kinds to censor the Internet?
- glangdale 8y ago
  One of the uncomfortable aspects of open source is that you don't get to choose who uses it. The majority of customers we were aware of were security customers looking to stop attacks on machines behind next-gen firewalls. We weren't aware of any "democracy suppression" use cases, but it would be naive to assume it's impossible. (edit - also worth noting that this question seems to refer to my old job; I don't currently work on Hyperscan, although I may go back to regex one day. I'm sympathetic with the line of questioning, though, so please stop downvoting the parent)
0xfaded 8y ago
The best you can do for an ARM latency table is ddging "aXX optimization guide" a57: http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Cortex_A57_Software_Optimization_Guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Co... a72: http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/cortex_a72_software_optimization_guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/co... Notably not public is the a53 version of this document. I did some work getting ORB_SLAM to run on the raspberry pi a couple years ago. My conclusion was that the instruction timings for NEON instructions are more or less the same as the a57, but just remember there are only two instructions issued per cycle and everything is in order. Edit: replace google with ddg
- glangdale 8y ago
  I've seen those. We also can get A55, A75 but notably not A76 either. And what's there is... terse. But generally OK (not a fan of the blank spot in latency/throughput tables, regardless of who is doing it). Sadly, ARM is the exception - Ampere, Marvell/Cavium and Apple don't have a _whiff_ of information like this out there.
inetknght 8y ago
I see your simdjson code appears to be C but has a dependency for jsoncpp. There's also a graph which compares several different json libraries except jsoncpp. I've used jsoncpp for some other projects and I'd be curious about your opinion and any comparisons you might have.
- glangdale 8y ago
  That could be clearer, sorry. We compare against RapidJSON and sajson for the bulk of the paper and for the 'splash' graphics as these are the fastest two libraries that validate to a similar extent (gason is a little faster than RapidJSON but accepts some crazy stuff). In the paper, we do also include measurements against 7 other libraries including jsoncpp. It is an order of magnitude slower than we are on the measurements we took. I am not familiar enough with it to know whether that's meaningful; it may have a lot of other attributes that make it more useful for what you're doing than what we have. We're pretty barebones, especially at this stage.
  - glangdale 8y ago
    (the broader comparison is in Table 9 at https://arxiv.org/pdf/1902.08318.pdf https://arxiv.org/pdf/1902.08318.pdf)
    - inetknght 8y ago
      That's pretty interesting, thanks!
gok 8y ago
You're using C intrinsics instead of assembly, right? Are you sure the compiler isn't doing the scheduling well enough on its own?
- glangdale 8y ago
  If you're think about algorithms in terms of how they might embed into a processor, you need to know the throughput and latency numbers even if the compiler is getting things right. For example, there are some nice possibilities with the TBL instruction (or PSHUFB, or VERMB, etc) for doing character class membership tests. Which version you use would have a lot to do with whether you think TBL is going to issue 2/cycle, 1/cycle or 0.5/cycle (just fr'instance) - you might design the algorithm quite differently. It's not just a case of picking the one design choice and then having the compiler schedule that - if you're not thinking about this during design, you're not going to be remotely near peak performance.
  - gok 8y ago
    Sure but the compiler can compile your intrinsics to different-but-equivalent instructions, and modern compilers actually do that pretty aggressively. Are you sure the code output was actually using the instructions you expected?
    - burntsushi 8y ago
      Often times you wind up needing to change the entire algorithm to use different instructions with different latency/throughput numbers. The classic case off the top of my head (for Intel at least) is PCMPESTRI versus completely different approaches for substring search.
    - glangdale 8y ago
      Yes. I look at the asm all the time, having not fallen off the turnip truck yesterday.
- Rebelgecko 8y ago
  I'm not an expert assembly programmer and I'm definitely not an expert at using SIMD instructions, but the last time I created a NEON implementation of an algorithm I was able to beat GCC's output (from compiling C intrinsics) after an afternoon of tweaking things by hand. From looking at GCC's output it didn't seem like it was doing a great job interleaving ARM and NEON ops to take advantage of that pipeline, although maybe it's gotten better since GCC 5.
snvzz 8y ago
I'm curious about what you think about RISC-V's draft V extension.
dragontamer 8y ago
BTW: I would suggest you look into the closely related art of GPGPU programming. OpenCL, CUDA, or whatever. The architecture of GPUs is SIMD on steroids. You'll likely be very happy with the metrics associated with GPU-shared memory (NVidia) or Local-memory (AMD). NVidia publishes throughput metrics in their PTX page: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#maximize-instruction-throughput https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... AMD's information is harder to find, but it does exist: http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings -------- I haven't been doing too much SIMD programming myself, but I know the general intrinsics. I think ARM's biggest problem is documentation. Intel's intrinsics page is great: https://software.intel.com/sites/landingpage/IntrinsicsGuide/ https://software.intel.com/sites/landingpage/IntrinsicsGuide... ARM's intrinsics page is... not as good. But I guess still workable. As you've noted, ARM's intrinsics documentation is missing latency / throughput metrics. https://developer.arm.com/technologies/neon/intrinsics https://developer.arm.com/technologies/neon/intrinsics POWER9 by the way has latency/throughput metrics published. I'm most happy with POWER9 as an alternative CPU, too bad its so expensive though. ---------- Intel + Agner Fog is definitely the best documentation for how things work and optimization information. NVidia CUDA and POWER9 seem to be #2 and #3 with documentation. Since GPUs are available for cheap these days (under $300 and they plug into any motherboard), GPUs seem to be the best alternative computer these days to learn. Since they're decently documented, its good to explore IMO.
- glangdale 8y ago
  I got my start on performance programming doing GPGPU regex (a terrible idea in our context - networked IPS - but that's for another time). This was in 2006, so this meant doing GPGPU using "Cg", including such delights as using textures as arrays and floats for everything. CUDA 0.9 and a engineering sample G80 card were a revelation. I keep meaning to go back and do some GPGPU again. I am very fond of architectures that actually exist. GPU is a very different bet than SIMD but a lot of fun. Incidentally PTX is a fiction - more of an IR than something with true throughput and latency numbers. I don't know if anyone programs GPUs directly but that would be interesting.
  - 8y ago
Iwan-Zotow 8y ago
Scalar Vector Extensions?! Used to be Scalable VE
- glangdale 8y ago
  Fixed, thanks. I type the word 'scalar' so much, apparently, I didn't notice.

sam_lowry_ 8y ago

How do you feel working on code that is (will be) used by governments and organizations of all kinds to censor the Internet?

glangdale 8y ago
One of the uncomfortable aspects of open source is that you don't get to choose who uses it. The majority of customers we were aware of were security customers looking to stop attacks on machines behind next-gen firewalls. We weren't aware of any "democracy suppression" use cases, but it would be naive to assume it's impossible. (edit - also worth noting that this question seems to refer to my old job; I don't currently work on Hyperscan, although I may go back to regex one day. I'm sympathetic with the line of questioning, though, so please stop downvoting the parent)

0xfaded 8y ago

The best you can do for an ARM latency table is ddging "aXX optimization guide" a57: http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Cortex_A57_Software_Optimization_Guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0015b/Co... a72: http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/cortex_a72_software_optimization_guide_external.pdf http://infocenter.arm.com/help/topic/com.arm.doc.uan0016a/co... Notably not public is the a53 version of this document. I did some work getting ORB_SLAM to run on the raspberry pi a couple years ago. My conclusion was that the instruction timings for NEON instructions are more or less the same as the a57, but just remember there are only two instructions issued per cycle and everything is in order. Edit: replace google with ddg

glangdale 8y ago
I've seen those. We also can get A55, A75 but notably not A76 either. And what's there is... terse. But generally OK (not a fan of the blank spot in latency/throughput tables, regardless of who is doing it). Sadly, ARM is the exception - Ampere, Marvell/Cavium and Apple don't have a _whiff_ of information like this out there.

inetknght 8y ago

I see your simdjson code appears to be C but has a dependency for jsoncpp. There's also a graph which compares several different json libraries except jsoncpp. I've used jsoncpp for some other projects and I'd be curious about your opinion and any comparisons you might have.

glangdale 8y ago
That could be clearer, sorry. We compare against RapidJSON and sajson for the bulk of the paper and for the 'splash' graphics as these are the fastest two libraries that validate to a similar extent (gason is a little faster than RapidJSON but accepts some crazy stuff). In the paper, we do also include measurements against 7 other libraries including jsoncpp. It is an order of magnitude slower than we are on the measurements we took. I am not familiar enough with it to know whether that's meaningful; it may have a lot of other attributes that make it more useful for what you're doing than what we have. We're pretty barebones, especially at this stage.
- glangdale 8y ago
  (the broader comparison is in Table 9 at https://arxiv.org/pdf/1902.08318.pdf https://arxiv.org/pdf/1902.08318.pdf)
  - inetknght 8y ago
    That's pretty interesting, thanks!

gok 8y ago

You're using C intrinsics instead of assembly, right? Are you sure the compiler isn't doing the scheduling well enough on its own?

glangdale 8y ago
If you're think about algorithms in terms of how they might embed into a processor, you need to know the throughput and latency numbers even if the compiler is getting things right. For example, there are some nice possibilities with the TBL instruction (or PSHUFB, or VERMB, etc) for doing character class membership tests. Which version you use would have a lot to do with whether you think TBL is going to issue 2/cycle, 1/cycle or 0.5/cycle (just fr'instance) - you might design the algorithm quite differently. It's not just a case of picking the one design choice and then having the compiler schedule that - if you're not thinking about this during design, you're not going to be remotely near peak performance.
- gok 8y ago
  Sure but the compiler can compile your intrinsics to different-but-equivalent instructions, and modern compilers actually do that pretty aggressively. Are you sure the code output was actually using the instructions you expected?
  - burntsushi 8y ago
    Often times you wind up needing to change the entire algorithm to use different instructions with different latency/throughput numbers. The classic case off the top of my head (for Intel at least) is PCMPESTRI versus completely different approaches for substring search.
  - glangdale 8y ago
    Yes. I look at the asm all the time, having not fallen off the turnip truck yesterday.
Rebelgecko 8y ago
I'm not an expert assembly programmer and I'm definitely not an expert at using SIMD instructions, but the last time I created a NEON implementation of an algorithm I was able to beat GCC's output (from compiling C intrinsics) after an afternoon of tweaking things by hand. From looking at GCC's output it didn't seem like it was doing a great job interleaving ARM and NEON ops to take advantage of that pipeline, although maybe it's gotten better since GCC 5.

snvzz 8y ago

I'm curious about what you think about RISC-V's draft V extension.

dragontamer 8y ago

BTW: I would suggest you look into the closely related art of GPGPU programming. OpenCL, CUDA, or whatever. The architecture of GPUs is SIMD on steroids. You'll likely be very happy with the metrics associated with GPU-shared memory (NVidia) or Local-memory (AMD). NVidia publishes throughput metrics in their PTX page: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#maximize-instruction-throughput https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... AMD's information is harder to find, but it does exist: http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings http://clrx.nativeboinc.org/wiki2/wiki/wiki/GcnTimings -------- I haven't been doing too much SIMD programming myself, but I know the general intrinsics. I think ARM's biggest problem is documentation. Intel's intrinsics page is great: https://software.intel.com/sites/landingpage/IntrinsicsGuide/ https://software.intel.com/sites/landingpage/IntrinsicsGuide... ARM's intrinsics page is... not as good. But I guess still workable. As you've noted, ARM's intrinsics documentation is missing latency / throughput metrics. https://developer.arm.com/technologies/neon/intrinsics https://developer.arm.com/technologies/neon/intrinsics POWER9 by the way has latency/throughput metrics published. I'm most happy with POWER9 as an alternative CPU, too bad its so expensive though. ---------- Intel + Agner Fog is definitely the best documentation for how things work and optimization information. NVidia CUDA and POWER9 seem to be #2 and #3 with documentation. Since GPUs are available for cheap these days (under $300 and they plug into any motherboard), GPUs seem to be the best alternative computer these days to learn. Since they're decently documented, its good to explore IMO.

glangdale 8y ago
I got my start on performance programming doing GPGPU regex (a terrible idea in our context - networked IPS - but that's for another time). This was in 2006, so this meant doing GPGPU using "Cg", including such delights as using textures as arrays and floats for everything. CUDA 0.9 and a engineering sample G80 card were a revelation. I keep meaning to go back and do some GPGPU again. I am very fond of architectures that actually exist. GPU is a very different bet than SIMD but a lot of fun. Incidentally PTX is a fiction - more of an IR than something with true throughput and latency numbers. I don't know if anyone programs GPUs directly but that would be interesting.
- 8y ago

Iwan-Zotow 8y ago

Scalar Vector Extensions?! Used to be Scalable VE

glangdale 8y ago
Fixed, thanks. I type the word 'scalar' so much, apparently, I didn't notice.