4 ms·

LLVM's Libc Gets Much Faster memcpy For RISC-V

44 points by kristianp 3y ago

saagarjha 3y ago
Actual patch: https://reviews.llvm.org/rGf4a35492504d https://reviews.llvm.org/rGf4a35492504d. It seems like this changes memcpy to use aligned accesses because some processors may implement other loads very slowly?
- moonchild 3y ago
  What I've heard is that riscv architecturally guarantees that unaligned accesses are legal, but many processors don't implement them natively, so they are trapped and emulated. This seems like a serious flaw—if the architecture designers did not believe the hardware designers would be able to commit to supporting fast unaligned accesses, they should not have mandated that unaligned accesses be available everywhere.
  - saagarjha 3y ago
    Yeah I’ve generally wondered how this will work on that ecosystem. On other platforms there are different rules that range from “unaligned accesses will trap” to “don’t bother they’re too slow” and “they’re ok actually” but I think there are ways to identify the last case that people typically use. What’s the equivalent for RISC-V? Will there be one?
    - tux3 3y ago
      I suspect small microcontrollers that optimize for cost will have terribly slow emulated unaligned accesses, but in the other direction if you're going through the trouble of writing a big OoO application core with vectors, virtualization, and the whole shebang, teaching your memory pipeline to make unaligned accesses fast is well within reason. Hardware designers have added much more complex features for low single digit percent gains.
      - moonchild 3y ago
        Then make it an optional feature software has to check for. God knows riscv is fragmented enough already...
        zamadatix 3y ago
        On one hand, it seems fine to allow the CPU to run anything for the family without requiring the compiler make up to 2^n code paths of each routine that uses this kind of "sometimes implemented fast" feature in order for the binary to be portable across the family. On the other hand, it seems useful to allow the compiler to choose when such optimizations might be worthwhile. Maybe, rather than require software to always check, it'd be good enough to let there be a CPU flag which hints at the implementation the compiler can optionally look at if it thinks it'd be worthwhile.
wyldfire 3y ago
My only beef with LLVM's libc is that it does not permit specialization of things (like memcpy) with assembly implementations. Of course -- I get the goal, we want to be able to use sanitizers and the like to validate the implementation. I wish there were a way to have it both ways, somehow.
- chmod600 3y ago
  Can you please explain the trade off in more detail?
  - wyldfire 3y ago
    Sure. Some of the C library functions are red-hot code that is a bottleneck for many programs (memcpy is a striking example, but sometimes strcpy variants are too). So popular C libraries like musl, glibc, bionic, etc - they sometimes create a dedicated assembly implementation of these functions that get called a lot. This requires more care than a C implementation would (or C++ in LLVM's case). It also requires the additional burden of maintaining sometimes both a C implementation and assembly for each supported architecture. AFAIK the LLVM libc maintainers recognize the value of the assembly implementation and specify as an anti-goal because it conflicts with their primary goal of having a portable C implementation that can be instrumented by the compiler. The compiler instrumentation can help detect defects like buffer overruns -- even in production, in some cases. Note that the intent here is to detect defects in the C library itself, so maybe enabling it in production (w/HWASan, e.g.) is not a use case they tend to focus on. But Address Sanitizer is a feature for "C-style" languages (that term I am using covers many languages). But it cannot be used for low-level languages like assembly. So assembly was ruled out by design to preserve this capability.
- rurban 3y ago
  Of course assembler spezializations are an anti-pattern, because the optimizer should be fixed to do it much better. Better C code is often 2x faster than hand optimized assembler. Eg my C memcpy with inlined and vectorized clang beats glibc or gcc memcpy in assembler easily. https://github.com/rurban/safeclib/blob/master/tests/perf_memcpy_s.c https://github.com/rurban/safeclib/blob/master/tests/perf_me...

44 points by kristianp 3y ago

Actual patch: https://reviews.llvm.org/rGf4a35492504d https://reviews.llvm.org/rGf4a35492504d. It seems like this changes memcpy to use aligned accesses because some processors may implement other loads very slowly?

moonchild 3y ago
What I've heard is that riscv architecturally guarantees that unaligned accesses are legal, but many processors don't implement them natively, so they are trapped and emulated. This seems like a serious flaw—if the architecture designers did not believe the hardware designers would be able to commit to supporting fast unaligned accesses, they should not have mandated that unaligned accesses be available everywhere.
- saagarjha 3y ago
  Yeah I’ve generally wondered how this will work on that ecosystem. On other platforms there are different rules that range from “unaligned accesses will trap” to “don’t bother they’re too slow” and “they’re ok actually” but I think there are ways to identify the last case that people typically use. What’s the equivalent for RISC-V? Will there be one?
  - tux3 3y ago
    I suspect small microcontrollers that optimize for cost will have terribly slow emulated unaligned accesses, but in the other direction if you're going through the trouble of writing a big OoO application core with vectors, virtualization, and the whole shebang, teaching your memory pipeline to make unaligned accesses fast is well within reason. Hardware designers have added much more complex features for low single digit percent gains.
    - moonchild 3y ago
      Then make it an optional feature software has to check for. God knows riscv is fragmented enough already...
      - zamadatix 3y ago
        On one hand, it seems fine to allow the CPU to run anything for the family without requiring the compiler make up to 2^n code paths of each routine that uses this kind of "sometimes implemented fast" feature in order for the binary to be portable across the family. On the other hand, it seems useful to allow the compiler to choose when such optimizations might be worthwhile. Maybe, rather than require software to always check, it'd be good enough to let there be a CPU flag which hints at the implementation the compiler can optionally look at if it thinks it'd be worthwhile.

wyldfire 3y ago

My only beef with LLVM's libc is that it does not permit specialization of things (like memcpy) with assembly implementations. Of course -- I get the goal, we want to be able to use sanitizers and the like to validate the implementation. I wish there were a way to have it both ways, somehow.

chmod600 3y ago
Can you please explain the trade off in more detail?
- wyldfire 3y ago
  Sure. Some of the C library functions are red-hot code that is a bottleneck for many programs (memcpy is a striking example, but sometimes strcpy variants are too). So popular C libraries like musl, glibc, bionic, etc - they sometimes create a dedicated assembly implementation of these functions that get called a lot. This requires more care than a C implementation would (or C++ in LLVM's case). It also requires the additional burden of maintaining sometimes both a C implementation and assembly for each supported architecture. AFAIK the LLVM libc maintainers recognize the value of the assembly implementation and specify as an anti-goal because it conflicts with their primary goal of having a portable C implementation that can be instrumented by the compiler. The compiler instrumentation can help detect defects like buffer overruns -- even in production, in some cases. Note that the intent here is to detect defects in the C library itself, so maybe enabling it in production (w/HWASan, e.g.) is not a use case they tend to focus on. But Address Sanitizer is a feature for "C-style" languages (that term I am using covers many languages). But it cannot be used for low-level languages like assembly. So assembly was ruled out by design to preserve this capability.
rurban 3y ago
Of course assembler spezializations are an anti-pattern, because the optimizer should be fixed to do it much better. Better C code is often 2x faster than hand optimized assembler. Eg my C memcpy with inlined and vectorized clang beats glibc or gcc memcpy in assembler easily. https://github.com/rurban/safeclib/blob/master/tests/perf_memcpy_s.c https://github.com/rurban/safeclib/blob/master/tests/perf_me...