4 ms·

I have a dumb compiler question that I should probably know and I’m embarrassed to ask. AArch64 really highlights how CPU features differ greatly. There are in

I have a dumb compiler question that I should probably know and I’m embarrassed to ask. AArch64 really highlights how CPU features differ greatly. There are instructions available on some CPUs that do some amazing things, but are simply not available in very similar models. To wn extent the same problem also exists in x86: for instance, there are a AES functions that are not available on all CPUs. When using an AOT language, how does one take full advantage of all cpu features at runtime? This seems like a place JIT would really outpace AOT.

Vogtinator 4y ago
The easiest option is runtime dispatch, for instance by using compiler support for function clones: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/
- kevincox 4y ago
  Yes. To provide a quick summary, the compiler generates multiple versions of the code, then at runtime does feature detection and picks the best supported implementation. Generally this will only be done for particular functions or blocks that will get a notable benefit from these extra instructions to avoid too much code bloat (unlikely to significantly affect performance but will raise binary size).
saidinesh5 4y ago
Just compile multiple libraries, one per target CPU architecture and load the one you want to use (typically with dlopen/dlsym etc..): Example from whisper.cpp: https://github.com/ggerganov/whisper.cpp/blob/master/examples/whisper.android/app/src/main/java/com/whispercppdemo/whisper/LibWhisper.kt#L112 https://github.com/ggerganov/whisper.cpp/blob/master/example... if (loadVfpv4) { Log.d(LOG_TAG, "Loading libwhisper_vfpv4.so") System.loadLibrary("whisper_vfpv4") } else if (loadV8fp16) { Log.d(LOG_TAG, "Loading libwhisper_v8fp16_va.so") System.loadLibrary("whisper_v8fp16_va") } else { Log.d(LOG_TAG, "Loading libwhisper.so") System.loadLibrary("whisper") } You can also just compile for x86 with all the features enabled, and then choose the functions you want to execute at runtime. This way you don't need to deal with yet another JIT compiler..
deleted 4y ago
[deleted]
dxf 4y ago
>When using an AOT language, how does one take full advantage of all cpu features at runtime? This seems like a place JIT would really outpace AOT. One model is to not try and take advantage of the features at runtime, but to compile per-platform versions of your binary (with whatever flags and optimization levels you want, support for sanitizers, etc.) and deploy those appropriately. E.g. build one binary that supports "generic" x86 variants, one optimized for haswell, etc. This model works well if you have control over how your binaries get deployed (though not so well if you don't), and especially well if you can build the libc the same way at compile time also and statically link it into your binary. (This last point is one reason Google is investing in LLVM's libc.)
- p_l 4y ago
  GCC and ICC both support generation of function variants compiled with different flags - you can have a function compiled for generic, lowest common denominator target, plus variants that depend on various optional features, and it will be dynamically selected at runtime which one is appropriate (iirc it will be done during dynamic linking)

Vogtinator 4y ago

The easiest option is runtime dispatch, for instance by using compiler support for function clones: https://lwn.net/Articles/691932/ https://lwn.net/Articles/691932/

kevincox 4y ago
Yes. To provide a quick summary, the compiler generates multiple versions of the code, then at runtime does feature detection and picks the best supported implementation. Generally this will only be done for particular functions or blocks that will get a notable benefit from these extra instructions to avoid too much code bloat (unlikely to significantly affect performance but will raise binary size).

saidinesh5 4y ago

Just compile multiple libraries, one per target CPU architecture and load the one you want to use (typically with dlopen/dlsym etc..): Example from whisper.cpp: https://github.com/ggerganov/whisper.cpp/blob/master/examples/whisper.android/app/src/main/java/com/whispercppdemo/whisper/LibWhisper.kt#L112 https://github.com/ggerganov/whisper.cpp/blob/master/example... if (loadVfpv4) { Log.d(LOG_TAG, "Loading libwhisper_vfpv4.so") System.loadLibrary("whisper_vfpv4") } else if (loadV8fp16) { Log.d(LOG_TAG, "Loading libwhisper_v8fp16_va.so") System.loadLibrary("whisper_v8fp16_va") } else { Log.d(LOG_TAG, "Loading libwhisper.so") System.loadLibrary("whisper") } You can also just compile for x86 with all the features enabled, and then choose the functions you want to execute at runtime. This way you don't need to deal with yet another JIT compiler..

deleted 4y ago

[deleted]

dxf 4y ago

>When using an AOT language, how does one take full advantage of all cpu features at runtime? This seems like a place JIT would really outpace AOT. One model is to not try and take advantage of the features at runtime, but to compile per-platform versions of your binary (with whatever flags and optimization levels you want, support for sanitizers, etc.) and deploy those appropriately. E.g. build one binary that supports "generic" x86 variants, one optimized for haswell, etc. This model works well if you have control over how your binaries get deployed (though not so well if you don't), and especially well if you can build the libc the same way at compile time also and statically link it into your binary. (This last point is one reason Google is investing in LLVM's libc.)

p_l 4y ago
GCC and ICC both support generation of function variants compiled with different flags - you can have a function compiled for generic, lowest common denominator target, plus variants that depend on various optional features, and it will be dynamically selected at runtime which one is appropriate (iirc it will be done during dynamic linking)