3 ms·
-Ofast also doubles the performance for me to 200tok/sec, and -march=native got me up to 230tok/sec. -Ofast does break some compliance but I seriously doubt it
by sodality2 3y ago
-Ofast also doubles the performance for me to 200tok/sec, and -march=native got me up to 230tok/sec.
-Ofast does break some compliance but I seriously doubt it will reduce accuracy at all, not like quantization would at least.
- SomeRndName11 3y agoone can also try profile guide optimisation and CLANG. GCC an LLVM outperform each other on different code,.