7 ms·

Float-Parsing Benchmark: Regular Visual Studio, ClangCL and Linux GCC

37 points by ibobev 4y ago

Const-me 4y ago
Visual studio C++ compiler is very comparable to the rest of them. I think the OP got screwed by cmake. While it technically supports Windows, the level of support is not great. To make fast binaries with Visual C++, use Visual Studio to create and configure the project. The most important settings are following: • C/C++ / Code Generation / Enable Enhanced Instruction Set • C/C++ / Optimization / Whole Program Optimization • Linker / Optimization / Link Time Code Generation While it’s technically possible to set up the correct VC++ options with CMakeLists.txt, it’s relatively hard: need to detect VC++ compiler, and pass these weird command-line switches. Most cross-platform C++ libraries aren’t doing that.
- blondin 4y ago
  interesting! the builder is making the program slow? would be interesting to see the benchmark result using cl commands directly.
  - tialaramex 4y ago
    Microsoft got where they did by prioritising extreme backward compatibility, hence the SimCity on Windows 95 story - SimCity has a bug, Windows 95 specifically works around the bug so now SimCity works on Windows 95. For ordinary end user software this is unabiguously good news, for a compiler it's more of a mixed bag.
    - pcdevils 4y ago
      The ADK used a lookup db to apply those fixes though; it wasn't a global fudge. Windows compatibility has probably been the main differentiator over the decades and every break (vista driver signing, dropping support for a bunch of scanners/printers, os quality issues) then 10 and even more so 11 making upgrading a mess of seemingly huge hardware replacement costs for next to no user/use gains, has caused uproar. Hell even xp was a mess; windows compatibility mode barely did what it suggested; but the intention was there.
- TinkersW 4y ago
  I told it to generate a VS solution to see what build settings it had applied.. and it is set up very poorly, not surprising performance is ass. This is release settings: intrinsics = off SIMD = off whole program optimizations = off link time code gen = off
- jcranmer 4y ago
  The latter two flags are roughly equivalent to `-flto` on gcc/clang, and it's not a fair comparison to enable it for VC++ but not gcc/clang. (It seems that you can set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON) to make cmake enable lto).
- DannyBee 4y ago
  Comparing lto vs not does make sense. You'd want to compare lto vs lto and not vs not. This is not vs not. It is a fair comparison in that sense
- leni536 4y ago
  AFAIK the fastfloat benchmark is a single translation unit anyway. Whole program optimization and LTO buys you nothing there. GCC and clang are both conservative with instruction set withou march option. I'm often disappointed by MSVC codegen when examining it in godbolt, I'm not surprised by the results here.
benj111 4y ago
Slightly off topic but why did we end up with floats as the standard? Surely some kind of fixed point format would be better for a lot of use cases? I suppose at this point we're kind of locked in but it seems to me you could have instructions on the processor to do maths where a lower register is below the decimal point so an add would be ADD {R1 R2 }{R3 R4} You wouldn't need all new silicon for a FP unit. And these could be used for doing double width math aswell.
- tialaramex 4y ago
  If you want fixed point arithmetic, the integers are already right there, so, you can just use those, today. What are you imagining is better about your idea ?
  - abainbridge 4y ago
    Efficient fixed-point arithmetic requires hardware support. "Some DSP architectures offer native support for specific fixed-point formats, for example signed n-bit numbers with n−1 fraction bits (whose values may range between −1 and almost +1). The support may include a multiply instruction that includes renormalization—the scaling conversion of the product from 2n−2 to n−1 fraction bits.[citation needed] If the CPU does not provide that feature, the programmer must save the product in a large enough register or temporary variable, and code the renormalization explicitly." From: https://en.wikipedia.org/wiki/Fixed-point_arithmetic#Hardware_support https://en.wikipedia.org/wiki/Fixed-point_arithmetic#Hardwar...
    - tialaramex 4y ago
      The most valuable thing here is the renormalization on multiply, but even that's not actually much. Take the Analog Devices Blackfin range, if you want fixed point, which you're welcome to, they want you to use 1.15, so that's a 16-bit two's complement value between -32768/32768 and +32767/32768. Additions and subtractions are the same as integer maths, we're adding fractions but the adder doesn't care. For the multiply, we're doing the same operation, 16-bit x 16-bit = 32-bit, but we can shift away the left bit and then snap off the bottom 16-bits to get another 1.15 fixed point value. If you want this sort of fixed point type, lack of "hardware support" is not what's holding you back, nothing remotely modern would struggle to perform that shift operation on general purpose hardware.
SloopJon 4y ago
Daniel chose three options: Visual C++, ClangCL, and GCC under WSL (2?). He says that ClangCL "couples the LLVM compiler (commonly used by Apple) with the Visual Studio backend." I'm unclear on what he means by the backend. Is ClangCL essentially the same as using LLVM for Windows with the Microsoft linker, or is there a difference in code generation?
- DannyBee 4y ago
  Clang-cl is a driver for clang+llvm made for compatibility with cl.exe. It uses the llvm backend. Unless Daniel is using something special, he is incorrect. It is not the vs backend. Microsoft did have a clang frontend + c2 backend project, but I thought they deprecated it.

37 points by ibobev 4y ago

Visual studio C++ compiler is very comparable to the rest of them. I think the OP got screwed by cmake. While it technically supports Windows, the level of support is not great. To make fast binaries with Visual C++, use Visual Studio to create and configure the project. The most important settings are following: • C/C++ / Code Generation / Enable Enhanced Instruction Set • C/C++ / Optimization / Whole Program Optimization • Linker / Optimization / Link Time Code Generation While it’s technically possible to set up the correct VC++ options with CMakeLists.txt, it’s relatively hard: need to detect VC++ compiler, and pass these weird command-line switches. Most cross-platform C++ libraries aren’t doing that.

blondin 4y ago
interesting! the builder is making the program slow? would be interesting to see the benchmark result using cl commands directly.
- tialaramex 4y ago
  Microsoft got where they did by prioritising extreme backward compatibility, hence the SimCity on Windows 95 story - SimCity has a bug, Windows 95 specifically works around the bug so now SimCity works on Windows 95. For ordinary end user software this is unabiguously good news, for a compiler it's more of a mixed bag.
  - pcdevils 4y ago
    The ADK used a lookup db to apply those fixes though; it wasn't a global fudge. Windows compatibility has probably been the main differentiator over the decades and every break (vista driver signing, dropping support for a bunch of scanners/printers, os quality issues) then 10 and even more so 11 making upgrading a mess of seemingly huge hardware replacement costs for next to no user/use gains, has caused uproar. Hell even xp was a mess; windows compatibility mode barely did what it suggested; but the intention was there.
TinkersW 4y ago
I told it to generate a VS solution to see what build settings it had applied.. and it is set up very poorly, not surprising performance is ass. This is release settings: intrinsics = off SIMD = off whole program optimizations = off link time code gen = off
jcranmer 4y ago
The latter two flags are roughly equivalent to `-flto` on gcc/clang, and it's not a fair comparison to enable it for VC++ but not gcc/clang. (It seems that you can set(CMAKE_INTERPROCEDURAL_OPTIMIZATION ON) to make cmake enable lto).
DannyBee 4y ago
Comparing lto vs not does make sense. You'd want to compare lto vs lto and not vs not. This is not vs not. It is a fair comparison in that sense
leni536 4y ago
AFAIK the fastfloat benchmark is a single translation unit anyway. Whole program optimization and LTO buys you nothing there. GCC and clang are both conservative with instruction set withou march option. I'm often disappointed by MSVC codegen when examining it in godbolt, I'm not surprised by the results here.

benj111 4y ago

Slightly off topic but why did we end up with floats as the standard? Surely some kind of fixed point format would be better for a lot of use cases? I suppose at this point we're kind of locked in but it seems to me you could have instructions on the processor to do maths where a lower register is below the decimal point so an add would be ADD {R1 R2 }{R3 R4} You wouldn't need all new silicon for a FP unit. And these could be used for doing double width math aswell.

tialaramex 4y ago
If you want fixed point arithmetic, the integers are already right there, so, you can just use those, today. What are you imagining is better about your idea ?
- abainbridge 4y ago
  Efficient fixed-point arithmetic requires hardware support. "Some DSP architectures offer native support for specific fixed-point formats, for example signed n-bit numbers with n−1 fraction bits (whose values may range between −1 and almost +1). The support may include a multiply instruction that includes renormalization—the scaling conversion of the product from 2n−2 to n−1 fraction bits.[citation needed] If the CPU does not provide that feature, the programmer must save the product in a large enough register or temporary variable, and code the renormalization explicitly." From: https://en.wikipedia.org/wiki/Fixed-point_arithmetic#Hardware_support https://en.wikipedia.org/wiki/Fixed-point_arithmetic#Hardwar...
  - tialaramex 4y ago
    The most valuable thing here is the renormalization on multiply, but even that's not actually much. Take the Analog Devices Blackfin range, if you want fixed point, which you're welcome to, they want you to use 1.15, so that's a 16-bit two's complement value between -32768/32768 and +32767/32768. Additions and subtractions are the same as integer maths, we're adding fractions but the adder doesn't care. For the multiply, we're doing the same operation, 16-bit x 16-bit = 32-bit, but we can shift away the left bit and then snap off the bottom 16-bits to get another 1.15 fixed point value. If you want this sort of fixed point type, lack of "hardware support" is not what's holding you back, nothing remotely modern would struggle to perform that shift operation on general purpose hardware.

SloopJon 4y ago

Daniel chose three options: Visual C++, ClangCL, and GCC under WSL (2?). He says that ClangCL "couples the LLVM compiler (commonly used by Apple) with the Visual Studio backend." I'm unclear on what he means by the backend. Is ClangCL essentially the same as using LLVM for Windows with the Microsoft linker, or is there a difference in code generation?

DannyBee 4y ago
Clang-cl is a driver for clang+llvm made for compatibility with cl.exe. It uses the llvm backend. Unless Daniel is using something special, he is incorrect. It is not the vs backend. Microsoft did have a clang frontend + c2 backend project, but I thought they deprecated it.