5 ms·

Eh, even Clang does this. Install it on Windows and you'll get clang-cl.exe, clang++.exe, clang-cpp.exe, and clang.exe, all 90 MB executables, and almost entire

Eh, even Clang does this. Install it on Windows and you'll get clang-cl.exe, clang++.exe, clang-cpp.exe, and clang.exe, all 90 MB executables, and almost entirely the same executables, but not bit-for-bit identical, so you can't hardlink them. I actually hate this too, but my point is it doesn't necessarily say much about software quality.

rgovostes 5y ago
This makes slightly more sense if it's the case that they are statically linking the Clang and LLVM libraries into each tool. Each one has a slightly different main() (e.g., one parses gcc-style args) but ends up using the same library functionality.
- vchuravy 5y ago
  They are the same binary and on on Linux they are just soft-links.
  - dataflow 5y ago
    Interestingly they're also the same binary on Windows in MSYS2. But not if you install through whatever installer Visual Studio uses (probably the native one).
  - deleted 5y ago
    [deleted]
jfrunyon 5y ago
But these are bit-for-bit identical.
- dataflow 5y ago
  Edit: You're talking about the Backblaze binaries? Yeah that's why I said Clang's is almost bit-for-bit identical, so it's clear to everyone it's not 100% the same as the Backblaze situation. Not sure what your complaint is about my comment though? It's still practically the same problem, and you can solve it just as well with the argv[0] tricks or whatever. Or are you saying your heuristic for determining the underlying engineering quality changes its output based on 1 bit flip? Identical executables is awful engineering, but a few bytes different (out of 90MB) is great engineering?
  - somehnguy 5y ago
    The 'these' being referenced here are the Backblaze executables, not your clang executables.
  - monsieurbanana 5y ago
    They're not saying the clang are bit by bit identical, but the blackblaze ones.
  - jfrunyon 5y ago
    Not sure why you think clang having multiple not-identical executables is relevant to Backblaze having multiple identical executables.
    - dataflow 5y ago
      > Not sure why you think clang having multiple not-identical executables is relevant to Backblaze having multiple identical executables. Really? You genuinely don't see why I would think a case with >99.99% identical executables might be relevant to a case with 100% identical executables?
      - 300bps 5y ago
        I genuinely don’t see why either. With 100% identical executables, there does not appear to be a good reason to have multiple copies under any circumstance. With anything < 100% identical, well, maybe there’s a good reason to have multiples. Who knows? Id probably give someone the benefit of the doubt and figure there was some engineering challenge that made it faster/easier to do it that way. So yes, 100% identical is completely different than almost 100% identical.
        dataflow 5y ago
        > So yes, 100% identical is completely different than almost 100% identical. They're completely (!) different? And you're saying this despite the fact that the comment I replied to was discussing cases where one could "just execute one binary n times (with different argv[0] if they'd like)"... which is something you can do with different executables just as well as with identical ones? It's not just a little different but completely different? So different that not only you don't see any similarity, but you also cannot fathom why I might think there's some similarity?!
- awinter-py 5y ago
  maybe they're not identical but backblaze has discovered 20 hash collisions that are also valid executables? that would be impressive in a way
  - selcuka 5y ago
    Bit-for-bit identical means that they are actually identical though.
mshockwave 5y ago
This only happens on Windows, on *nix systems these are all pointing to `clang-<version>`
meinersbur 5y ago
They are identical copies of each other. Since symlink support is not universal on Windows (Possible since around the time WSL was introduced, but still requires either special privileges or developer mode enabled), many tools including GNU autotools fall back to copying. Since recently (https://reviews.llvm.org/D99170 https://reviews.llvm.org/D99170), LLVM et least tries to create a symlink on Windows, but still caused problems because this is kind of unexpected by Windows. The NSIS installer used by the official Windows installer (https://llvm.org/builds/ https://llvm.org/builds/) does not support symlinks at all such that the installer includes 4 identical copies of Clang. A clean solution would be to have stub executables that link to a central libClang.dll/libLLVM.dll (-DCLANG_LINK_CLANG_DYLIB=ON), but this is not supported under Windows because a DLL can export at most 2^16 symbols. Some work would need to be invested to make process launching on Windows work differently than on other platforms, but then disk space is not that much of a problem.
- dataflow 5y ago
  The Clangs are different from each other in some installations, and identical in others. In particular the Visual Studio installation doesn't have identical binaries, but the MSYS2 installation does. You'll have to check yours. For example, if you dir "%ProgramFiles%\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\bin\clang*.exe" you'll see they're slightly different sizes. (Also note that in this case you don't need symlinks per se, just identical binaries with hardlinks. Each executable could inspect argv[0] and figure out what it's invoked as, and behave accordingly.) What I don't understand is why can't they just bundle everything into a single DLL, then make whatever stub executables they want just call the exported main() in that DLL. I think that might be what MSYS2's copy already does, though I haven't checked. That DLL wouldn't need to export anything else, just the main() function that each .exe stub would forward to. And it can handle everything else internally if it so desires.
  - cma 5y ago
    Different sizes doesn't always mean different sizes (I'm not sure if that is completely true using 'dir'): https://superuser.com/questions/1353064/why-does-size-on-disk-vary-when-size-does-not-with-the-same-set-of-files https://superuser.com/questions/1353064/why-does-size-on-dis...

rgovostes 5y ago

This makes slightly more sense if it's the case that they are statically linking the Clang and LLVM libraries into each tool. Each one has a slightly different main() (e.g., one parses gcc-style args) but ends up using the same library functionality.

vchuravy 5y ago
They are the same binary and on on Linux they are just soft-links.
- dataflow 5y ago
  Interestingly they're also the same binary on Windows in MSYS2. But not if you install through whatever installer Visual Studio uses (probably the native one).
- deleted 5y ago
  [deleted]

jfrunyon 5y ago

But these are bit-for-bit identical.

dataflow 5y ago
Edit: You're talking about the Backblaze binaries? Yeah that's why I said Clang's is almost bit-for-bit identical, so it's clear to everyone it's not 100% the same as the Backblaze situation. Not sure what your complaint is about my comment though? It's still practically the same problem, and you can solve it just as well with the argv[0] tricks or whatever. Or are you saying your heuristic for determining the underlying engineering quality changes its output based on 1 bit flip? Identical executables is awful engineering, but a few bytes different (out of 90MB) is great engineering?
- somehnguy 5y ago
  The 'these' being referenced here are the Backblaze executables, not your clang executables.
- monsieurbanana 5y ago
  They're not saying the clang are bit by bit identical, but the blackblaze ones.
- jfrunyon 5y ago
  Not sure why you think clang having multiple not-identical executables is relevant to Backblaze having multiple identical executables.
  - dataflow 5y ago
    > Not sure why you think clang having multiple not-identical executables is relevant to Backblaze having multiple identical executables. Really? You genuinely don't see why I would think a case with >99.99% identical executables might be relevant to a case with 100% identical executables?
    - 300bps 5y ago
      I genuinely don’t see why either. With 100% identical executables, there does not appear to be a good reason to have multiple copies under any circumstance. With anything < 100% identical, well, maybe there’s a good reason to have multiples. Who knows? Id probably give someone the benefit of the doubt and figure there was some engineering challenge that made it faster/easier to do it that way. So yes, 100% identical is completely different than almost 100% identical.
      - dataflow 5y ago
        > So yes, 100% identical is completely different than almost 100% identical. They're completely (!) different? And you're saying this despite the fact that the comment I replied to was discussing cases where one could "just execute one binary n times (with different argv[0] if they'd like)"... which is something you can do with different executables just as well as with identical ones? It's not just a little different but completely different? So different that not only you don't see any similarity, but you also cannot fathom why I might think there's some similarity?!
awinter-py 5y ago
maybe they're not identical but backblaze has discovered 20 hash collisions that are also valid executables? that would be impressive in a way
- selcuka 5y ago
  Bit-for-bit identical means that they are actually identical though.

mshockwave 5y ago

This only happens on Windows, on *nix systems these are all pointing to `clang-<version>`

meinersbur 5y ago

They are identical copies of each other. Since symlink support is not universal on Windows (Possible since around the time WSL was introduced, but still requires either special privileges or developer mode enabled), many tools including GNU autotools fall back to copying. Since recently (https://reviews.llvm.org/D99170 https://reviews.llvm.org/D99170), LLVM et least tries to create a symlink on Windows, but still caused problems because this is kind of unexpected by Windows. The NSIS installer used by the official Windows installer (https://llvm.org/builds/ https://llvm.org/builds/) does not support symlinks at all such that the installer includes 4 identical copies of Clang. A clean solution would be to have stub executables that link to a central libClang.dll/libLLVM.dll (-DCLANG_LINK_CLANG_DYLIB=ON), but this is not supported under Windows because a DLL can export at most 2^16 symbols. Some work would need to be invested to make process launching on Windows work differently than on other platforms, but then disk space is not that much of a problem.

dataflow 5y ago
The Clangs are different from each other in some installations, and identical in others. In particular the Visual Studio installation doesn't have identical binaries, but the MSYS2 installation does. You'll have to check yours. For example, if you dir "%ProgramFiles%\Microsoft Visual Studio\2022\Community\VC\Tools\Llvm\bin\clang*.exe" you'll see they're slightly different sizes. (Also note that in this case you don't need symlinks per se, just identical binaries with hardlinks. Each executable could inspect argv[0] and figure out what it's invoked as, and behave accordingly.) What I don't understand is why can't they just bundle everything into a single DLL, then make whatever stub executables they want just call the exported main() in that DLL. I think that might be what MSYS2's copy already does, though I haven't checked. That DLL wouldn't need to export anything else, just the main() function that each .exe stub would forward to. And it can handle everything else internally if it so desires.
- cma 5y ago
  Different sizes doesn't always mean different sizes (I'm not sure if that is completely true using 'dir'): https://superuser.com/questions/1353064/why-does-size-on-disk-vary-when-size-does-not-with-the-same-set-of-files https://superuser.com/questions/1353064/why-does-size-on-dis...