3 ms·

This is largely solved with approaches such as Flatpak or Snap, but graphics drivers are still an issue - they're expected to be supplied by the distribution, a

by mjg59 4y ago

This is largely solved with approaches such as Flatpak or Snap, but graphics drivers are still an issue - they're expected to be supplied by the distribution, and components of them end up in-process in the application even if the rest of the application's runtime is shipped with the application. If there's an incompatibility between the application runtime and assumptions made by that driver code (as there appears to be in this case - TF2 ships its own malloc() implementation, but the graphics driver code ends up using it inconsistently and so blows up) then you're going to have problems. I don't think there's anything about Windows that would fundamentally change things here. Windows apps aren't shipping their own graphics drivers, even if they're bundling everything else.

cesarb 4y ago
> TF2 ships its own malloc() implementation, but the graphics driver code ends up using it inconsistently and so blows up) then you're going to have problems. I don't think there's anything about Windows that would fundamentally change things here. Yes, there is: on Windows, due to the way DLL linking works there, the graphics driver wouldn't use the malloc() implementation from TF2. The flat linking namespace in which you can globally replace the memory allocator for every dynamic library does not exist on Windows; if the graphics driver is linked to the memory allocator from the C library, it will get the memory allocator from that C library, not from some other DLL in the same process. That's not to say Windows is free of dynamic linking problems. While on Linux it's mostly only NSS and the graphics driver (and only when explicitly requested), on Windows it's common for unrelated third party software to inject DLLs and threads all over every process on the system. And it's not uncommon for these injected DLLs to do things like hooking into system DLLs (by overwriting the entry point of exported functions, or even internal functions), leading to hard-to-diagnose crashes when things are not like they expected.
- eklitzke 4y ago
  Linux does not have a flat linking namespace (for example, see RTLD_NEXT in the dlsym man page). If it did, this wouldn't be a problem because everything would use the libc malloc or everything would use the tcmalloc implementation. It's just that glibc exports pretty much everything as a weak symbol, most programs and libraries that link against libc bind to whatever ld.so picks as the first in the search object search order, and messing with linker settings is the last thing most developers want to think about. I just read TFA so I haven't dived into all the details but my wild speculation is that the graphics driver is doing an aligned new and an unaligned delete. The alignment parameters in the new and delete give them different symbol names but you're free to implement them by dispatching to the unaligned version, so this wouldn't necessarily cause a problem (even in C++17 mode) if the malloc implementation doesn't actually handle them differently. However if tcmalloc was built for C++14 then it would only have the unaligned operators, and therefore the aligned new would resolve to the glibc implementation and the unaligned delete would resolve to the tcmalloc implementation, or something like that.
- yrro 4y ago
  Isn't this what protected linkage is for?
- pjmlp 4y ago
  Not only Windows, Aix is also COFF based and has a similar approach to dynamic libraries, including export definitions and import files.
Dalewyn 4y ago
>I don't think there's anything about Windows that would fundamentally change things here. Windows apps aren't shipping their own graphics drivers, even if they're bundling everything else. Over on Windows, GPU drivers are provided and distributed by the manufacturer, Microsoft themselves might also distribute them through Windows Update. GPU manufacturers also work together with Microsoft and bigger game dev studios (read: studios with sufficient cash/influence) to make sure everything works well together. The drivers are also signed off by Microsoft, both figuratively and literally. Linux has none of this. Drivers are provided primarily by volunteers (most of whom couldn't care less about proprietary code), packaged and distributed by each distro, and most game devs couldn't care less about issues concerning less than one percent of their customers.
- pedrocr 4y ago
  Getting consistent quality out of Linux drivers is usually much easier than Windows ones exactly because they're not supplied by a random hardware maker but are almost all upstream in the kernel. GPU drivers used to be a nightmare for this exact reason and are now finally all becoming properly upstream. Meanwhile the horror stories about Windows GPU drivers are all too common. Having to "clean up" old drivers. Having to stay on older versions because the new ones have a relevant broken feature, etc.
  - zamnos 4y ago
    Let's be real, the two contenders for GPU performance (which, gamers appreciate) are Nvidia, which is not in the kernel, and AMD, which has an in-kernel driver, plus their own proprietary driver, same as Nvidia. TF2 is old enough that its runnable even on old Intel hardware, but let's not pretend that "almost all [drivers are] upstream in the kernel", so long as Nvidia and AMD both have proprietary, out-of-kernel-tree drivers, which mean that the "windows GPU horror stories" are also Linux GPU driver horror stories. Having to "clean up" old drivers. Having to stay on older versions because the new ones have a relevant broken feature, or the best: staying on an older version because the newer one doesn't support your hardware. I've spent way too long on Nvidia's legacy unix driver page[1], but only Nvidia cards have Cuda support, or the performance, so I'm stuck there. [1] https://www.nvidia.com/en-us/drivers/unix/ https://www.nvidia.com/en-us/drivers/unix/

by mjg59 4y ago

cesarb 4y ago

> TF2 ships its own malloc() implementation, but the graphics driver code ends up using it inconsistently and so blows up) then you're going to have problems. I don't think there's anything about Windows that would fundamentally change things here. Yes, there is: on Windows, due to the way DLL linking works there, the graphics driver wouldn't use the malloc() implementation from TF2. The flat linking namespace in which you can globally replace the memory allocator for every dynamic library does not exist on Windows; if the graphics driver is linked to the memory allocator from the C library, it will get the memory allocator from that C library, not from some other DLL in the same process. That's not to say Windows is free of dynamic linking problems. While on Linux it's mostly only NSS and the graphics driver (and only when explicitly requested), on Windows it's common for unrelated third party software to inject DLLs and threads all over every process on the system. And it's not uncommon for these injected DLLs to do things like hooking into system DLLs (by overwriting the entry point of exported functions, or even internal functions), leading to hard-to-diagnose crashes when things are not like they expected.

eklitzke 4y ago
Linux does not have a flat linking namespace (for example, see RTLD_NEXT in the dlsym man page). If it did, this wouldn't be a problem because everything would use the libc malloc or everything would use the tcmalloc implementation. It's just that glibc exports pretty much everything as a weak symbol, most programs and libraries that link against libc bind to whatever ld.so picks as the first in the search object search order, and messing with linker settings is the last thing most developers want to think about. I just read TFA so I haven't dived into all the details but my wild speculation is that the graphics driver is doing an aligned new and an unaligned delete. The alignment parameters in the new and delete give them different symbol names but you're free to implement them by dispatching to the unaligned version, so this wouldn't necessarily cause a problem (even in C++17 mode) if the malloc implementation doesn't actually handle them differently. However if tcmalloc was built for C++14 then it would only have the unaligned operators, and therefore the aligned new would resolve to the glibc implementation and the unaligned delete would resolve to the tcmalloc implementation, or something like that.
yrro 4y ago
Isn't this what protected linkage is for?
pjmlp 4y ago
Not only Windows, Aix is also COFF based and has a similar approach to dynamic libraries, including export definitions and import files.

Dalewyn 4y ago

>I don't think there's anything about Windows that would fundamentally change things here. Windows apps aren't shipping their own graphics drivers, even if they're bundling everything else. Over on Windows, GPU drivers are provided and distributed by the manufacturer, Microsoft themselves might also distribute them through Windows Update. GPU manufacturers also work together with Microsoft and bigger game dev studios (read: studios with sufficient cash/influence) to make sure everything works well together. The drivers are also signed off by Microsoft, both figuratively and literally. Linux has none of this. Drivers are provided primarily by volunteers (most of whom couldn't care less about proprietary code), packaged and distributed by each distro, and most game devs couldn't care less about issues concerning less than one percent of their customers.

pedrocr 4y ago
Getting consistent quality out of Linux drivers is usually much easier than Windows ones exactly because they're not supplied by a random hardware maker but are almost all upstream in the kernel. GPU drivers used to be a nightmare for this exact reason and are now finally all becoming properly upstream. Meanwhile the horror stories about Windows GPU drivers are all too common. Having to "clean up" old drivers. Having to stay on older versions because the new ones have a relevant broken feature, etc.
- zamnos 4y ago
  Let's be real, the two contenders for GPU performance (which, gamers appreciate) are Nvidia, which is not in the kernel, and AMD, which has an in-kernel driver, plus their own proprietary driver, same as Nvidia. TF2 is old enough that its runnable even on old Intel hardware, but let's not pretend that "almost all [drivers are] upstream in the kernel", so long as Nvidia and AMD both have proprietary, out-of-kernel-tree drivers, which mean that the "windows GPU horror stories" are also Linux GPU driver horror stories. Having to "clean up" old drivers. Having to stay on older versions because the new ones have a relevant broken feature, or the best: staying on an older version because the newer one doesn't support your hardware. I've spent way too long on Nvidia's legacy unix driver page[1], but only Nvidia cards have Cuda support, or the performance, so I'm stuck there. [1] https://www.nvidia.com/en-us/drivers/unix/ https://www.nvidia.com/en-us/drivers/unix/