3 ms·

> Ideally every application that is unlikely to use more than 2GB of memory should use the X32 ABI. It's often useful to be able to address in 64bits even if y

> Ideally every application that is unlikely to use more than 2GB of memory should use the X32 ABI. It's often useful to be able to address in 64bits even if you are never going to use 2GB of memory. And it's probably more and more true the more you care about performance (e.g., mmap() and friends). So I don't think the set of useful workloads that get a performance boost from smaller pointers and don't get hindered by a small address space is as large as you are assuming.

geocar 8y ago
> So I don't think the set of useful workloads that get a performance boost... https://link.springer.com/chapter/10.1007%2F11688839_14?LI=true https://link.springer.com/chapter/10.1007%2F11688839_14?LI=t... found speedups of 13.4% for integer workloads. That's significant for my space, hence my interest. Using a single 1G page per process means switching is faster, and might have other benefits in a post-spectre world...
- pedrocr 8y ago
  A 13% speedup in the critical sections seems much too low to pay for the loss in caching and other optimizations from the 64bit address space. But I'm considering general purpose applications where the critical sections are not run 100%. If all you do is simulation work in clusters for example, I could see how this would be significant. I was replying to saying that most apps should use x32 if they don't use much memory. That sounds like a nightmare in a general purpose distribution, with having every library duplicated and interoperability issues.
  - geocar 8y ago
    What do you mean by "loss in caching and other optimizations"? I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits.
    - noselasd 8y ago
      You'll need 2 or 3 sets of all libraries on a system. One for x86_64, one for x32. And if you want to support traditional 32 bit application you'll need the i686 variant as well. None of these variants are ABI compatible with eachother. If nothing else, that's a burden on logistics.
    - pedrocr 8y ago
      > What do you mean by "loss in caching and other optimizations"? The most common use case is to just mmap() a bunch of cache files and just fully outsource the cache hierarchy of those to the Linux VM subsystem which will do a much better job than almost any app even can (because it does global optimization across all processes). > I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits. This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those. A general purpose distro will never take on that much complexity for a 13% speedup. If they wanted even a fraction of that complexity they'd compile x86_64 at several levels of supported instructions for bigger gains from just tweaking a few compiler flags.
      - geocar 8y ago
        > The most common use case is to just mmap() a bunch of cache files and just fully outsource the cache hierarchy of those to the Linux VM subsystem which will do a much better job than you ever will. I don't understand what you're saying here. Why do you think having the pointers be 64-bit instead of 32-bit help if the program isn't using more than 2G? Can you give me a specific example? > This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those. mmap() and brk() are the two big ones, and almost everything else derives from them. The tricky part is to get MAP_32BIT into every call to mmap(). That means hacking up the dynamic linker, or having a personality flag that just pins it kernel-side. I don't think either is onerous though (probably a few hours work).
  - AstralStorm 8y ago
    If your application really wants to random access over 2 GB of files you might have a point. Most applications do not, except database servers. Cache works with all your memory and is handled with 64 bit offsets in kernel, the limitation is in mmap, but you can mmap parts of a big file no problem with large file support in libc.
    - pedrocr 8y ago
      The use case I'm thinking of is not random access over large files. It's using lots of very small files mmap()'ed so that you get the Linux VM helping you run an efficient cache. I've run across this with thumbnail caches in image processing software. It's easy for these kinds of things to be present in generally used libraries that your standard app will be using just because it has a GTK file selector for example. > you can mmap parts of a big file no problem with large file support in libc. For the speedups discussed here I definitely don't want to spend any time writing special code just to be able to read a 3GB file. That ship has long sailed and good riddance to all those subtle issues where things seem to work fine until they don't and it's not clear why.
    - CJefferson 8y ago
      Actually, you can get a lot less than that once libraries are loaded and such like. I've found on some 32 bit linuxes you can't mmap a single file bigger than around 500mb, which means I couldn't check out a git repository containing 650mb cd images on a 32 bit Linux.
      - geocar 8y ago
        You might check resource limits and/or disable address randomisation (or other "security" features) since they can really eat up your (limited) address space. Don't forget to not put the bugs in, though!
  - pantalaimon 8y ago
    You already have every lib duplicated if you want to run 32bit x86 applications like steam. Distributions handle it just fine. And as the kernel is responsible for caching (or what kind of caching do you mean?) You will not lose that.

geocar 8y ago

> So I don't think the set of useful workloads that get a performance boost... https://link.springer.com/chapter/10.1007%2F11688839_14?LI=true https://link.springer.com/chapter/10.1007%2F11688839_14?LI=t... found speedups of 13.4% for integer workloads. That's significant for my space, hence my interest. Using a single 1G page per process means switching is faster, and might have other benefits in a post-spectre world...

pedrocr 8y ago
A 13% speedup in the critical sections seems much too low to pay for the loss in caching and other optimizations from the 64bit address space. But I'm considering general purpose applications where the critical sections are not run 100%. If all you do is simulation work in clusters for example, I could see how this would be significant. I was replying to saying that most apps should use x32 if they don't use much memory. That sounds like a nightmare in a general purpose distribution, with having every library duplicated and interoperability issues.
- geocar 8y ago
  What do you mean by "loss in caching and other optimizations"? I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits.
  - noselasd 8y ago
    You'll need 2 or 3 sets of all libraries on a system. One for x86_64, one for x32. And if you want to support traditional 32 bit application you'll need the i686 variant as well. None of these variants are ABI compatible with eachother. If nothing else, that's a burden on logistics.
  - pedrocr 8y ago
    > What do you mean by "loss in caching and other optimizations"? The most common use case is to just mmap() a bunch of cache files and just fully outsource the cache hierarchy of those to the Linux VM subsystem which will do a much better job than almost any app even can (because it does global optimization across all processes). > I get the part about duplicated libraries, but I'm not certain it's that big of a deal: Most people don't have more than 2G of code so it should be possible for the 64-bit libraries to be used (and just mapped low) with an m32-built application, if the things that create new pointers can be limited to 32-bits. This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those. A general purpose distro will never take on that much complexity for a 13% speedup. If they wanted even a fraction of that complexity they'd compile x86_64 at several levels of supported instructions for bigger gains from just tweaking a few compiler flags.
    - geocar 8y ago
      > The most common use case is to just mmap() a bunch of cache files and just fully outsource the cache hierarchy of those to the Linux VM subsystem which will do a much better job than you ever will. I don't understand what you're saying here. Why do you think having the pointers be 64-bit instead of 32-bit help if the program isn't using more than 2G? Can you give me a specific example? > This is way too much complexity. "Things that create new pointers" are any functions that return new memory and there are too many of those. mmap() and brk() are the two big ones, and almost everything else derives from them. The tricky part is to get MAP_32BIT into every call to mmap(). That means hacking up the dynamic linker, or having a personality flag that just pins it kernel-side. I don't think either is onerous though (probably a few hours work).
- AstralStorm 8y ago
  If your application really wants to random access over 2 GB of files you might have a point. Most applications do not, except database servers. Cache works with all your memory and is handled with 64 bit offsets in kernel, the limitation is in mmap, but you can mmap parts of a big file no problem with large file support in libc.
  - pedrocr 8y ago
    The use case I'm thinking of is not random access over large files. It's using lots of very small files mmap()'ed so that you get the Linux VM helping you run an efficient cache. I've run across this with thumbnail caches in image processing software. It's easy for these kinds of things to be present in generally used libraries that your standard app will be using just because it has a GTK file selector for example. > you can mmap parts of a big file no problem with large file support in libc. For the speedups discussed here I definitely don't want to spend any time writing special code just to be able to read a 3GB file. That ship has long sailed and good riddance to all those subtle issues where things seem to work fine until they don't and it's not clear why.
  - CJefferson 8y ago
    Actually, you can get a lot less than that once libraries are loaded and such like. I've found on some 32 bit linuxes you can't mmap a single file bigger than around 500mb, which means I couldn't check out a git repository containing 650mb cd images on a 32 bit Linux.
    - geocar 8y ago
      You might check resource limits and/or disable address randomisation (or other "security" features) since they can really eat up your (limited) address space. Don't forget to not put the bugs in, though!
- pantalaimon 8y ago
  You already have every lib duplicated if you want to run 32bit x86 applications like steam. Distributions handle it just fine. And as the kernel is responsible for caching (or what kind of caching do you mean?) You will not lose that.