3 ms·

It's mind warping terminology to call them huge pages. Memory sizes have increased 2000-4000x since x86 Linux picked 4 kB, so 2 MB pages are still in relative t

It's mind warping terminology to call them huge pages. Memory sizes have increased 2000-4000x since x86 Linux picked 4 kB, so 2 MB pages are still in relative terms smaller than 4 kB pages were back then and it should be a no-brainer to use them by default.

_a_a_a_ 4y ago
Were there no downside it would be a default. Perhaps ask "isn't it a no-brainer to use large pages" rather than declare it is without saying why you think so. Edit: I did a quick search. upside of less TLB thrashing, downside of potentially more disk thrashing as pages are paged in and out.
tux3 4y ago
Memory sizes have increased, but so have the workloads that people run. It's not like our computers still only consume a few MBs of RAM and we have 1000x times more free memory than we used to. Going from 4k to 2M doesn't waste a fixed amount of memory, it wastes a percentage of memory. So the amount of memory lost continues to scale with memory size. Losing 10% of your memory when you only have 32MB hurts just as much as losing 10% when you have 32GB, because we didn't have a 1000x times increase in free RAM. Applications just use more RAM than they used to. Now an increase to 8k, 16k, or even 64k might be defensible. And even that last one will draw some scowls from very serious people. But going straight to 2M pages by default will not be a good tradeoff. Especially since the current system with differently sized pages gets you most of the performance gain for a much more reasonable cost.
- fulafel 4y ago
  The percentage of memory idea isn't quite right. Consider what would happen in 2 computers, one with 4MB of ram and one with 4 GB of ram. The former would waste a large percentage from moving from 4k to 2MB pages than the latter one.
  - tux3 4y ago
    That's true, although the effect here is because the page size you picked (2MB) is close to the physical memory size (4MB). Between a computer with 1GB and a computer with 64GB, any vaguely reasonable default page size you pick (whether that's 64k or 2M) is far from the total memory size, so the percentage wasted doesn't get much smaller as RAM continues increasing. I.e. the relative benefit of larger pages grows much slower than total system memory. So in other words, my point is that default page size should not just grow linearly with memory size, the best tradeoff is something a little more complex. Saying that RAM grew 1000x, therefore page size should grow a lot is aiming too high too fast. Especially given that hardware already supports mixed page sizes (despite lackluster software support at the moment)
zajio1am 4y ago
Note that more memory does not necessary mean that everything is uniformly bigger, as there is some strongly-non-uniform distribution of memory map sizes, and more memory most likely means that large maps goes larger. Take for example bash: its RSS is about ~4 MB, but it has 43 memory maps (on my system). One is > 1MB, two are 512-1024 kB, most are very small. Each requires at least one page, so with mandatory 2 MB pages, its RSS would be 86 MB instead of 4 MB.

_a_a_a_ 4y ago

Were there no downside it would be a default. Perhaps ask "isn't it a no-brainer to use large pages" rather than declare it is without saying why you think so. Edit: I did a quick search. upside of less TLB thrashing, downside of potentially more disk thrashing as pages are paged in and out.

tux3 4y ago

Memory sizes have increased, but so have the workloads that people run. It's not like our computers still only consume a few MBs of RAM and we have 1000x times more free memory than we used to. Going from 4k to 2M doesn't waste a fixed amount of memory, it wastes a percentage of memory. So the amount of memory lost continues to scale with memory size. Losing 10% of your memory when you only have 32MB hurts just as much as losing 10% when you have 32GB, because we didn't have a 1000x times increase in free RAM. Applications just use more RAM than they used to. Now an increase to 8k, 16k, or even 64k might be defensible. And even that last one will draw some scowls from very serious people. But going straight to 2M pages by default will not be a good tradeoff. Especially since the current system with differently sized pages gets you most of the performance gain for a much more reasonable cost.

fulafel 4y ago
The percentage of memory idea isn't quite right. Consider what would happen in 2 computers, one with 4MB of ram and one with 4 GB of ram. The former would waste a large percentage from moving from 4k to 2MB pages than the latter one.
- tux3 4y ago
  That's true, although the effect here is because the page size you picked (2MB) is close to the physical memory size (4MB). Between a computer with 1GB and a computer with 64GB, any vaguely reasonable default page size you pick (whether that's 64k or 2M) is far from the total memory size, so the percentage wasted doesn't get much smaller as RAM continues increasing. I.e. the relative benefit of larger pages grows much slower than total system memory. So in other words, my point is that default page size should not just grow linearly with memory size, the best tradeoff is something a little more complex. Saying that RAM grew 1000x, therefore page size should grow a lot is aiming too high too fast. Especially given that hardware already supports mixed page sizes (despite lackluster software support at the moment)

zajio1am 4y ago

Note that more memory does not necessary mean that everything is uniformly bigger, as there is some strongly-non-uniform distribution of memory map sizes, and more memory most likely means that large maps goes larger. Take for example bash: its RSS is about ~4 MB, but it has 43 memory maps (on my system). One is > 1MB, two are 512-1024 kB, most are very small. Each requires at least one page, so with mandatory 2 MB pages, its RSS would be 86 MB instead of 4 MB.