4 ms·

You could try to turn on transparent hugepages: echo always >/sys/kernel/mm/transparent_hugepage/defrag echo always >/sys/kernel/mm/transparent_hugepage/enabl

You could try to turn on transparent hugepages: echo always >/sys/kernel/mm/transparent_hugepage/defrag echo always >/sys/kernel/mm/transparent_hugepage/enabled For more details see: http://lwn.net/Articles/423584/ http://lwn.net/Articles/423584/

Tuna-Fish 14y ago
The transparent huge page code still only gets you 2M/4M pages, which will still fault most of the time when you handle multi-gigabyte in-memory structures. To avoid ever faulting, you really need to use the 1G pages. And no-one is crazy enough to build support for making them transparent. :)
- Andys 14y ago
  We are not really asking for them to be transparent - TFA is talking about low-level kernel fiddling to achieve what would take a couple of lines for a custom hugepage size?
  - Tuna-Fish 14y ago
    I was specifically replying to neopallium, who suggested turning on transparent hugepages, which, however, would not really do any good at all in this situation. Using (non-transparent) 1G large pages is the correct approach here.
- neopallium 14y ago
  epistasis called manual management of huge pages a "royal pain in the ass", which is why I suggested trying transparent huge pages since they are easier to work with (i.e. no code change). I would be interested in seeing some benchmarks that shows the affect of using huge pages of different sizes with hash tables or other data structures. Here is a memory bandwidth benchmark that uses huge pages (only uses the default size which is 2Mbyte on my system): http://blogs.utexas.edu/jdm4372/2010/11/11/optimizing-amd-opteron-memory-bandwidth-part-5-single-thread-read-only/ http://blogs.utexas.edu/jdm4372/2010/11/11/optimizing-amd-op... It only sums the values from 32 million doubles (about 256Mbytes of ram). So it is only useful as a memory read speed benchmark. But is does show a lot of different ways to optimize memory bandwidth (prefetch, large pages, sse).
epistasis 14y ago
Thanks, I had not seen that article, but LWN's coverage of huge pages is what made me purchase a subscription, it was absolutely fantastic documentation. It does turn out that to fully remove TLB misses I'd need 1GB pages, which as was pointed out, are rather scary to imagine being paged to disk, as it would take approximately 10s to page it out. As machines with 512GB or 1TB of RAM become more commonplace, I wonder if there could be a mechanism to better assist with their allocation. Right now, one has to use a bunch of manual management and be extremely aware of any other processes that might start consuming memory.

Tuna-Fish 14y ago

The transparent huge page code still only gets you 2M/4M pages, which will still fault most of the time when you handle multi-gigabyte in-memory structures. To avoid ever faulting, you really need to use the 1G pages. And no-one is crazy enough to build support for making them transparent. :)

Andys 14y ago
We are not really asking for them to be transparent - TFA is talking about low-level kernel fiddling to achieve what would take a couple of lines for a custom hugepage size?
- Tuna-Fish 14y ago
  I was specifically replying to neopallium, who suggested turning on transparent hugepages, which, however, would not really do any good at all in this situation. Using (non-transparent) 1G large pages is the correct approach here.
neopallium 14y ago
epistasis called manual management of huge pages a "royal pain in the ass", which is why I suggested trying transparent huge pages since they are easier to work with (i.e. no code change). I would be interested in seeing some benchmarks that shows the affect of using huge pages of different sizes with hash tables or other data structures. Here is a memory bandwidth benchmark that uses huge pages (only uses the default size which is 2Mbyte on my system): http://blogs.utexas.edu/jdm4372/2010/11/11/optimizing-amd-opteron-memory-bandwidth-part-5-single-thread-read-only/ http://blogs.utexas.edu/jdm4372/2010/11/11/optimizing-amd-op... It only sums the values from 32 million doubles (about 256Mbytes of ram). So it is only useful as a memory read speed benchmark. But is does show a lot of different ways to optimize memory bandwidth (prefetch, large pages, sse).

epistasis 14y ago

Thanks, I had not seen that article, but LWN's coverage of huge pages is what made me purchase a subscription, it was absolutely fantastic documentation. It does turn out that to fully remove TLB misses I'd need 1GB pages, which as was pointed out, are rather scary to imagine being paged to disk, as it would take approximately 10s to page it out. As machines with 512GB or 1TB of RAM become more commonplace, I wonder if there could be a mechanism to better assist with their allocation. Right now, one has to use a bunch of manual management and be extremely aware of any other processes that might start consuming memory.