3 ms·

Good thing that we have Twitter since Apple doesn't know what they're doing, right? ;-) And it's too bad that apparently the page size can't be changed for lar

Good thing that we have Twitter since Apple doesn't know what they're doing, right? ;-) And it's too bad that apparently the page size can't be changed for larger memory systems to reduce TLB misses (at the expense of fragmentation) which is one traditional solution that doesn't require software changes. 16KB seems like a very small page size on a 64GB system. If we scaled an 8MB VAX with 512B pages up to 64GB it would have a page size of something like 4MB.

jjtheblunt 5y ago
why do you say the page size apparently can't be changed for larger memory setups?
scottlamb 5y ago
It sounds like it might be possible, at least at the hardware level. https://github.com/xmrig/xmrig/issues/2060#issuecomment-770291571 https://github.com/xmrig/xmrig/issues/2060#issuecomment-7702... (bug on some random crypto miner benchmark software, I guess) mentions a 32 MiB page size, although that might be with Linux on M1 rather than macOS on M1. This "32MB TLB bottleneck" is a weird thing to say. The TLB size is typically stated in terms of pages, right? with huge pages (or "superpages", as macOS may call them?), that should be a lot more than 32MB of total memory.
- Sirened 5y ago
  It absolutely can! On ARM you can just enter a block entree in a non-leaf page table to allocate an entire block of physically contiguous memory in a single TLB entry. From the ARM docs [1], any CPU supporting 16K translation granules will support 32MB L2 blocks. I suspect, however, that apple can't really allocate that many (if any at all) 32MB regions of memory at runtime due to fragmentation unless they've substantially changed their contiguous memory allocator since I last looked. [1] https://developer.arm.com/documentation/den0024/a/The-Memory-Management-Unit/Translation-tables-in-ARMv8-A/Effect-of-granule-sizes-on-translation-tables https://developer.arm.com/documentation/den0024/a/The-Memory...
  - scottlamb 5y ago
    Interesting, thanks! > I suspect, however, that apple can't really allocate that many (if any at all) 32MB regions of memory at runtime due to fragmentation unless they've substantially changed their contiguous memory allocator since I last looked. That's unfortunate but at least is something they could fix in upcoming software versions. Some folks have put a lot of effort into huge pages on Linux, and it's still ongoing. [1] Not too surprising that macOS could have some room for improvement... [1] https://lwn.net/Articles/887753/ https://lwn.net/Articles/887753/

jjtheblunt 5y ago

why do you say the page size apparently can't be changed for larger memory setups?

scottlamb 5y ago

It sounds like it might be possible, at least at the hardware level. https://github.com/xmrig/xmrig/issues/2060#issuecomment-770291571 https://github.com/xmrig/xmrig/issues/2060#issuecomment-7702... (bug on some random crypto miner benchmark software, I guess) mentions a 32 MiB page size, although that might be with Linux on M1 rather than macOS on M1. This "32MB TLB bottleneck" is a weird thing to say. The TLB size is typically stated in terms of pages, right? with huge pages (or "superpages", as macOS may call them?), that should be a lot more than 32MB of total memory.

Sirened 5y ago
It absolutely can! On ARM you can just enter a block entree in a non-leaf page table to allocate an entire block of physically contiguous memory in a single TLB entry. From the ARM docs [1], any CPU supporting 16K translation granules will support 32MB L2 blocks. I suspect, however, that apple can't really allocate that many (if any at all) 32MB regions of memory at runtime due to fragmentation unless they've substantially changed their contiguous memory allocator since I last looked. [1] https://developer.arm.com/documentation/den0024/a/The-Memory-Management-Unit/Translation-tables-in-ARMv8-A/Effect-of-granule-sizes-on-translation-tables https://developer.arm.com/documentation/den0024/a/The-Memory...
- scottlamb 5y ago
  Interesting, thanks! > I suspect, however, that apple can't really allocate that many (if any at all) 32MB regions of memory at runtime due to fragmentation unless they've substantially changed their contiguous memory allocator since I last looked. That's unfortunate but at least is something they could fix in upcoming software versions. Some folks have put a lot of effort into huge pages on Linux, and it's still ongoing. [1] Not too surprising that macOS could have some room for improvement... [1] https://lwn.net/Articles/887753/ https://lwn.net/Articles/887753/