3 ms·
The Mill model is kind of cool, but today, many peripherals (including GPUs and NICs) have the ability to dump bytes straight into L3 cache. This improves late
by pclmulqdq 4y ago
The Mill model is kind of cool, but today, many peripherals (including GPUs and NICs) have the ability to dump bytes straight into L3 cache. This improves latency in a lot of tasks, including the server-side ones that the Mill is designed for. This is possible due to the fact that MMUs are above the L3 cache.
Honestly, I'm happy waiting for 4k pages to die and be replaced by huge pages. Page tables were added to the x86 architecture in 1985, when 1MB of memory was a ton of memory to have. Having 256 pages worth of memory in your computer was weird and exotic. Fast forward to today, and the average user has several GB of memory - mainstream computers can be expanded to over 128 GB today - and we still mainly use 4k pages. That is the problem here. If we could swap to 2M pages in most applications, we would be able to reduce page table sizes by a factor of 512, and they would still be a lot larger than page tables when virtual memory was invented. And we wouldn't waste much memory!
But no, 4k pages for backwards compatibility. 4k pages forever. And while we're at it, let's add features to Linux (like TCP zero copy) that rely on having 4k pages.
- rsaxvc 4y agoOnce on an embedded system, we found 30% of our processing time was spent handling TLB misses. We also didn't have access to the code that set up the page tables. Somehow I got the linker to emit a function hook which I used to defrag the page tables after they were written but before the MMU was enabled. I remember thinking, "This is not what I learned about in school."
- pclmulqdq 4y agoIn a server system I worked on recently, 10% of its time was spent on page table walks of a multi-GB structure that it allocates flat. That was nuts to me. It could have been 0.
- StillBored 4y agoI can probably guess the OS you were running, but lets just say, some OS's put a lot more effort into assure that virtually continuous mappings tend to also be physically continuous because it then becomes possible to map larger blocks of ram continuously. AKA, both X86 and ARM (as do various other processors) have intermediate bits in their page directory structures that flag 'instead of using another level of PTE/etc below this level assume we are just pointing at an equal size physical range" which on x86-64 can be 2M or 1G. Given the latest Intel x86's have a multilevel TLB, with 8 1G L1 entries, and 1K of L2 TLB entries it should be fairly straightforward to fix with appropriate huge page tweaks unless your data structure exceeds a TB. And even if it does, keeping everything in 1G pages means that the data caches should be able to keep the top level global page directory/etc cached avoiding main ram hits, and resulting in fairly quick TLB refills.
- Symmetry 4y agoThe obvious thing to do is put the address translation between the CPU cache and system level cache.