3 ms·
Okay, lets say page-faults are off the table for some reason. Lets think about what can happen even if everything is in RAM still. * MMAP is still on the table
by dragontamer 6y ago
Okay, lets say page-faults are off the table for some reason. Lets think about what can happen even if everything is in RAM still.
* MMAP is still on the table: different processes can share RAM at different addresses. (Process#1 thinks the data is at memory location 0x90000000, Process#2 thinks the data is at 0x70000000, but in both cases, the data is at physical location 0x42).
* Physical location 0x42 is a far-read on a far-away NUMA node. Which means the CPU#0 now needs to send a message to a CPU#1 very far away to get a copy of that RAM. This message traverses Intel Ultrapath Interconnect or AMD Infinity fabric (proprietary details), but its a remote message that happens nonetheless.
* Turns out CPU#1 has modified location 0x42. Now CPU#1 must push the most recent copy out of L1 cache, into L2 cache... then into L3 cache, and then send it back to CPU#0. CPU#0 has to wait until this process is done. If CPU#1 wants to modify the data again (or even read it), it may require messages from CPU#0 (who is now the owner of the data, according to simple MESI models).
Modern computers work very hard to hold the illusion of a singular memory space. Eventually, these details are turned into a consistent memory model and become well-ordered sequential operations. The CPU is a good place for that.
---------------
That's how stuff works _today_. If you wanted to make a new programming model that's incompatible, that's fine. (CUDA does it: GPUs don't have as many virtual-memory features as a CPU. And __shared__ memory has a different model than L1 cache.)
But if you invent a new memory model that does things differently, it means that it won't work for the vast majority of code. Which means you need to bootstrap a new programming environment (much like how CUDA bootstrapped a new community from scratch).
- vlovich123 6y agoOr the mapping info for your process is shared with that coprocessor (the OS just keeps this around in RAM anyway). Heck, you could have an OS-provided code that needs to be loaded so that it can properly resolve that mapping (which is what the TLB does by the way - it has a well-defined structure and when it’s missing from the cache if I recall correctly it’ll fetch some info directly from RAM until it gets to a point where it has to generate a page fault). I don’t disagree that the memory model becomes more complex. For one CPU cache invalidation becomes really tricky. So do memory coherence rules. I’m less clear how mmap matters here. That’s just a mechanism the OS uses to hand out views into the page cache to the process - if you’ve solve the virtual-physical mapping (which you have to do) then mmap is not relevant. You’re spot on though that the particular design decisions are critical for this to be successful - pick the wrong point on the complexity/cost/perf curve and your solution will definitely be DOA.