3 ms·
I'm only semi-familiar with the issues, but on the surface swapping across nodes rather than to disk seems like it has to be a win. I think the problem might b
by nkurz 16y ago
I'm only semi-familiar with the issues, but on the surface swapping across nodes rather than to disk seems like it has to be a win. I think the problem might be that on a long running system there is rarely any truly "free" memory. Rather, one chooses to dump cached pages. It's possible that the cost of copying across nodes plus the eventual reread of the cache makes it a negative? Although I'd have to think that it's better than a certain write.
I can see the logic where allocating on a non-local node is potentially a mistake. Depending on how many times the memory will be accessed, it may well be worth the immediate hit to swap a page to disk and keep all your accesses local. For the swap, you at least have evidence that it hasn't been used that recently, thus may never be used again. It would be sad to work yourself into a corner where lots of long lived processes are constantly cross-allocating.
Edit to add:
Looks like a good reference paper here: http//www.kernel.org/pub/linux/kernel/people/christoph/.../numamemory.pdf
Only skimmed, but it makes it sound like 'page migration' is already in place.
I'm particularly interested in the idea of migration partly because it might help provide an answer to my recent StackOverflow question: http://stackoverflow.com/questions/3784434/inserting-pages-into-large-mmap-files-without-copying-data http://stackoverflow.com/questions/3784434/inserting-pages-i...
- nkurz 16y agoThat was odd. I must have submitted at the same time that the edit window ended, and it took me to a broken page. Anyway, the proper link is: http://www.kernel.org/pub/linux/kernel/people/christoph/pmig/numamemory.pdf http://www.kernel.org/pub/linux/kernel/people/christoph/pmig...