4 ms·

Sharing page tables with mshare()

41 points by chmaynard 4y ago

scottlamb 4y ago
> [Matthew Wilcox] said: why not just use threads? The answer was that "mmap_lock sucks". Not a good enough answer IMHO. I've hit problems with mmap_lock scalability too, but my understanding is there's ongoing efforts to solve them via Maple Trees. I'd so much rather have threads Just Work Well than a Linux-only, half-process half-thread interface that almost no one will use.
- cyphar 4y ago
  Matthew Wilcox is one of the people who developed Maple Trees in the first place[1]. I don't particularly like the sound of this interface either but if he views the mmap_lock issue to be that fundamental then he's one of the few people who would know. [1]: https://lwn.net/Articles/845507/ https://lwn.net/Articles/845507/
  - scottlamb 4y ago
    Oh, thanks. I feel silly for not associating his name with that. > if he views the mmap_lock issue to be that fundamental then he's one of the few people who would know. Very disappointing news if so. :-( But I don't see anything in this recent article to suggest that it's unsolvable. https://lwn.net/Articles/893906/ https://lwn.net/Articles/893906/ It'd be a shame to carry around a weird extra interface indefinitely because mmap_lock scalability didn't happen quite soon enough.
geocar 4y ago
I have known for some time to open /proc/self/mem or use hugetblfs, then send the fd to another process to mmap() so I can share pages; I've made a similar API to the proposed by combining memfd and /proc/self/fd/X. It seems like this API more-directly implements these patterns, but is that it? Or is it something else? Like, is this API going to be faster than that? Or is it just nice to give our pages cute names?
- eklitzke 4y ago
  The use case apparently is for sharing some memory region between thousands of processes, which has a lot of overhead because each process would normally need its own PTEs which add up. I agree this is really weird though, if you want thousands of things sharing the same memory region it would make more sense to me to just use threads.
  - touisteur 4y ago
    Well, threads come with their own myriad problems. Aside from isolation, with separate process one can start/stop/kill from outside in a more sysadmin-friendly way, but also upgrade gradually. I often use these kinds of mechanisms for control-planes or one writer / thousands of readers schemes (to share huge parameters or look up tables...). It's somehow far more efficient and easier to design than pub/sub or database requests and wasting loads of memory on N copies of the same data. Start the process, map the structures and go.

41 points by chmaynard 4y ago

> [Matthew Wilcox] said: why not just use threads? The answer was that "mmap_lock sucks". Not a good enough answer IMHO. I've hit problems with mmap_lock scalability too, but my understanding is there's ongoing efforts to solve them via Maple Trees. I'd so much rather have threads Just Work Well than a Linux-only, half-process half-thread interface that almost no one will use.

cyphar 4y ago
Matthew Wilcox is one of the people who developed Maple Trees in the first place[1]. I don't particularly like the sound of this interface either but if he views the mmap_lock issue to be that fundamental then he's one of the few people who would know. [1]: https://lwn.net/Articles/845507/ https://lwn.net/Articles/845507/
- scottlamb 4y ago
  Oh, thanks. I feel silly for not associating his name with that. > if he views the mmap_lock issue to be that fundamental then he's one of the few people who would know. Very disappointing news if so. :-( But I don't see anything in this recent article to suggest that it's unsolvable. https://lwn.net/Articles/893906/ https://lwn.net/Articles/893906/ It'd be a shame to carry around a weird extra interface indefinitely because mmap_lock scalability didn't happen quite soon enough.

geocar 4y ago

I have known for some time to open /proc/self/mem or use hugetblfs, then send the fd to another process to mmap() so I can share pages; I've made a similar API to the proposed by combining memfd and /proc/self/fd/X. It seems like this API more-directly implements these patterns, but is that it? Or is it something else? Like, is this API going to be faster than that? Or is it just nice to give our pages cute names?

eklitzke 4y ago
The use case apparently is for sharing some memory region between thousands of processes, which has a lot of overhead because each process would normally need its own PTEs which add up. I agree this is really weird though, if you want thousands of things sharing the same memory region it would make more sense to me to just use threads.
- touisteur 4y ago
  Well, threads come with their own myriad problems. Aside from isolation, with separate process one can start/stop/kill from outside in a more sysadmin-friendly way, but also upgrade gradually. I often use these kinds of mechanisms for control-planes or one writer / thousands of readers schemes (to share huge parameters or look up tables...). It's somehow far more efficient and easier to design than pub/sub or database requests and wasting loads of memory on N copies of the same data. Start the process, map the structures and go.