3 ms·

It's a fairly hypothetical problem, like it may not even be a problem, other than the intuition that having 8 applications all thrashing wildly and competing fo

It's a fairly hypothetical problem, like it may not even be a problem, other than the intuition that having 8 applications all thrashing wildly and competing for the same page table may not be the most performant way of allocating resources.

ewalk153 3y ago
Have you thought of experimenting/benchmarking with two or three different architectures right now before you settle on one? As you wrote in the article, you still have the old server that can support your current load. That likely won’t be an option in the future as your load continues to grow.
- marginalia_nu 3y ago
  This is sort of what I'm doing, except not really considering virtualization as a serious option. I've worked with it a lot in the past and it's been awkward and annoying. I'll go as far as containers, but I'm seriously not seeing what virtualization would add here other than obstacles. I'm also feeling out what's a way of working with this machine that isn't a huge pain in the ass. When you've got one instance running on one machine, manual deployments is fine, but I think something more CI-driven is probably going to be necessary to keep sane with 8 index shards and a test environment as well. Kubernetes would be an option but I have bad experiences with that too. A bit too much spooky action at a distance for my taste. Whole ecosystem feels very fragile and churny in a way I'm not very happy with, and the abstractions designed for hiding away the complexities of dealing with a cluster make running it on a single machine where those abstractions aren't necessary just pointlessly awkward.
  - daoudc 3y ago
    I've been using dokku for Mwmbl and have been fairly happy with it. Congrats on the recent success by the way ;)
amluto 3y ago
> having 8 applications all thrashing wildly and competing for the same page table may not be the most performant way of allocating resources. They’re not, though. There’s a tree of “page tables” (that is, a tree with branching factor 512 or so, in the format used by the CPU) per process. Also, per process, there’s a tree of VMAs (the logical maps from contiguous virtual address ranges to whatever logically backs them) — these are created by mmap and friends. And, regrettably, a lock, also per process (although this lock is a read-write lock, and page faults are reads). If you have a whole bunch of processes mmapping the same file and thrashing against it, you could end up with contention for that files’s data structures that track mappings, but that seems unlikely. Mappings of different pages of ordinary memory should scale well. And a VM, for this purpose, is more or less like a process. QEMU (or whatever other userspace host you use) literally maps everything that the VM logically maps, and VM faults are handled as though QEMU triggered a page fault.
- marginalia_nu 3y ago
  Oh great, means I'm worrying about nothing :-)
- foota 3y ago
  VMA locking will soon be per VMA instead of shared within the process! https://lwn.net/Articles/937943/ https://lwn.net/Articles/937943/
  - amluto 3y ago
    It’s been a long time coming :)

ewalk153 3y ago

Have you thought of experimenting/benchmarking with two or three different architectures right now before you settle on one? As you wrote in the article, you still have the old server that can support your current load. That likely won’t be an option in the future as your load continues to grow.

marginalia_nu 3y ago
This is sort of what I'm doing, except not really considering virtualization as a serious option. I've worked with it a lot in the past and it's been awkward and annoying. I'll go as far as containers, but I'm seriously not seeing what virtualization would add here other than obstacles. I'm also feeling out what's a way of working with this machine that isn't a huge pain in the ass. When you've got one instance running on one machine, manual deployments is fine, but I think something more CI-driven is probably going to be necessary to keep sane with 8 index shards and a test environment as well. Kubernetes would be an option but I have bad experiences with that too. A bit too much spooky action at a distance for my taste. Whole ecosystem feels very fragile and churny in a way I'm not very happy with, and the abstractions designed for hiding away the complexities of dealing with a cluster make running it on a single machine where those abstractions aren't necessary just pointlessly awkward.
- daoudc 3y ago
  I've been using dokku for Mwmbl and have been fairly happy with it. Congrats on the recent success by the way ;)

amluto 3y ago

> having 8 applications all thrashing wildly and competing for the same page table may not be the most performant way of allocating resources. They’re not, though. There’s a tree of “page tables” (that is, a tree with branching factor 512 or so, in the format used by the CPU) per process. Also, per process, there’s a tree of VMAs (the logical maps from contiguous virtual address ranges to whatever logically backs them) — these are created by mmap and friends. And, regrettably, a lock, also per process (although this lock is a read-write lock, and page faults are reads). If you have a whole bunch of processes mmapping the same file and thrashing against it, you could end up with contention for that files’s data structures that track mappings, but that seems unlikely. Mappings of different pages of ordinary memory should scale well. And a VM, for this purpose, is more or less like a process. QEMU (or whatever other userspace host you use) literally maps everything that the VM logically maps, and VM faults are handled as though QEMU triggered a page fault.

marginalia_nu 3y ago
Oh great, means I'm worrying about nothing :-)
foota 3y ago
VMA locking will soon be per VMA instead of shared within the process! https://lwn.net/Articles/937943/ https://lwn.net/Articles/937943/
- amluto 3y ago
  It’s been a long time coming :)