3 ms·

These systems seem to be, in practice, relying on having working sets that mostly fit in RAM. It's a nice world when you can stack 64GB in a box and never worry

by herf 16y ago

These systems seem to be, in practice, relying on having working sets that mostly fit in RAM. It's a nice world when you can stack 64GB in a box and never worry about disk, but having worked on systems that need more working set than this, I don't think this idea actually scales up as well as is claimed. Two issues: 1. Small objects aren't batched by a VM, so they can't be combined into long streaming writes and reads. Doing 4k writes and reads is useless on a modern disk. 128k to 1MB is more appropriate if you can organize things that way. 2. If your workload "mostly" fits in RAM, and then as your application scales, the working set exceeds RAM, you wind up doing some queries that are instant, and some that are a million times slower (a real, uncached disk seek). Unless you plan for this, your app can fall over and you just won't know what happened. You need to know the fraction of requests that will go to disk and plan to have spindles (or RAM) to handle it. I don't like systems that make measuring what's going on at this level any harder to do. The author's claim is that you get to share RAM with the VM, which is great, but in practice a lot of systems use kernel-supported sendfile() to do work (i.e. have a small streaming buffer per network connection), or cache super-hot objects in a relatively small amount of RAM. The assumption that all user-mode caches try to allocate as much RAM as possible is not true. Alternately, larger systems separate RAM cache entirely from disk-bound boxes (e.g., dedicated memcached). I think a more complete treatment of this problem would explain scaling over working set more aggressively, and it would explain using instrumentation how the system degrades at scale.

jacquesm 16y ago
Currently serving up billions of images per day from memory images that are more than 100 times the available physical memory (and there's plenty of that). Small objects can be batched in to a single page at the application level, the VM will then move these in and out of the resident pool as one unit. The operating system uses elevator sorting and knows enough about the disk that it will attempt to sequentially invalidate pages. The assumption is not that the working set will fit (mostly) in to RAM, the assumption is that a page fault is a relatively rare occurrence and that other threads will not be stalled by the IO done for one. It is the changes to the working set that determine page faults, not the size of the total set. On my boxes I solved one issue with this (and the maximum number of systemwide sockets) by running a varnish instance for every physical CPU in the machines.

by herf 16y ago

jacquesm 16y ago

Currently serving up billions of images per day from memory images that are more than 100 times the available physical memory (and there's plenty of that). Small objects can be batched in to a single page at the application level, the VM will then move these in and out of the resident pool as one unit. The operating system uses elevator sorting and knows enough about the disk that it will attempt to sequentially invalidate pages. The assumption is not that the working set will fit (mostly) in to RAM, the assumption is that a page fault is a relatively rare occurrence and that other threads will not be stalled by the IO done for one. It is the changes to the working set that determine page faults, not the size of the total set. On my boxes I solved one issue with this (and the maximum number of systemwide sockets) by running a varnish instance for every physical CPU in the machines.