3 ms·

So, I'm the architect of Cloudflare Workers. In a previous life, I created Sandstorm.io, including implementing its container engine from scratch. I do know som

So, I'm the architect of Cloudflare Workers. In a previous life, I created Sandstorm.io, including implementing its container engine from scratch. I do know some things about containers. With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data. With Workers, we're seeing an order of magnitude better. There is no one, simple reason for this difference -- rather, it's a large number of factors working together. If I enumerated every one of them, you probably wouldn't find any of them compelling on its own. Sure, if we could convince all our customers to write tiny C programs, with just the right constraints, maybe they could be as efficient in terms of RAM usage. Maybe. In Sandstorm, a raw C/C++/Rust app could indeed fit in a couple megabytes. But there's still context switching overhead. And no one wants to write C, they want to write JavaScript. ¯\_(ツ)_/¯

cyphar 8y ago
> I do know some things about containers. Great! :D > With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data. I hate to keep harping on this, but what do you mean by "used" here? Are you saying that the RSS was 100MB per container, or that the sum of private mappings used was 100MB (/proc/$pid/smaps)? Did you use a filesystem like overlayfs that facilitates page-cache sharing by allowing read-only inode sharing between different containers, or a driver like {btrfs,devicemapper,aufs} that didn't? (Sandstorm was kicking around a while ago, so this might've been before overlayfs was in mainstream use.) I know I probably look like an asshole, but I actually don't think I understand what you mean by used -- because saying that a container uses 100MB (which I take to mean that each new container spawned costs 100MB of real mmeory) simply doesn't sound right. It implies you could run less than 80 containers on an 8GB machine, and I doubt that Sandstorm programs were this big. It's like someone telling you that they spent $1500 on lunch -- I'm actually confused what the word "spent" means here. I'm sure that you'd see less RSS memory usage by only having one process, but it is very possible that this memory usage benefit is not actually real -- I could be completely wrong but I'm just having trouble understanding how putting the same code inside a single program could make such a difference (given that the page cache already shares the memory for the V8 process). If you were to run 10k V8 processes your RSS would increase by 10k but your real memory usage would only increase very slightly because of the minor kernel memory cost of "struct task_struct". As for context switches, yeah okay that's a cost you pay by having more than one process on a machine. But you do still have context switch overhead (though obviously it's much smaller) if you're running more than one program's state in a single process -- you have to switch context to a different set of protected variables right? > And no one wants to write C, they want to write JavaScript. My point about page caches is that the code for V8 is in the page cache and thus running more V8 processes doesn't take up any more real memory than just running one. So that cost of running JavaScript on top should be similar (though probably slightly larger because V8 stores other program information in memory -- but definitely not 10x larger).
- marktangotango 8y ago
  My understanding is that he’s talking about app specific code, like yeah the os is smart enough to only load libc once, but php/node/python/rails/x are all going load up there language run times and app specific libraries into their processes. The non .so parts. Then the 100mb number makes sense.
  - cyphar 8y ago
    In theory (if most of the runtime code being loaded is the same files underneath) then the page cache will also help with loading them -- you get benefits from the page cache as long as you're opening the same inode (which is why I'm talking about overlayfs -- because overlayfs allows page-cache sharing for base container images).
    - marktangotango 8y ago
      Even then, to get significant sharing every container would need to be deploying the same frameworks, libraries, and versions, which the single process, restricted environment moots all that anyway.
      - cyphar 8y ago
        I'm confused -- the article is specifically about deploying JavaScript (or something that targets WebAssembly). From where I'm standing that seems to be a fairly homogeneous thing to be deploying (if you need to have many versions of JavaScript frameworks I don't see how that's not a problem with this solution either). "Application containers" get a bad rap because Docker's model of a bunch of tar archives with separate root filesystems isn't actually the best way of getting density (nor is it what a lot of people really want).

cyphar 8y ago

> I do know some things about containers. Great! :D > With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data. I hate to keep harping on this, but what do you mean by "used" here? Are you saying that the RSS was 100MB per container, or that the sum of private mappings used was 100MB (/proc/$pid/smaps)? Did you use a filesystem like overlayfs that facilitates page-cache sharing by allowing read-only inode sharing between different containers, or a driver like {btrfs,devicemapper,aufs} that didn't? (Sandstorm was kicking around a while ago, so this might've been before overlayfs was in mainstream use.) I know I probably look like an asshole, but I actually don't think I understand what you mean by used -- because saying that a container uses 100MB (which I take to mean that each new container spawned costs 100MB of real mmeory) simply doesn't sound right. It implies you could run less than 80 containers on an 8GB machine, and I doubt that Sandstorm programs were this big. It's like someone telling you that they spent $1500 on lunch -- I'm actually confused what the word "spent" means here. I'm sure that you'd see less RSS memory usage by only having one process, but it is very possible that this memory usage benefit is not actually real -- I could be completely wrong but I'm just having trouble understanding how putting the same code inside a single program could make such a difference (given that the page cache already shares the memory for the V8 process). If you were to run 10k V8 processes your RSS would increase by 10k but your real memory usage would only increase very slightly because of the minor kernel memory cost of "struct task_struct". As for context switches, yeah okay that's a cost you pay by having more than one process on a machine. But you do still have context switch overhead (though obviously it's much smaller) if you're running more than one program's state in a single process -- you have to switch context to a different set of protected variables right? > And no one wants to write C, they want to write JavaScript. My point about page caches is that the code for V8 is in the page cache and thus running more V8 processes doesn't take up any more real memory than just running one. So that cost of running JavaScript on top should be similar (though probably slightly larger because V8 stores other program information in memory -- but definitely not 10x larger).

marktangotango 8y ago
My understanding is that he’s talking about app specific code, like yeah the os is smart enough to only load libc once, but php/node/python/rails/x are all going load up there language run times and app specific libraries into their processes. The non .so parts. Then the 100mb number makes sense.
- cyphar 8y ago
  In theory (if most of the runtime code being loaded is the same files underneath) then the page cache will also help with loading them -- you get benefits from the page cache as long as you're opening the same inode (which is why I'm talking about overlayfs -- because overlayfs allows page-cache sharing for base container images).
  - marktangotango 8y ago
    Even then, to get significant sharing every container would need to be deploying the same frameworks, libraries, and versions, which the single process, restricted environment moots all that anyway.
    - cyphar 8y ago
      I'm confused -- the article is specifically about deploying JavaScript (or something that targets WebAssembly). From where I'm standing that seems to be a fairly homogeneous thing to be deploying (if you need to have many versions of JavaScript frameworks I don't see how that's not a problem with this solution either). "Application containers" get a bad rap because Docker's model of a bunch of tar archives with separate root filesystems isn't actually the best way of getting density (nor is it what a lot of people really want).