4 ms·

I appreciate the novel approach, but you should try to use more realistic numbers for the arguments in the post. A node process doing nothing does not consume 3

I appreciate the novel approach, but you should try to use more realistic numbers for the arguments in the post. A node process doing nothing does not consume 30MB of private memory (shared mappings are shared, just like the "shared runtime"), nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first and almost 2 orders on the second. Cold starts are a symptom of something else (likely shipping function data), which I don't think is really a property of the sandbox.

IanCal 8y ago
> nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first I'm relatively sure 10ms is not the worst case cold start time of lambda. > Cold starts are a symptom of something else (likely shipping function data) Which is all part of spinning up a containerised function in lambda. About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.
- cyphar 8y ago
  > About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. What do you mean by "use" in this context? Does every program that links to glibc "use" the memory for glibc? (The answer is no, because the page cache is shared. But if you just looked at memory usage you might be fooled into thinking it is copied for every program.) Containers use a drastically tiny amount of in-kernel memory. We are talking fewer than a couple of kilobytes in the worst case. It appears to me that the main worry was with using Kubernetes, and they've applied this to containers as a whole. I agree that in this sort of usecase Kubernetes really doesn't make sense (using container images at all makes no sense in this case), but the core kernel primitives and simple container runtimes available are more than suitable. There is an argument that having no userland-kernel context switches has a performance improvement (and it does) but I would like to see data before accepting at face value that "context switches are expensive" is an appropriate thing to optimise for. After all they are still doing context switches, it's just in userspace. And if v8 is multi-threaded they've just reinvented the N-to-M scheduling model that is very well known to be fundamentally broken because of various well known pathologies.
  - marktangotango 8y ago
    Your argument is moot when it comes to original point though, isn’t it? Can a single machine handle 10,000 containers with acceptable perfmance at load? Versus the single process handling work for 10,000 customers? Substitute n for 10k here it’s an arbitrary number.
    - cyphar 8y ago
      I'm not sure how I could, without having access to CloudFlare-scale compute and load, be able to answer your question. My point is that the article in question makes claims that I'm not sure are correct, and I wonder whether the engineers behind this solution decided against containers because of their initial experience (this isn't a dig against them, this is a very common trait of almost everyone -- you don't want to waste your time if the first impression you have is negative). Several of the statements (especially about memory) appear to be based on misunderstandings of how containers would actually operate in such a scenario (or based on testing with sub-optimal configuration). Maybe I'm completely wrong that containers would work under this kind of load in this scenario, but reading the article I didn't find many arguments I would expect to see if someone had really tried to make it work with containers and found the flaws. Instead it reads (at least to me) to be more of a "at first glance this doesn't appear to work" -- which is a fine thing to base product decisions on, but it isn't really okay to then spread (what appears to me to be) misinformation unless you have done significant testing to justify it. So I disagree that my argument is moot -- and I would like to know how much cheaper the userland context switches are in V8 (I just looked and it appears that V8 does have a multi-threaded core now -- but hopefully CloudFlare doesn't use that with their userland threading+isolation model...).
      - kentonv 8y ago
        So, I'm the architect of Cloudflare Workers. In a previous life, I created Sandstorm.io, including implementing its container engine from scratch. I do know some things about containers. With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data. With Workers, we're seeing an order of magnitude better. There is no one, simple reason for this difference -- rather, it's a large number of factors working together. If I enumerated every one of them, you probably wouldn't find any of them compelling on its own. Sure, if we could convince all our customers to write tiny C programs, with just the right constraints, maybe they could be as efficient in terms of RAM usage. Maybe. In Sandstorm, a raw C/C++/Rust app could indeed fit in a couple megabytes. But there's still context switching overhead. And no one wants to write C, they want to write JavaScript. ¯\_(ツ)_/¯
- amscanne 8y ago
  A lambda cold start is not equivalent to starting a process. There's obviously a lot of things happening beyond that. Does it take 500ms every time you type 'ls' in a console? That's starting a process. > About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. It's not the claim. I wouldn't object to someone saying "we charge X, lambda charge Y". But the post seems to confound implementation decisions in lamda with fundamental properties of containers and processes, which are simply not correct.
zackbloom 8y ago
35MB is what Amazon reports and bills you based on. If that number is wrong, a great question is what are they charging people for? Launching a process is different than launching a language runtime or interpreter. They generally are not optimized for initial start time in the way an Isolate is. The duration of a context switch varies, but when dealing with tens of thousands of requests per second per machine they are very significant.
- nroets 8y ago
  Using billing to measure the size or space of something is the worst kind of measure. For example, I recently got on a Grey Hound with my bicycle, packaged in a box. My ticket was $27, but the bike cost $30, even though I weight 3 to 4 times what my bike does. Humans also need thousands of liters of space, while the bicycle box only took up 200 liter or so. And the time the bus driver took to load the bicycle is negligible.
- amscanne 8y ago
  I'm not sure why you think process start-up is such a heavyweight operation. Of course there's lots of optimization for start-up, it's the whole basis for Unix. A context switch is O(10000) cycles and 10k is totally doable. I agree that the Isolate architecture is more efficient, I just object to the straw man comparison. (And would love some real data.)
- blasdel 8y ago
  AWS uses virtualization domains to separate customer workloads, and Meltdown required that they all be HVM to provide strong guarantees: https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00436.html https://lists.xenproject.org/archives/html/xen-devel/2018-01... Your account's Lambda functions won't share a kernel with anyone else's

IanCal 8y ago

> nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first I'm relatively sure 10ms is not the worst case cold start time of lambda. > Cold starts are a symptom of something else (likely shipping function data) Which is all part of spinning up a containerised function in lambda. About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.

cyphar 8y ago
> About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. What do you mean by "use" in this context? Does every program that links to glibc "use" the memory for glibc? (The answer is no, because the page cache is shared. But if you just looked at memory usage you might be fooled into thinking it is copied for every program.) Containers use a drastically tiny amount of in-kernel memory. We are talking fewer than a couple of kilobytes in the worst case. It appears to me that the main worry was with using Kubernetes, and they've applied this to containers as a whole. I agree that in this sort of usecase Kubernetes really doesn't make sense (using container images at all makes no sense in this case), but the core kernel primitives and simple container runtimes available are more than suitable. There is an argument that having no userland-kernel context switches has a performance improvement (and it does) but I would like to see data before accepting at face value that "context switches are expensive" is an appropriate thing to optimise for. After all they are still doing context switches, it's just in userspace. And if v8 is multi-threaded they've just reinvented the N-to-M scheduling model that is very well known to be fundamentally broken because of various well known pathologies.
- marktangotango 8y ago
  Your argument is moot when it comes to original point though, isn’t it? Can a single machine handle 10,000 containers with acceptable perfmance at load? Versus the single process handling work for 10,000 customers? Substitute n for 10k here it’s an arbitrary number.
  - cyphar 8y ago
    I'm not sure how I could, without having access to CloudFlare-scale compute and load, be able to answer your question. My point is that the article in question makes claims that I'm not sure are correct, and I wonder whether the engineers behind this solution decided against containers because of their initial experience (this isn't a dig against them, this is a very common trait of almost everyone -- you don't want to waste your time if the first impression you have is negative). Several of the statements (especially about memory) appear to be based on misunderstandings of how containers would actually operate in such a scenario (or based on testing with sub-optimal configuration). Maybe I'm completely wrong that containers would work under this kind of load in this scenario, but reading the article I didn't find many arguments I would expect to see if someone had really tried to make it work with containers and found the flaws. Instead it reads (at least to me) to be more of a "at first glance this doesn't appear to work" -- which is a fine thing to base product decisions on, but it isn't really okay to then spread (what appears to me to be) misinformation unless you have done significant testing to justify it. So I disagree that my argument is moot -- and I would like to know how much cheaper the userland context switches are in V8 (I just looked and it appears that V8 does have a multi-threaded core now -- but hopefully CloudFlare doesn't use that with their userland threading+isolation model...).
    - kentonv 8y ago
      So, I'm the architect of Cloudflare Workers. In a previous life, I created Sandstorm.io, including implementing its container engine from scratch. I do know some things about containers. With Sandstorm.io, the rule of thumb we landed on was that a container takes 100MB of RAM. Some apps used more, some used less. This is real, empirical data. With Workers, we're seeing an order of magnitude better. There is no one, simple reason for this difference -- rather, it's a large number of factors working together. If I enumerated every one of them, you probably wouldn't find any of them compelling on its own. Sure, if we could convince all our customers to write tiny C programs, with just the right constraints, maybe they could be as efficient in terms of RAM usage. Maybe. In Sandstorm, a raw C/C++/Rust app could indeed fit in a couple megabytes. But there's still context switching overhead. And no one wants to write C, they want to write JavaScript. ¯\_(ツ)_/¯
amscanne 8y ago
A lambda cold start is not equivalent to starting a process. There's obviously a lot of things happening beyond that. Does it take 500ms every time you type 'ls' in a console? That's starting a process. > About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. It's not the claim. I wouldn't object to someone saying "we charge X, lambda charge Y". But the post seems to confound implementation decisions in lamda with fundamental properties of containers and processes, which are simply not correct.

zackbloom 8y ago

35MB is what Amazon reports and bills you based on. If that number is wrong, a great question is what are they charging people for? Launching a process is different than launching a language runtime or interpreter. They generally are not optimized for initial start time in the way an Isolate is. The duration of a context switch varies, but when dealing with tens of thousands of requests per second per machine they are very significant.

nroets 8y ago
Using billing to measure the size or space of something is the worst kind of measure. For example, I recently got on a Grey Hound with my bicycle, packaged in a box. My ticket was $27, but the bike cost $30, even though I weight 3 to 4 times what my bike does. Humans also need thousands of liters of space, while the bicycle box only took up 200 liter or so. And the time the bus driver took to load the bicycle is negligible.
amscanne 8y ago
I'm not sure why you think process start-up is such a heavyweight operation. Of course there's lots of optimization for start-up, it's the whole basis for Unix. A context switch is O(10000) cycles and 10k is totally doable. I agree that the Isolate architecture is more efficient, I just object to the straw man comparison. (And would love some real data.)
blasdel 8y ago
AWS uses virtualization domains to separate customer workloads, and Meltdown required that they all be HVM to provide strong guarantees: https://lists.xenproject.org/archives/html/xen-devel/2018-01/msg00436.html https://lists.xenproject.org/archives/html/xen-devel/2018-01... Your account's Lambda functions won't share a kernel with anyone else's