10 ms·

Hi, I'm the tech lead of Workers. Note that the core point here is multi-tenancy. With Isolates, we can host 10,000+ tenants on one machine -- something that's

Hi, I'm the tech lead of Workers. Note that the core point here is multi-tenancy. With Isolates, we can host 10,000+ tenants on one machine -- something that's totally unrealistic with containers or VMs. Why does this matter? Two things: 1) It means we can run your code in many more places, because the cost of each additional location is so low. So, your code can run close to the end user rather than in a central location. 2) If your Worker makes requests to a third-party API that is also implemented as a Cloudflare Worker, this request doesn't even leave the machine. The other worker runs locally. The idea that you can have a stack of cloud services that depend on each other without incurring any latency or bandwidth costs will, I think, be game-changing.

mdip 8y ago
Greetings Tech Lead of Workers! I have yet to use this specific product and after reading the blog post I'm actually really frustrated that I didn't fully understand the things that I could do with Workers as I had a proof-of-concept app recently that would have been nice to test with since we've had issues with cold start delays on Lambda in similar projects in the past. And that second point that you make, above, ... holy cow; that's going to change the way I design systems and applications for customers. Personally speaking, I am a fan of CloudFlare like some of the more obsessed are fans of Apple[0]. Every time your company comes out with a new announcement, it's solves some problem I'm experiencing on projects at that moment. I could write paragraph after paragraph of all of the grief that using your services has eliminated from mine and my customer's lives, but I am certain you get that enough, already. There is no way my customer's appreciate your free tier as much as I, the developer, do. And your UX is awesome. Whichever one of your team mates designed the API button with instructions next to each option in the control panel as a way to onboard people to the API ... it's those stupid little things -- because that's there, when I'm setting up a client with a unique configuration for the first time, I start from a baseline, set it manually, test, and click into that link to see what commands I'm putting into the script should I have another client with similar needs. I probably wouldn't have thought to even look to see if you had an API were it not for those links. I'm excited to try out Workers next time I have a Lambda need that fits well. Keep up the good work -- it's taking a ton of grief out of my life. [0] A gentleman from your company sent me a T-Shirt for providing feedback for the Argo tunnel (called something different back then in beta) product -- you guys even do the damn T-Shirts right ... subtle logo on the shoulder and front, no words, and really soft cotton. It's my favorite "free vendor t-shirt" shirt. edit: A sentence ... got away from me and I had a dangling bullet
- eastdakota 8y ago
  Thank you. Made my day.
le-mark 8y ago
Does V8 allow sandboxing on: cpu cycles, heap, stack and system resources? Are requests to these workers http requests? Is there any backend storage or file access (or blob) from workers?
- kentonv 8y ago
  > Does V8 allow sandboxing on: cpu cycles, heap, stack and system resources? Yes, and that is critical to our use case. > Are requests to these workers http requests? Currently, yes. > Is there any backend storage or file access (or blob) from workers? Currently there is Workers KV (key-value storage), and we have some cooler stuff in the works.
zackbloom 8y ago
(I sit next to Kenton) I would actually argue with #1. The truth is, this appears to be a better tech even if it isn't globally distributed. Lightweight serverless functions with no cold-start are a really great primitive to build on. We figured them out in this way to enable global distribution, but that's a bonus, not necessary to really love the concept.
devit 8y ago
Have you tried simply using a seccomp-isolated Linux executable for each tenant? (with rlimit or cgroup-based resource limits) You have to create a process for each tenant, but that should be much faster and take much less memory than V8 JITting JavaScript or WebAssembly (Linux process creation is very well optimized). You can try static linking, dynamic linking or even just forking plus an ad-hoc executable loader in userspace.
- elithrar 8y ago
  Clients are intentionally writing JavaScript against (a subset of) the Service Worker API; so you will have to spawn a V8 process & context anyway. V8 isolates are the same separation as browser tabs, and thus are also designed to be initialized extremely quickly. I’d be surprised if spawning a new process is faster, and it’s not going to be faster if your goal is to run JS/WASM.
- wmf 8y ago
  Basically CGIs. This would be great if worker functions were written in C, but these days Web programming is done in dynamic languages with fat runtimes where hello world uses 50 MB of RAM. They're saving all that RAM by not using multiple processes.
tigershark 8y ago
Sorry, I just read the linked article so I don’t really know the exact specific of your system, but I think I’m having a flashback. These are exactly all the promises of the sandboxed JVM running in a browser, probably with 10-20x performance deficit to account for JavaScript vs java. I think that after about 20 years we all know why that kind of “security” model failed. I’m actually quite stunned seeing that you are running code from thousands of different websites in the same process. The only game changing part that I can see is giving a huge incentive to a lot of not so good people to try to break the sandbox. And I was hoping that after the intel spectre debacle we were finally going to move in the right direction... Apparently we are still going in the wrong one. I really hope that I misunderstood everything and that you are not really running code from thousands of different websites in the same process.
- zackbloom 8y ago
  One of the fundamental truths behind any innovation is it breaks at least one 'sacred' belief of the last generation. The JVM was not a technology built for this kind of multi-tenancy, it was bolted on later. V8 is one of the most well tested and secured pieces of software in existance, and that it has a much smaller surface area than the Linux process isolation you're referring to.
  - donmcronald 8y ago
    > 10,000+ tenants on one machine > a huge incentive to a lot of not so good people to try to break the sandbox > $15k bounty on this exact type of exploit Lol. What a joke.
  - doctorless 8y ago
    On the other hand, containerization typically occurs on an independent (from other owners) instance of a virtual machine, which typically is running on separate processor cores, helping increase the overall isolation despite residing on shared hardware. Exposed processor caches due to exploits like Meltdown are a significantly higher risk on a platform of this kind than on a containerized environment. V8 exists at a much higher level than hardware-level exploits. How does your platform mitigate these kinds of concerns? Presumably you have some kind of virtualization above this to manage roll out of your execution environment, but adding a shared execution context like V8 feels to me like the risk factor is doubled, not reduced.
amirhirsch 8y ago
I've been closely following CF Workers and have KV beta access, but the mere fact that we had to ask and wait for access to KV essentially made our decision for us to not use workers. When will KV be out of beta? It's hard to commit to a platform like CF Workers when it's obviously still not intended for mainstream usage and there is no public timeline.
- alexnewman 8y ago
  By the way kentonv, thanks for the protobuffs and sandstorm. I work with amir and I just wanted to provide a positive note of it's cool tech. Amir is correct that we still don't feel like the kvstore is mainstream? When does kvstore go ga?
- zackbloom 8y ago
  Hi! I work on KV at Cloudflare. Thank you so much for your interest in it, and for considering Workers at all. I'm pretty convinced if you keep considering it when situations allow, you'll choose it eventually when the moment is right for the product and your company. I'm happy to give you beta access for you to experiment. That said, we take the distinction between a beta and GA very seriously. We don't want you relying on software we're not, at the very least, using in production ourselves. That day will come very soon! We're moving several projects into KV (allowing us to move them out of centralized data centers), and we expect KV to track GA in the next few months.
Nokinside 8y ago
Hi, Do you have plans to deploy this tech at cell towers in metro areas and support 5G-enabled applications in the future?
- zackbloom 8y ago
  Yes, we would love to do that. Workers are lightweight enough that they appear to be a much more viable way to distribute compute than technologies like Kubernetes.
  - orf 8y ago
    *More viable way to distribute JavaScript compute?
    - zackbloom 8y ago
      C/C++/Rust/Go all target WebAssembly.
- eastdakota 8y ago
  Yes. And a really interesting business model to incent mobile providers to install them. Stay tuned!
agwa 8y ago
> If your Worker makes requests to a third-party API that is also implemented as a Cloudflare Worker, this request doesn't even leave the machine. The other worker runs locally. Doesn't this undermine your assumption that external timers can't be used in a timing side channel attack because "the network is extremely noisy"[1]? Consider a third-party API implemented as a Cloudflare Worker that just returns Date.now(). How much noise will there be if that runs locally? [1] https://news.ycombinator.com/item?id=18280156 https://news.ycombinator.com/item?id=18280156
- kentonv 8y ago
  We control the target isolate's Date.now() too, and we can make it return the same value in both workers. Indeed, we can run the workers on the same thread, which makes the whole thing look more like a function call than a network request.
  - agwa 8y ago
    Nice! Thanks for your response.
emmelaich 8y ago
That's cool. Do you use any of your tech from sandstorm days?
discoball 8y ago
At 1/10,000th of a server per each tenant (assuming 10,000 tenants) you can achieve “hard” isolation and far more cpu/memory per tenant using a TinkerBoard or something like that per client and charging a flat $2/mo with no hourly fees. That’s a business model that I see materializing that can eat into your business. EDIT to clarify: I’m assuming they can do 10,000 tenants per server. If they did 1 tenant per 1 TinkerBoard and charged $2/mo for it flat no hourly fee that would be an interesting business model IMO and it achieves hard isolation between tenacts.
- geofft 8y ago
  I think you're assuming that Cloudflare has only one server?
  - discoball 8y ago
    Nope. I’m assuming they can do 10,000 tenants per server. If they did 1 tenant per 1 TinkerBoard and charged $2/mo for it flat no hourly fee that would be an interesting business model IMO and it achieves hard isolation between tenants.
    - geofft 8y ago
      But a Workers customer gets to use more than one of their servers at once....
      - discoball 8y ago
        You can have multiple tenants per low cost edge device. ARM servers are finding so many such applications. Not sure what is the hang up?
- kentonv 8y ago
  We replicate each tenant to thousands of servers across more than 150 locations worldwide. That's why we need to support so many tenants per server.
scottlamb 8y ago
Chrome recently started running each site's JavaScript in an isolated process, as a Spectre mitigation. [1] Why do you trust V8's isolation more than Chrome does? [1] https://security.googleblog.com/2018/07/mitigating-spectre-with-site-isolation.html https://security.googleblog.com/2018/07/mitigating-spectre-w...
- scottlamb 8y ago
  Never mind, I missed the previous discussion of that here: https://news.ycombinator.com/item?id=18418476 https://news.ycombinator.com/item?id=18418476
binarycrusader 8y ago
This isn’t true. Solaris 11 can have thousands of zones (containers) on one machine. Solaris zones are a very lightweight technology compared to many others on the market. Linux may not, but Solaris can. Yes those containers run Solaris not Linux, but I just wanted to point out that this ability is not unique.
- cyphar 8y ago
  It can also be done on Linux. Docker might not be able to do it, but that's because of its architecture and the language it's written in. There's no kernel limitation like that.
  - shandor 8y ago
    What are some container technologies that could be used for this right now?
    - cyphar 8y ago
      Namespaces and cgroups (using overlayfs to get page cache sharing for the main process mappings in each container). This is a bit of a coy response, but if you have a usecase where other container solutions have a high start-up cost -- write your own. You can write a basic container runtime in tens of lines of shell script. For existing container glue -- you could probably do this with runc (though unfortunately we are also written in Go) and the architecture is such that runc has no long-running process (only your container continues running after we've set it up). I believe LXC can also produce similar tenancy (and they're written in C which avoids quite a few of the Go problems).
    - neilwilson 8y ago
      chroot, cgroup, and the systemd run facilities (systemd-nspawn for example).
      - cyphar 8y ago
        pivot_root, not chroot. chroot is vulnerable to many trivial escapes.
amscanne 8y ago
I appreciate the novel approach, but you should try to use more realistic numbers for the arguments in the post. A node process doing nothing does not consume 30MB of private memory (shared mappings are shared, just like the "shared runtime"), nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first and almost 2 orders on the second. Cold starts are a symptom of something else (likely shipping function data), which I don't think is really a property of the sandbox.
- IanCal 8y ago
  > nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first I'm relatively sure 10ms is not the worst case cold start time of lambda. > Cold starts are a symptom of something else (likely shipping function data) Which is all part of spinning up a containerised function in lambda. About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.
  - cyphar 8y ago
    > About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. What do you mean by "use" in this context? Does every program that links to glibc "use" the memory for glibc? (The answer is no, because the page cache is shared. But if you just looked at memory usage you might be fooled into thinking it is copied for every program.) Containers use a drastically tiny amount of in-kernel memory. We are talking fewer than a couple of kilobytes in the worst case. It appears to me that the main worry was with using Kubernetes, and they've applied this to containers as a whole. I agree that in this sort of usecase Kubernetes really doesn't make sense (using container images at all makes no sense in this case), but the core kernel primitives and simple container runtimes available are more than suitable. There is an argument that having no userland-kernel context switches has a performance improvement (and it does) but I would like to see data before accepting at face value that "context switches are expensive" is an appropriate thing to optimise for. After all they are still doing context switches, it's just in userspace. And if v8 is multi-threaded they've just reinvented the N-to-M scheduling model that is very well known to be fundamentally broken because of various well known pathologies.
cyphar 8y ago
> we can host 10,000+ tenants on one machine -- something that's totally unrealistic with containers or VMs. This is definitely possible with containers. Docker has historically had density problems for a variety of reasons (its architecture as well as Go being awful for systems programming in this context). But you can definitely get this density with a simple container runtime that doesn't have large long running processes. Saying you can't run 10k containers is like saying you can't run 10k processes on Linux. I'm sure that it might be somewhat faster to have it supported in V8 directly, but I don't want people to think such density is impossible with containers. (And with page-cache sharing you'd be able to not have to load 10k V8 programs into memory as well.)
- hamandcheese 8y ago
  I think you are missing the point. Regardless of the number of processes you can run, the overhead of starting them and context switching between them is dramatically higher than the equivalent operations within a single V8 process.
  - cyphar 8y ago
    The part of the comment I was responding to was talking about it being impractical to have that many containers running, which isn't true. Later I mentioned that there are probably other benefits (having everything in one process does remove the need for TLB flushes and so on), but that isn't the only thing they said. They didn't say it's drastically faster, they said it's "unrealistic" which simply isn't true. Also things like page cache sharing would make a huge difference to the real memory cost of having many processes. When you run 10k programs you don't have 10k copies of glibc.
- bambataa 8y ago
  Out of interest, why do you say that Go is awful for systems programming in this context?
ec109685 8y ago
How do you protect two isolates from interfearing with one another if they land in same process? If two of them happen to both start a calculation that takes 50ms of cpu time, do both calculations end up taking 100ms to complete? Or do you prevent concurrent execution some how?
shaklee3 8y ago
Being a non-js person, it wasn't clear if this is just for JavaScript or not. If it is, then it's not really fair to compare against containers or vms, right?
- rickycook 8y ago
  that was my thought as well. in the section about the down sides, they do mention though that they should be able to run anything that compiles to web assembly. they specifically call out rust and go. wasm really has quite a lot of interesting and unexpected applications! (though of course non-wasm tech can accomplish the same thing, but it’s nice that we are getting close to some form of universal binary format)
rodgerd 8y ago
So hyperthreading and multicores are a security mess, but a JavaScript VM is going to safely allow "10,000 tenants" on a single machine? Sure.
- zackbloom 8y ago
  I think it's very reasonable that the best possible security is not implemented in microcode.
- alexnewman 8y ago
  Hyperthread, multicores, rings, were all stability boundries hijacked to be security boundries. It'll never work
rawoke083600 8y ago
Well done! Very creative solution to the problem and using the V8 system as a starting point is brilliant!
zeckalpha 8y ago
The marketing around “No VM” is interesting. It sounds like you’re relying on V8. What did you do instead of using its VM? Precompile? Or did you reimplement V8s isolates in another VM-less system? Or do you mean more specifically no OS-level VM?
mwcampbell 8y ago
> The idea that you can have a stack of cloud services that depend on each other without incurring any latency or bandwidth costs will, I think, be game-changing. This looks to me like just another form of vendor lock-in.
espeed 8y ago
Does your implementation support vectorized SIMD or SSE instructions?
vicaya 8y ago
Running 10k containers on one machine is totally realistic though. This is just yet another language vm (e.g. jvm) isolation all over again.

mdip 8y ago

Greetings Tech Lead of Workers! I have yet to use this specific product and after reading the blog post I'm actually really frustrated that I didn't fully understand the things that I could do with Workers as I had a proof-of-concept app recently that would have been nice to test with since we've had issues with cold start delays on Lambda in similar projects in the past. And that second point that you make, above, ... holy cow; that's going to change the way I design systems and applications for customers. Personally speaking, I am a fan of CloudFlare like some of the more obsessed are fans of Apple[0]. Every time your company comes out with a new announcement, it's solves some problem I'm experiencing on projects at that moment. I could write paragraph after paragraph of all of the grief that using your services has eliminated from mine and my customer's lives, but I am certain you get that enough, already. There is no way my customer's appreciate your free tier as much as I, the developer, do. And your UX is awesome. Whichever one of your team mates designed the API button with instructions next to each option in the control panel as a way to onboard people to the API ... it's those stupid little things -- because that's there, when I'm setting up a client with a unique configuration for the first time, I start from a baseline, set it manually, test, and click into that link to see what commands I'm putting into the script should I have another client with similar needs. I probably wouldn't have thought to even look to see if you had an API were it not for those links. I'm excited to try out Workers next time I have a Lambda need that fits well. Keep up the good work -- it's taking a ton of grief out of my life. [0] A gentleman from your company sent me a T-Shirt for providing feedback for the Argo tunnel (called something different back then in beta) product -- you guys even do the damn T-Shirts right ... subtle logo on the shoulder and front, no words, and really soft cotton. It's my favorite "free vendor t-shirt" shirt. edit: A sentence ... got away from me and I had a dangling bullet

eastdakota 8y ago
Thank you. Made my day.

le-mark 8y ago

Does V8 allow sandboxing on: cpu cycles, heap, stack and system resources? Are requests to these workers http requests? Is there any backend storage or file access (or blob) from workers?

kentonv 8y ago
> Does V8 allow sandboxing on: cpu cycles, heap, stack and system resources? Yes, and that is critical to our use case. > Are requests to these workers http requests? Currently, yes. > Is there any backend storage or file access (or blob) from workers? Currently there is Workers KV (key-value storage), and we have some cooler stuff in the works.

zackbloom 8y ago

(I sit next to Kenton) I would actually argue with #1. The truth is, this appears to be a better tech even if it isn't globally distributed. Lightweight serverless functions with no cold-start are a really great primitive to build on. We figured them out in this way to enable global distribution, but that's a bonus, not necessary to really love the concept.

devit 8y ago

Have you tried simply using a seccomp-isolated Linux executable for each tenant? (with rlimit or cgroup-based resource limits) You have to create a process for each tenant, but that should be much faster and take much less memory than V8 JITting JavaScript or WebAssembly (Linux process creation is very well optimized). You can try static linking, dynamic linking or even just forking plus an ad-hoc executable loader in userspace.

elithrar 8y ago
Clients are intentionally writing JavaScript against (a subset of) the Service Worker API; so you will have to spawn a V8 process & context anyway. V8 isolates are the same separation as browser tabs, and thus are also designed to be initialized extremely quickly. I’d be surprised if spawning a new process is faster, and it’s not going to be faster if your goal is to run JS/WASM.
wmf 8y ago
Basically CGIs. This would be great if worker functions were written in C, but these days Web programming is done in dynamic languages with fat runtimes where hello world uses 50 MB of RAM. They're saving all that RAM by not using multiple processes.

tigershark 8y ago

Sorry, I just read the linked article so I don’t really know the exact specific of your system, but I think I’m having a flashback. These are exactly all the promises of the sandboxed JVM running in a browser, probably with 10-20x performance deficit to account for JavaScript vs java. I think that after about 20 years we all know why that kind of “security” model failed. I’m actually quite stunned seeing that you are running code from thousands of different websites in the same process. The only game changing part that I can see is giving a huge incentive to a lot of not so good people to try to break the sandbox. And I was hoping that after the intel spectre debacle we were finally going to move in the right direction... Apparently we are still going in the wrong one. I really hope that I misunderstood everything and that you are not really running code from thousands of different websites in the same process.

zackbloom 8y ago
One of the fundamental truths behind any innovation is it breaks at least one 'sacred' belief of the last generation. The JVM was not a technology built for this kind of multi-tenancy, it was bolted on later. V8 is one of the most well tested and secured pieces of software in existance, and that it has a much smaller surface area than the Linux process isolation you're referring to.
- donmcronald 8y ago
  > 10,000+ tenants on one machine > a huge incentive to a lot of not so good people to try to break the sandbox > $15k bounty on this exact type of exploit Lol. What a joke.
- doctorless 8y ago
  On the other hand, containerization typically occurs on an independent (from other owners) instance of a virtual machine, which typically is running on separate processor cores, helping increase the overall isolation despite residing on shared hardware. Exposed processor caches due to exploits like Meltdown are a significantly higher risk on a platform of this kind than on a containerized environment. V8 exists at a much higher level than hardware-level exploits. How does your platform mitigate these kinds of concerns? Presumably you have some kind of virtualization above this to manage roll out of your execution environment, but adding a shared execution context like V8 feels to me like the risk factor is doubled, not reduced.

amirhirsch 8y ago

I've been closely following CF Workers and have KV beta access, but the mere fact that we had to ask and wait for access to KV essentially made our decision for us to not use workers. When will KV be out of beta? It's hard to commit to a platform like CF Workers when it's obviously still not intended for mainstream usage and there is no public timeline.

alexnewman 8y ago
By the way kentonv, thanks for the protobuffs and sandstorm. I work with amir and I just wanted to provide a positive note of it's cool tech. Amir is correct that we still don't feel like the kvstore is mainstream? When does kvstore go ga?
zackbloom 8y ago
Hi! I work on KV at Cloudflare. Thank you so much for your interest in it, and for considering Workers at all. I'm pretty convinced if you keep considering it when situations allow, you'll choose it eventually when the moment is right for the product and your company. I'm happy to give you beta access for you to experiment. That said, we take the distinction between a beta and GA very seriously. We don't want you relying on software we're not, at the very least, using in production ourselves. That day will come very soon! We're moving several projects into KV (allowing us to move them out of centralized data centers), and we expect KV to track GA in the next few months.

Nokinside 8y ago

Hi, Do you have plans to deploy this tech at cell towers in metro areas and support 5G-enabled applications in the future?

zackbloom 8y ago
Yes, we would love to do that. Workers are lightweight enough that they appear to be a much more viable way to distribute compute than technologies like Kubernetes.
- orf 8y ago
  *More viable way to distribute JavaScript compute?
  - zackbloom 8y ago
    C/C++/Rust/Go all target WebAssembly.
eastdakota 8y ago
Yes. And a really interesting business model to incent mobile providers to install them. Stay tuned!

agwa 8y ago

> If your Worker makes requests to a third-party API that is also implemented as a Cloudflare Worker, this request doesn't even leave the machine. The other worker runs locally. Doesn't this undermine your assumption that external timers can't be used in a timing side channel attack because "the network is extremely noisy"[1]? Consider a third-party API implemented as a Cloudflare Worker that just returns Date.now(). How much noise will there be if that runs locally? [1] https://news.ycombinator.com/item?id=18280156 https://news.ycombinator.com/item?id=18280156

kentonv 8y ago
We control the target isolate's Date.now() too, and we can make it return the same value in both workers. Indeed, we can run the workers on the same thread, which makes the whole thing look more like a function call than a network request.
- agwa 8y ago
  Nice! Thanks for your response.

emmelaich 8y ago

That's cool. Do you use any of your tech from sandstorm days?

discoball 8y ago

At 1/10,000th of a server per each tenant (assuming 10,000 tenants) you can achieve “hard” isolation and far more cpu/memory per tenant using a TinkerBoard or something like that per client and charging a flat $2/mo with no hourly fees. That’s a business model that I see materializing that can eat into your business. EDIT to clarify: I’m assuming they can do 10,000 tenants per server. If they did 1 tenant per 1 TinkerBoard and charged $2/mo for it flat no hourly fee that would be an interesting business model IMO and it achieves hard isolation between tenacts.

geofft 8y ago
I think you're assuming that Cloudflare has only one server?
- discoball 8y ago
  Nope. I’m assuming they can do 10,000 tenants per server. If they did 1 tenant per 1 TinkerBoard and charged $2/mo for it flat no hourly fee that would be an interesting business model IMO and it achieves hard isolation between tenants.
  - geofft 8y ago
    But a Workers customer gets to use more than one of their servers at once....
    - discoball 8y ago
      You can have multiple tenants per low cost edge device. ARM servers are finding so many such applications. Not sure what is the hang up?
kentonv 8y ago
We replicate each tenant to thousands of servers across more than 150 locations worldwide. That's why we need to support so many tenants per server.

scottlamb 8y ago

Chrome recently started running each site's JavaScript in an isolated process, as a Spectre mitigation. [1] Why do you trust V8's isolation more than Chrome does? [1] https://security.googleblog.com/2018/07/mitigating-spectre-with-site-isolation.html https://security.googleblog.com/2018/07/mitigating-spectre-w...

scottlamb 8y ago
Never mind, I missed the previous discussion of that here: https://news.ycombinator.com/item?id=18418476 https://news.ycombinator.com/item?id=18418476

binarycrusader 8y ago

This isn’t true. Solaris 11 can have thousands of zones (containers) on one machine. Solaris zones are a very lightweight technology compared to many others on the market. Linux may not, but Solaris can. Yes those containers run Solaris not Linux, but I just wanted to point out that this ability is not unique.

cyphar 8y ago
It can also be done on Linux. Docker might not be able to do it, but that's because of its architecture and the language it's written in. There's no kernel limitation like that.
- shandor 8y ago
  What are some container technologies that could be used for this right now?
  - cyphar 8y ago
    Namespaces and cgroups (using overlayfs to get page cache sharing for the main process mappings in each container). This is a bit of a coy response, but if you have a usecase where other container solutions have a high start-up cost -- write your own. You can write a basic container runtime in tens of lines of shell script. For existing container glue -- you could probably do this with runc (though unfortunately we are also written in Go) and the architecture is such that runc has no long-running process (only your container continues running after we've set it up). I believe LXC can also produce similar tenancy (and they're written in C which avoids quite a few of the Go problems).
  - neilwilson 8y ago
    chroot, cgroup, and the systemd run facilities (systemd-nspawn for example).
    - cyphar 8y ago
      pivot_root, not chroot. chroot is vulnerable to many trivial escapes.

amscanne 8y ago

I appreciate the novel approach, but you should try to use more realistic numbers for the arguments in the post. A node process doing nothing does not consume 30MB of private memory (shared mappings are shared, just like the "shared runtime"), nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first and almost 2 orders on the second. Cold starts are a symptom of something else (likely shipping function data), which I don't think is really a property of the sandbox.

IanCal 8y ago
> nor does it take 500ms-10s to start a process or 100us to context switch. You're off by 3+ orders of magnitude on the first I'm relatively sure 10ms is not the worst case cold start time of lambda. > Cold starts are a symptom of something else (likely shipping function data) Which is all part of spinning up a containerised function in lambda. About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim.
- cyphar 8y ago
  > About memory, despite being shared memory, does a container in lambda running a node process use 35mb of RAM? Because that's the claim. What do you mean by "use" in this context? Does every program that links to glibc "use" the memory for glibc? (The answer is no, because the page cache is shared. But if you just looked at memory usage you might be fooled into thinking it is copied for every program.) Containers use a drastically tiny amount of in-kernel memory. We are talking fewer than a couple of kilobytes in the worst case. It appears to me that the main worry was with using Kubernetes, and they've applied this to containers as a whole. I agree that in this sort of usecase Kubernetes really doesn't make sense (using container images at all makes no sense in this case), but the core kernel primitives and simple container runtimes available are more than suitable. There is an argument that having no userland-kernel context switches has a performance improvement (and it does) but I would like to see data before accepting at face value that "context switches are expensive" is an appropriate thing to optimise for. After all they are still doing context switches, it's just in userspace. And if v8 is multi-threaded they've just reinvented the N-to-M scheduling model that is very well known to be fundamentally broken because of various well known pathologies.

cyphar 8y ago

> we can host 10,000+ tenants on one machine -- something that's totally unrealistic with containers or VMs. This is definitely possible with containers. Docker has historically had density problems for a variety of reasons (its architecture as well as Go being awful for systems programming in this context). But you can definitely get this density with a simple container runtime that doesn't have large long running processes. Saying you can't run 10k containers is like saying you can't run 10k processes on Linux. I'm sure that it might be somewhat faster to have it supported in V8 directly, but I don't want people to think such density is impossible with containers. (And with page-cache sharing you'd be able to not have to load 10k V8 programs into memory as well.)

hamandcheese 8y ago
I think you are missing the point. Regardless of the number of processes you can run, the overhead of starting them and context switching between them is dramatically higher than the equivalent operations within a single V8 process.
- cyphar 8y ago
  The part of the comment I was responding to was talking about it being impractical to have that many containers running, which isn't true. Later I mentioned that there are probably other benefits (having everything in one process does remove the need for TLB flushes and so on), but that isn't the only thing they said. They didn't say it's drastically faster, they said it's "unrealistic" which simply isn't true. Also things like page cache sharing would make a huge difference to the real memory cost of having many processes. When you run 10k programs you don't have 10k copies of glibc.
bambataa 8y ago
Out of interest, why do you say that Go is awful for systems programming in this context?

ec109685 8y ago

How do you protect two isolates from interfearing with one another if they land in same process? If two of them happen to both start a calculation that takes 50ms of cpu time, do both calculations end up taking 100ms to complete? Or do you prevent concurrent execution some how?

shaklee3 8y ago

Being a non-js person, it wasn't clear if this is just for JavaScript or not. If it is, then it's not really fair to compare against containers or vms, right?

rickycook 8y ago
that was my thought as well. in the section about the down sides, they do mention though that they should be able to run anything that compiles to web assembly. they specifically call out rust and go. wasm really has quite a lot of interesting and unexpected applications! (though of course non-wasm tech can accomplish the same thing, but it’s nice that we are getting close to some form of universal binary format)

rodgerd 8y ago

So hyperthreading and multicores are a security mess, but a JavaScript VM is going to safely allow "10,000 tenants" on a single machine? Sure.

zackbloom 8y ago
I think it's very reasonable that the best possible security is not implemented in microcode.
alexnewman 8y ago
Hyperthread, multicores, rings, were all stability boundries hijacked to be security boundries. It'll never work

rawoke083600 8y ago

Well done! Very creative solution to the problem and using the V8 system as a starting point is brilliant!

zeckalpha 8y ago

The marketing around “No VM” is interesting. It sounds like you’re relying on V8. What did you do instead of using its VM? Precompile? Or did you reimplement V8s isolates in another VM-less system? Or do you mean more specifically no OS-level VM?

mwcampbell 8y ago

> The idea that you can have a stack of cloud services that depend on each other without incurring any latency or bandwidth costs will, I think, be game-changing. This looks to me like just another form of vendor lock-in.

espeed 8y ago

Does your implementation support vectorized SIMD or SSE instructions?

vicaya 8y ago

Running 10k containers on one machine is totally realistic though. This is just yet another language vm (e.g. jvm) isolation all over again.