8 ms·

Isolates, microVMs, and WebAssembly

123 points by charliermarsh 4y ago

rektide 4y ago
On Micro-VM's specifically, shout out to Wyrcan[1], which is open source software similar-ish to Fly.io in taking a container image & booting into it. There's no platform here, but it's an advanced & secure bootloader for a container images that looks like the core critical capability furthest under the hood. Back to isolates, I'd really love to see v8 isolates gain some more whatever it takes for people to be less critical about it's multitenancy. I havent really understood what the criticism is but it's fairly active. Since the process has to do it's owm scheduling of work, just having something like cgroups & resource priority seems like a fairly obvious absense: make sure everyone gets a turn. This scheduling seems semi obvious. But I think the security-minded folk are paniced over a lot more, and likely with legitacy, but the name isolates is afaik somewhat reasonably truthful, that data is fairly secure across isolates in the same process. Throwing in a bonus ask, it'd be sweet if isolates could migrated. Perhaps snapshotting can already maybe do this? Being able to load manage is important! Even if v8 doesnt want to have vast multi-tenant scheduling capabilities my trivial dumb feel is that just moving aggro processes elsewhere would be a great start to handle aggressive tenant sub-proceeses. I wish I had links on hand, but one of the things that most opened my horizons on wasm & it's role was considerations that browsers should be able to have multiple instances of a module. Like, if someone depends on a module, does everyone always get the one singleton instance? Can the browser start creating multiple instances if there's a lot of consumers? Very clear browserside question that really opened the floodgates that, oh, there's a lot of ways we could go forward with this! [1] https://gitlab.com/wyrcan/wyrcan https://gitlab.com/wyrcan/wyrcan
pyrolistical 4y ago
The future is here, people just haven’t noticed yet
muricula 4y ago
The article says whether a language runtime like v8 is a stronger security boundary than a hypervisor is a matter of debate, however v8 has boatloads of CVEs and the further isolation of browser sandboxes has been a driving factor in OS development for nearly two decades. Meanwhile, multiple Fortune 500 companies bet their multi-billion dollar cloud businesses on the security of their hypervisors. So when the author links to a confused HN thread about whether the v8 runtime is a security boundary and says "looks like there's debate", it makes the article look like a joke.
- 0x457 4y ago
  Well, V8 is a thing that writes and then executes memory (JIT), no wonder it had issues.
- charliermarsh 4y ago
  This feels a bit harsh. The question isn't whether V8 is a stronger security boundary than a hypervisor, it's whether Cloudflare's use of V8 demands process isolation. And the "confused HN thread" is Kenton Varda debating it with Kurt Mackey! But I'll try to make it clearer in the post.
  - eyelidlessness 4y ago
    The question is pretty definitively answered unless or until something changes in Google’s assessment of V8: if they don’t consider it safe for process multitenancy, which they have every incentive to do, it’s not safe. There are use cases I can imagine (and have imagined, I have no bandwidth to pursue them) which would probably be fine. But a multi tenant server isn’t one of them, and I’m still shocked, even at Cloudflare scale, that it’s a consideration.
    - butthatsit 4y ago
      Is Google using V8 on their own hardware paid for with their own money? I thought they're not using it on GCP for anything. What's the incentive for them to move to process multitenancy? They can simply (ab)use that computers are getting faster, the end users are not going to complain as visibly as losing some money on a cloud platform would be.
    - kentonv 4y ago
      > if they don’t consider it safe for process multitenancy, which they have every incentive to do, it’s not safe. It's more nuanced than that. Unfortunately security is not a boolean thing, where it's either secure or it's not secure. V8 is designed to be a secure sandbox on its own. But security is actually about risk management. V8 is complicated, and it has bugs. It tends to have more bugs than a typical hypervisor does. So it's more risky to rely on it as a secure sandbox, unless you can find risk mitigations to layer on top of it. After a decade of relying on V8 alone to isolate frames from each other (within a tab), Chrome chose to add strict process isolation as a second security layer. But Chrome definitely doesn't think of that second layer as being the "secure" one -- security comes from having both together, so an attack has to bypass both at the same time. Google still pays big bug bounties for V8 breakouts.
combyn8tor 4y ago
Does anyone know what Firebase uses for their functions service?
- kennethh 4y ago
  I presume this is the same as Google Cloud functions which use gVisor as mentioned in the article.
hardwaresofton 4y ago
Towards the end of the article it sounds like the author was looking for this: https://github.com/deislabs/wagi https://github.com/deislabs/wagi
- ignoramous 4y ago
  Microsoft Deis Labs folks are the ones who are running the show at Fermyon, which is repeatedly spoken about in the article.
  - hardwaresofton 4y ago
    Yup, but they have a lot of different awesome projects on the fire, it can be hard to know which one to use: - https://github.com/deislabs/runwasi https://github.com/deislabs/runwasi - https://github.com/deislabs/hippo https://github.com/deislabs/hippo - https://github.com/deislabs/wagi https://github.com/deislabs/wagi - https://github.com/deislabs/krustlet https://github.com/deislabs/krustlet The description at the end of the article sounds like CGI, on WASM, which is what WAGI is.
tptacek 4y ago
I'm a little fuzzy on the multitenant security promise of WebAssembly. I haven't dug deeply into it. It seems though that it can be asymptotically as secure as the host system wrapper you build around it: that is, the bugs won't be in the WebAssembly but in the bridge between WebAssembly and the host OS. This is approximately the same situation as with v8 isolates, except that we have reasons to believe that WASM has a more trustworthy and coherent design than v8 isolates, so we're not worried about things like memory corruption breaking boundaries between cotenant WASM programs running in the same process. At the end of the day, if your runtime relies on a whole shared OS kernel, you have to be concerned about the bugs in the kernel. That's true of VMs as well, but to a much more limited extent: KVM is (by definition) much smaller than the whole kernel, and KVM bugs are rare. I'm writing this mostly as a provocation; I don't have a clear enough understanding of backend WASM multitenant security to have strongly-held opinions about it.
- syrusakbary 4y ago
  While I mostly agree with you, let me play the devil a bit (just for fun!) Let's agree with the base that (probably) most of the bugs will happen within the host defined imports (aka bugs in the systemcalls/kernel) vs bugs on the Wasm runtime per se. In that front, Wasm host system call implementations will offer not that much advantage compared to other virtualization approaches such as Firecracker. However, those advantages are only perceived when you run software that's already fully containerized (in a OCI-like container). But here's the nit: there's many more software that is not containerized (for example: 3rd party libraries). Running this libraries in Wasm can actually be more secure than running the native ones (since we are bringing Firecracker-like security to the systemcalls the library may use). Specially, when running this software locally. So, in my view, Wasm forces most of the software to be secure, regardless of its kind (library or application/OCI-container). It's a small thing that might make a big difference!
- brundolf 4y ago
  > This is approximately the same situation as with v8 isolates, except that we have reasons to believe that WASM has a more trustworthy and coherent design than v8 isolates, so we're not worried about things like memory corruption breaking boundaries between cotenant WASM programs running in the same process My understanding is that WASM's spec isn't just more coherent than JavaScript's, but also orders of magnitude smaller, which means less surface-area to check for vulnerabilities
deleted 4y ago
[deleted]
deleted 4y ago
[deleted]
syrusakbary 4y ago
> Most other companies (like Wasmer) seem to be focused on runtimes Stay tuned. Exciting things coming on the Wasmer horizon :)
simonw 4y ago
This article says of Fly: "but they don’t scale down to zero". This isn't entirely true. A little known aspect of Fly Machines is that you can stop them and they will be automatically restarted by the Fly router (with a very short cold-start pause) when the next request arrives. So you can scale them to zero if you do a bit of extra work. https://fly.io/docs/reference/machines/ https://fly.io/docs/reference/machines/ hints at this: > Machines are also the spawning ground for new platform features like wake-on-request (also known as scale-to-zero). You can stop a running machine to save on compute costs. It then may be started automatically when a request arrives at the Fly proxy.
- ignoramous 4y ago
  I'm already experimenting moving workloads that I run on Workers to Fly Machines (because HTTP-only nature of Workers is limiting, and TCP Workers are no where in sight), and I like what I see. That said: - Fly is missing HTTP/3 (QUIC) support (and it doesn't look like its going to be easy for them to simply bolt it on). Cloudflare is quick to deploy newer standards (even when experimental) like they did with HTTP/3 and ECH. - Sometimes Fly Machines start-up, but don't get sent any request from the Fly router. - Workers have free cache (up to 512M per obj), which essentially functions like a free blob store. On Fly, 10G disks spin up alongside Machines, but those are wiped every start/stop cycle. Whenever Fly Machines could be paused/resumed (with state intact)-- likely in near future-- things will start getting really interesting. - Plenty other rough (but livable) edges on Fly with Machines (that's okay, given it is still in preview). With Workers, Cloudflare has a very stable and highly optimized offering, I doubt established cloud providers can match that in the short-term, let alone upstarts. While Fly Machines (incredibly) boots my NodeJS app up in 200ms to 500ms, it is no where close to the 0ms to 5ms start-time on Workers. That said, for an unrelated workload where I run OLTP queries on-demand on DuckDB from parquet files stored in R2, I am definitely moving over to Fly Machines from AWS AppRunner (and this workload I can never run on Workers because just the DuckDB binary for WASM goes beyond Workers' 1MB limit). To me, Fly Machines are more of an AWS Fargate / AWS Lambda alternative than Workers, at this point in time.
pclmulqdq 4y ago
I suspect that WebAssembly is actually the wrong abstraction here, and you might want something a little closer to BPF for non-process-isolated serverless compute, but also not BPF because BPF can do arbitrary memory reads. WebAssembly appears to be too permissive for this case, and BPF too restrictive to be useful. Ideally, if you could have a language-level VM that guarantees that you only ever read or write data inside memory allocated to you and calls a limited set of syscalls (which you intercept/rewrite at compile time), you probably don't need any other protection.
- kaba0 4y ago
  Sounds like what Singularity and later Midori tried to achieve: http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/
- kevingadd 4y ago
  You still need a way to protect the data inside the VM's memory from getting trampled, since the data could contain the spiritual equivalent of function pointers and those get passed out to the syscalls. Or at a simpler level, if gmail is running in a wasm sandbox, I still don't want an attacker corrupting gmail's heap and causing it to forward my entire inbox to the attacker even if my OS is safe.
  - pclmulqdq 4y ago
    There are ways to have a verifier check to make sure that this kind of code modification doesn't happen. It's a very hard problem, though, and maybe not suitable for things like Javascript.
    - kevingadd 4y ago
      What I'm referring to is not code modification, but heap corruption. The simplest example would be that if the to: and bcc: list for your email live in a wasm heap, any bug in that wasm application could be used to redirect your email. The same is true for JavaScript, C#, etc but the key difference is that those are memory-safe languages under normal circumstances, while WASM is only "memory safe" in that the memory accesses are contained to its sandbox (and part of the stack is outside of the heap and protected from stack overflows, like the return address). Everything else is still wide open for corruption and is easier to manipulate due to predictable function pointers. The wasm shadow stack is a great idea for mitigating various attacks against C but since you can only put ints and floats on that stack and you can't pass values by reference, the vast majority of applications have a bunch of critical stack data stored in the heap at a fixed address next to non-critical data and a stray memcpy/strcpy can trample a function pointer. Function #37 will always be the same function and its arguments are probably being loaded from nearby locals on the stack. If support for putting more complex types in arguments and locals ever ships in wasm, it would make it possible to mitigate a lot of this. I'm not sure whether that's in the cards, though.
hardwaregeek 4y ago
So far in my experience WebAssembly has been great for when I want to import a *little* bit of native code into my browser. I've yet to see it work well as a full on container based solution. I could certainly see a future where that is true, but it'd need the following: - A way way better tooling story. Emscripten is an incredibly fiddly tool that requires a lot of flags and config. wasm-pack is decent for Rust but seems to be focused on the browser. - Better interoperability. WASM works great if you write everything in Rust, or everything in C or everything in Zig. Not so much with multiple languages. - Memory64 and WASI need to land. We need more than 4 gigs and we need proper syscalls. - Near native needs to be, well, near native. I find it weird that people keep on saying WebAssembly is fast while in the same text decrying "a low-double-digits percentage hit" in performance. I'm fairly certain WebAssembly's "near native" is using near to mean "same order of magnitude", not "within 10%". I'm also just confused as to what WebAssembly will look like for something that's more than a lambda conceptually. Like if we're running a classic back-end server with a database, are we going to run the database in wasm? Are we running it in a different wasm process? How do you share memory? Is wasm going to reinvent the operating system? I like WebAssembly as an idea but I'm very unclear on how it'll get to the "replace Docker" level.
- jdmichaud 4y ago
  > - A way way better tooling story. I think this is a real paint point of WebAssembly but it's getting better. You can now compile C/C++ code to WASM using clang as it became a first class citizen in v8 (or later?). Of course, clang does not provide all the runtime goodies that emscripten does but if you limit yourself to use WASM for some part of your code that require performance then all you need to provide is a few Javascript glue code. For prototyping purposes I've bundle an npm package that bring clang to your frontend project without the need to install it through a package [0]. Chrome debugger now support WASM [1] but you still need to install an extension. I've started to work on a way to convert the DWARF symbols from the WASM binary into sourcemap on the fly [2][3] but it still rough around the edge and need some work but it can definitely be done. This would allow WASM debugging natively in any browser supporting sourcemaps. You also have all the WASM binary tools that you would find for any other kind of executable format in WABT [4]. [0]: https://www.npmjs.com/package/@jdmichaud/wasm-toolkit https://www.npmjs.com/package/@jdmichaud/wasm-toolkit [1]: https://developer.chrome.com/blog/wasm-debugging-2020/ https://developer.chrome.com/blog/wasm-debugging-2020/ [2]: https://github.com/jdmichaud/dwarf-2-sourcemap https://github.com/jdmichaud/dwarf-2-sourcemap [3]: http://site.novidee.com/blog/blog-entry.html?article=20220821-how-to-debug-wasm-in-firefox.html http://site.novidee.com/blog/blog-entry.html?article=2022082... [4]: https://github.com/WebAssembly/wabt https://github.com/WebAssembly/wabt
pjmlp 4y ago
It is so tragically funny whatching everyone re-inventing mainframe language environments, JVM application servers and the CLR. > > One of the exciting things in Visual Studio .NET is its language agnosticism. If a vendor has written a .NET-compliant language, you can use it in Visual Studio .NET. It'll work just as well as C# or C++ or Visual Basic. This isn't just a future feature-in-planning. There are already nearly two dozen languages being developed for Visual Studio .NET: Visual Basic, C#, C++, JScript, APL, Cobol, Eiffel, Fortran, Pascal, Perl, Python, RPG, Smalltalk, Oberon, Component Pascal, Haskell/Mondrian, Scheme, Mercury, Alice, and even the Java language. -- February 2002 issue of MSDN Magazine https://learn.microsoft.com/en-us/archive/msdn-magazine/2002/february/editor-s-note-welcome-visual-studio-net https://learn.microsoft.com/en-us/archive/msdn-magazine/2002... It is as if a newer generation never bothered to learn about what came before.
- spoiler 4y ago
  > It is as if a newer generation never bothered to learn about what came before. Why the pessimistic take? Maybe the newer generations improved upon what came before!
  - pjmlp 4y ago
    Sure, but then don't talk about the "new" stuff as if it was the best invention since sliced bread, without any clue that Egypitans, Babylons, Romans and ancient Greeks were already having it.
  - kaba0 4y ago
    All too often they didn’t even know about these systems, at least in my (limited) experience. Not necessarily true here, but it’s not like NIH/reinventing the wheel isn’t common in certain parts of the industry.
    - flohofwoe 4y ago
      AFAIK the most important difference between WASM and other VMs on the level of CLR or JVM is that WASM has a simple linear memory model, which allows languages which expect a linear heap (like C, C++, Rust, ...) to be supported by just adding new backends to existing compilers (and most importantly without requiring changes to the language). The CLR and JVM might be able to simulate a linear heap through a byte array (just like asm.js did), but it's questionable whether this allows the same performance as WASM when the whole VM instruction set is built around high level object references.
hestefisk 4y ago
The JS / wasm ecosystem is slowly becoming Java / JVM. Wasm has a virtual machine that can run wasm code, which different languages can target with a specialised compiler. It also abstracts away things like memory, storage and networking, just like EJB did.
ex3ndr 4y ago
We can have a better js vm like quickjs or Hermes (from react native).
torginus 4y ago
What is it with these companies who have untold resources, and scores of the smartest engineers, yet what they do is repurpose some existing tech that kinda works instead of building stuff properly? Like selling: - The browser scripting engine as a PaaS engine - The VM language/runtime meant for running binaries in the browser as a PaaS engine - The tooling meant to isolate Linux processes as a VM platform Can't they build tech properly, for their own purposes? Not to mention, the whole exercise seems like monkey-patching over the failures of modern OS design. A proper operating system should be able to run processes with zero trust and resource sandboxing.
- mwcampbell 4y ago
  One thing I learned from being on Microsoft's Windows accessibility team at its peak headcount is that it doesn't matter how big you are; there are always limited resources and competing priorities. By settling for "good enough" on a purely technical level you can focus more resources on things that are more important in the big (i.e. non-technical) picture (edit: like getting the thing shipped at all).

123 points by charliermarsh 4y ago

On Micro-VM's specifically, shout out to Wyrcan[1], which is open source software similar-ish to Fly.io in taking a container image & booting into it. There's no platform here, but it's an advanced & secure bootloader for a container images that looks like the core critical capability furthest under the hood. Back to isolates, I'd really love to see v8 isolates gain some more whatever it takes for people to be less critical about it's multitenancy. I havent really understood what the criticism is but it's fairly active. Since the process has to do it's owm scheduling of work, just having something like cgroups & resource priority seems like a fairly obvious absense: make sure everyone gets a turn. This scheduling seems semi obvious. But I think the security-minded folk are paniced over a lot more, and likely with legitacy, but the name isolates is afaik somewhat reasonably truthful, that data is fairly secure across isolates in the same process. Throwing in a bonus ask, it'd be sweet if isolates could migrated. Perhaps snapshotting can already maybe do this? Being able to load manage is important! Even if v8 doesnt want to have vast multi-tenant scheduling capabilities my trivial dumb feel is that just moving aggro processes elsewhere would be a great start to handle aggressive tenant sub-proceeses. I wish I had links on hand, but one of the things that most opened my horizons on wasm & it's role was considerations that browsers should be able to have multiple instances of a module. Like, if someone depends on a module, does everyone always get the one singleton instance? Can the browser start creating multiple instances if there's a lot of consumers? Very clear browserside question that really opened the floodgates that, oh, there's a lot of ways we could go forward with this! [1] https://gitlab.com/wyrcan/wyrcan https://gitlab.com/wyrcan/wyrcan

pyrolistical 4y ago

The future is here, people just haven’t noticed yet

muricula 4y ago

The article says whether a language runtime like v8 is a stronger security boundary than a hypervisor is a matter of debate, however v8 has boatloads of CVEs and the further isolation of browser sandboxes has been a driving factor in OS development for nearly two decades. Meanwhile, multiple Fortune 500 companies bet their multi-billion dollar cloud businesses on the security of their hypervisors. So when the author links to a confused HN thread about whether the v8 runtime is a security boundary and says "looks like there's debate", it makes the article look like a joke.

0x457 4y ago
Well, V8 is a thing that writes and then executes memory (JIT), no wonder it had issues.
charliermarsh 4y ago
This feels a bit harsh. The question isn't whether V8 is a stronger security boundary than a hypervisor, it's whether Cloudflare's use of V8 demands process isolation. And the "confused HN thread" is Kenton Varda debating it with Kurt Mackey! But I'll try to make it clearer in the post.
- eyelidlessness 4y ago
  The question is pretty definitively answered unless or until something changes in Google’s assessment of V8: if they don’t consider it safe for process multitenancy, which they have every incentive to do, it’s not safe. There are use cases I can imagine (and have imagined, I have no bandwidth to pursue them) which would probably be fine. But a multi tenant server isn’t one of them, and I’m still shocked, even at Cloudflare scale, that it’s a consideration.
  - butthatsit 4y ago
    Is Google using V8 on their own hardware paid for with their own money? I thought they're not using it on GCP for anything. What's the incentive for them to move to process multitenancy? They can simply (ab)use that computers are getting faster, the end users are not going to complain as visibly as losing some money on a cloud platform would be.
  - kentonv 4y ago
    > if they don’t consider it safe for process multitenancy, which they have every incentive to do, it’s not safe. It's more nuanced than that. Unfortunately security is not a boolean thing, where it's either secure or it's not secure. V8 is designed to be a secure sandbox on its own. But security is actually about risk management. V8 is complicated, and it has bugs. It tends to have more bugs than a typical hypervisor does. So it's more risky to rely on it as a secure sandbox, unless you can find risk mitigations to layer on top of it. After a decade of relying on V8 alone to isolate frames from each other (within a tab), Chrome chose to add strict process isolation as a second security layer. But Chrome definitely doesn't think of that second layer as being the "secure" one -- security comes from having both together, so an attack has to bypass both at the same time. Google still pays big bug bounties for V8 breakouts.

combyn8tor 4y ago

Does anyone know what Firebase uses for their functions service?

kennethh 4y ago
I presume this is the same as Google Cloud functions which use gVisor as mentioned in the article.

hardwaresofton 4y ago

Towards the end of the article it sounds like the author was looking for this: https://github.com/deislabs/wagi https://github.com/deislabs/wagi

ignoramous 4y ago
Microsoft Deis Labs folks are the ones who are running the show at Fermyon, which is repeatedly spoken about in the article.
- hardwaresofton 4y ago
  Yup, but they have a lot of different awesome projects on the fire, it can be hard to know which one to use: - https://github.com/deislabs/runwasi https://github.com/deislabs/runwasi - https://github.com/deislabs/hippo https://github.com/deislabs/hippo - https://github.com/deislabs/wagi https://github.com/deislabs/wagi - https://github.com/deislabs/krustlet https://github.com/deislabs/krustlet The description at the end of the article sounds like CGI, on WASM, which is what WAGI is.

tptacek 4y ago

I'm a little fuzzy on the multitenant security promise of WebAssembly. I haven't dug deeply into it. It seems though that it can be asymptotically as secure as the host system wrapper you build around it: that is, the bugs won't be in the WebAssembly but in the bridge between WebAssembly and the host OS. This is approximately the same situation as with v8 isolates, except that we have reasons to believe that WASM has a more trustworthy and coherent design than v8 isolates, so we're not worried about things like memory corruption breaking boundaries between cotenant WASM programs running in the same process. At the end of the day, if your runtime relies on a whole shared OS kernel, you have to be concerned about the bugs in the kernel. That's true of VMs as well, but to a much more limited extent: KVM is (by definition) much smaller than the whole kernel, and KVM bugs are rare. I'm writing this mostly as a provocation; I don't have a clear enough understanding of backend WASM multitenant security to have strongly-held opinions about it.

syrusakbary 4y ago
While I mostly agree with you, let me play the devil a bit (just for fun!) Let's agree with the base that (probably) most of the bugs will happen within the host defined imports (aka bugs in the systemcalls/kernel) vs bugs on the Wasm runtime per se. In that front, Wasm host system call implementations will offer not that much advantage compared to other virtualization approaches such as Firecracker. However, those advantages are only perceived when you run software that's already fully containerized (in a OCI-like container). But here's the nit: there's many more software that is not containerized (for example: 3rd party libraries). Running this libraries in Wasm can actually be more secure than running the native ones (since we are bringing Firecracker-like security to the systemcalls the library may use). Specially, when running this software locally. So, in my view, Wasm forces most of the software to be secure, regardless of its kind (library or application/OCI-container). It's a small thing that might make a big difference!
brundolf 4y ago
> This is approximately the same situation as with v8 isolates, except that we have reasons to believe that WASM has a more trustworthy and coherent design than v8 isolates, so we're not worried about things like memory corruption breaking boundaries between cotenant WASM programs running in the same process My understanding is that WASM's spec isn't just more coherent than JavaScript's, but also orders of magnitude smaller, which means less surface-area to check for vulnerabilities

deleted 4y ago

[deleted]

deleted 4y ago

[deleted]

syrusakbary 4y ago

> Most other companies (like Wasmer) seem to be focused on runtimes Stay tuned. Exciting things coming on the Wasmer horizon :)

simonw 4y ago

This article says of Fly: "but they don’t scale down to zero". This isn't entirely true. A little known aspect of Fly Machines is that you can stop them and they will be automatically restarted by the Fly router (with a very short cold-start pause) when the next request arrives. So you can scale them to zero if you do a bit of extra work. https://fly.io/docs/reference/machines/ https://fly.io/docs/reference/machines/ hints at this: > Machines are also the spawning ground for new platform features like wake-on-request (also known as scale-to-zero). You can stop a running machine to save on compute costs. It then may be started automatically when a request arrives at the Fly proxy.

ignoramous 4y ago
I'm already experimenting moving workloads that I run on Workers to Fly Machines (because HTTP-only nature of Workers is limiting, and TCP Workers are no where in sight), and I like what I see. That said: - Fly is missing HTTP/3 (QUIC) support (and it doesn't look like its going to be easy for them to simply bolt it on). Cloudflare is quick to deploy newer standards (even when experimental) like they did with HTTP/3 and ECH. - Sometimes Fly Machines start-up, but don't get sent any request from the Fly router. - Workers have free cache (up to 512M per obj), which essentially functions like a free blob store. On Fly, 10G disks spin up alongside Machines, but those are wiped every start/stop cycle. Whenever Fly Machines could be paused/resumed (with state intact)-- likely in near future-- things will start getting really interesting. - Plenty other rough (but livable) edges on Fly with Machines (that's okay, given it is still in preview). With Workers, Cloudflare has a very stable and highly optimized offering, I doubt established cloud providers can match that in the short-term, let alone upstarts. While Fly Machines (incredibly) boots my NodeJS app up in 200ms to 500ms, it is no where close to the 0ms to 5ms start-time on Workers. That said, for an unrelated workload where I run OLTP queries on-demand on DuckDB from parquet files stored in R2, I am definitely moving over to Fly Machines from AWS AppRunner (and this workload I can never run on Workers because just the DuckDB binary for WASM goes beyond Workers' 1MB limit). To me, Fly Machines are more of an AWS Fargate / AWS Lambda alternative than Workers, at this point in time.

pclmulqdq 4y ago

I suspect that WebAssembly is actually the wrong abstraction here, and you might want something a little closer to BPF for non-process-isolated serverless compute, but also not BPF because BPF can do arbitrary memory reads. WebAssembly appears to be too permissive for this case, and BPF too restrictive to be useful. Ideally, if you could have a language-level VM that guarantees that you only ever read or write data inside memory allocated to you and calls a limited set of syscalls (which you intercept/rewrite at compile time), you probably don't need any other protection.

kaba0 4y ago
Sounds like what Singularity and later Midori tried to achieve: http://joeduffyblog.com/2015/11/03/blogging-about-midori/ http://joeduffyblog.com/2015/11/03/blogging-about-midori/
kevingadd 4y ago
You still need a way to protect the data inside the VM's memory from getting trampled, since the data could contain the spiritual equivalent of function pointers and those get passed out to the syscalls. Or at a simpler level, if gmail is running in a wasm sandbox, I still don't want an attacker corrupting gmail's heap and causing it to forward my entire inbox to the attacker even if my OS is safe.
- pclmulqdq 4y ago
  There are ways to have a verifier check to make sure that this kind of code modification doesn't happen. It's a very hard problem, though, and maybe not suitable for things like Javascript.
  - kevingadd 4y ago
    What I'm referring to is not code modification, but heap corruption. The simplest example would be that if the to: and bcc: list for your email live in a wasm heap, any bug in that wasm application could be used to redirect your email. The same is true for JavaScript, C#, etc but the key difference is that those are memory-safe languages under normal circumstances, while WASM is only "memory safe" in that the memory accesses are contained to its sandbox (and part of the stack is outside of the heap and protected from stack overflows, like the return address). Everything else is still wide open for corruption and is easier to manipulate due to predictable function pointers. The wasm shadow stack is a great idea for mitigating various attacks against C but since you can only put ints and floats on that stack and you can't pass values by reference, the vast majority of applications have a bunch of critical stack data stored in the heap at a fixed address next to non-critical data and a stray memcpy/strcpy can trample a function pointer. Function #37 will always be the same function and its arguments are probably being loaded from nearby locals on the stack. If support for putting more complex types in arguments and locals ever ships in wasm, it would make it possible to mitigate a lot of this. I'm not sure whether that's in the cards, though.

hardwaregeek 4y ago

So far in my experience WebAssembly has been great for when I want to import a *little* bit of native code into my browser. I've yet to see it work well as a full on container based solution. I could certainly see a future where that is true, but it'd need the following: - A way way better tooling story. Emscripten is an incredibly fiddly tool that requires a lot of flags and config. wasm-pack is decent for Rust but seems to be focused on the browser. - Better interoperability. WASM works great if you write everything in Rust, or everything in C or everything in Zig. Not so much with multiple languages. - Memory64 and WASI need to land. We need more than 4 gigs and we need proper syscalls. - Near native needs to be, well, near native. I find it weird that people keep on saying WebAssembly is fast while in the same text decrying "a low-double-digits percentage hit" in performance. I'm fairly certain WebAssembly's "near native" is using near to mean "same order of magnitude", not "within 10%". I'm also just confused as to what WebAssembly will look like for something that's more than a lambda conceptually. Like if we're running a classic back-end server with a database, are we going to run the database in wasm? Are we running it in a different wasm process? How do you share memory? Is wasm going to reinvent the operating system? I like WebAssembly as an idea but I'm very unclear on how it'll get to the "replace Docker" level.

jdmichaud 4y ago
> - A way way better tooling story. I think this is a real paint point of WebAssembly but it's getting better. You can now compile C/C++ code to WASM using clang as it became a first class citizen in v8 (or later?). Of course, clang does not provide all the runtime goodies that emscripten does but if you limit yourself to use WASM for some part of your code that require performance then all you need to provide is a few Javascript glue code. For prototyping purposes I've bundle an npm package that bring clang to your frontend project without the need to install it through a package [0]. Chrome debugger now support WASM [1] but you still need to install an extension. I've started to work on a way to convert the DWARF symbols from the WASM binary into sourcemap on the fly [2][3] but it still rough around the edge and need some work but it can definitely be done. This would allow WASM debugging natively in any browser supporting sourcemaps. You also have all the WASM binary tools that you would find for any other kind of executable format in WABT [4]. [0]: https://www.npmjs.com/package/@jdmichaud/wasm-toolkit https://www.npmjs.com/package/@jdmichaud/wasm-toolkit [1]: https://developer.chrome.com/blog/wasm-debugging-2020/ https://developer.chrome.com/blog/wasm-debugging-2020/ [2]: https://github.com/jdmichaud/dwarf-2-sourcemap https://github.com/jdmichaud/dwarf-2-sourcemap [3]: http://site.novidee.com/blog/blog-entry.html?article=20220821-how-to-debug-wasm-in-firefox.html http://site.novidee.com/blog/blog-entry.html?article=2022082... [4]: https://github.com/WebAssembly/wabt https://github.com/WebAssembly/wabt

pjmlp 4y ago

It is so tragically funny whatching everyone re-inventing mainframe language environments, JVM application servers and the CLR. > > One of the exciting things in Visual Studio .NET is its language agnosticism. If a vendor has written a .NET-compliant language, you can use it in Visual Studio .NET. It'll work just as well as C# or C++ or Visual Basic. This isn't just a future feature-in-planning. There are already nearly two dozen languages being developed for Visual Studio .NET: Visual Basic, C#, C++, JScript, APL, Cobol, Eiffel, Fortran, Pascal, Perl, Python, RPG, Smalltalk, Oberon, Component Pascal, Haskell/Mondrian, Scheme, Mercury, Alice, and even the Java language. -- February 2002 issue of MSDN Magazine https://learn.microsoft.com/en-us/archive/msdn-magazine/2002/february/editor-s-note-welcome-visual-studio-net https://learn.microsoft.com/en-us/archive/msdn-magazine/2002... It is as if a newer generation never bothered to learn about what came before.

spoiler 4y ago
> It is as if a newer generation never bothered to learn about what came before. Why the pessimistic take? Maybe the newer generations improved upon what came before!
- pjmlp 4y ago
  Sure, but then don't talk about the "new" stuff as if it was the best invention since sliced bread, without any clue that Egypitans, Babylons, Romans and ancient Greeks were already having it.
- kaba0 4y ago
  All too often they didn’t even know about these systems, at least in my (limited) experience. Not necessarily true here, but it’s not like NIH/reinventing the wheel isn’t common in certain parts of the industry.
  - flohofwoe 4y ago
    AFAIK the most important difference between WASM and other VMs on the level of CLR or JVM is that WASM has a simple linear memory model, which allows languages which expect a linear heap (like C, C++, Rust, ...) to be supported by just adding new backends to existing compilers (and most importantly without requiring changes to the language). The CLR and JVM might be able to simulate a linear heap through a byte array (just like asm.js did), but it's questionable whether this allows the same performance as WASM when the whole VM instruction set is built around high level object references.

hestefisk 4y ago

The JS / wasm ecosystem is slowly becoming Java / JVM. Wasm has a virtual machine that can run wasm code, which different languages can target with a specialised compiler. It also abstracts away things like memory, storage and networking, just like EJB did.

ex3ndr 4y ago

We can have a better js vm like quickjs or Hermes (from react native).

torginus 4y ago

What is it with these companies who have untold resources, and scores of the smartest engineers, yet what they do is repurpose some existing tech that kinda works instead of building stuff properly? Like selling: - The browser scripting engine as a PaaS engine - The VM language/runtime meant for running binaries in the browser as a PaaS engine - The tooling meant to isolate Linux processes as a VM platform Can't they build tech properly, for their own purposes? Not to mention, the whole exercise seems like monkey-patching over the failures of modern OS design. A proper operating system should be able to run processes with zero trust and resource sandboxing.

mwcampbell 4y ago
One thing I learned from being on Microsoft's Windows accessibility team at its peak headcount is that it doesn't matter how big you are; there are always limited resources and competing priorities. By settling for "good enough" on a purely technical level you can focus more resources on things that are more important in the big (i.e. non-technical) picture (edit: like getting the thing shipped at all).