4 ms·

So presumably this will also open up avenues for doing QUIC and thus HTTP/3 on Fly?

mrkurt 6y ago
Yep! We have a "Firecracker that accepts QUIC" running with this. People usually want HTTP + TLS handled for them, though. So when we ship QUIC + HTTP3 as a first class feature, we'll terminate QUIC and give people whatever their app process can accept.
- dochtman 6y ago
  Any insight into what QUIC/H3 stack you'll be using for the proxy?
  - jeromegn 6y ago
    To be determined. We're hoping to contribute and use what's going to come out of hyper's h3 efforts (we use Rust for our reverse-proxy). There's not much there yet though: https://github.com/hyperium/h3 https://github.com/hyperium/h3 We're not in a huge hurry to support QUIC / H3 given its current adoption. However, our users' apps will be able to support it once UDP is fully launched, if they want to.
    - anderspitman 6y ago
      Are you using a custom reverse proxy? For a recent project I started with Caddy but ended up needing some functionality it didn't have, and didn't need most of what it did have. I'm currently using a custom proxy layer, but I'm concerned I might end up having to implement more than I want to (I know I'll at least need gzip). Curious what your experience at fly has been with this.
      - mrkurt 6y ago
        We are! It's Rust + Hyper. It is a _lot_ of work, but that's because we're trying to proxy arbitrary client traffic to arbitrary backends AND give them geo load balancing. Writing proxies is fun. Highly recommended.
        ignoramous 6y ago
        another stupid question, but can't help it: golang seems like a popular choice among network developers. Any reason that made fly.io choose Rust over golang for the proxy?
        mrkurt 6y ago
        Because of JavaScript. Really! We settled on Rust back when we were building a JS runtime into our proxy. It's a great language for hosting v8. When we realized our customers just wanted to run any ol' code, and not be stuck in a v8 isolate, we extracted the proxy bits and kept them around. I think you could build our proxy just fine in Go. One really nice thing about Rust, though, is the Hyper HTTP library. It's _much_ more flexible than Go's built in HTTP package, which turns out to be really useful for our particular product.
        anderspitman 6y ago
        Cool, thanks! I was actually just playing with Hyper for a few hours last night. Are you guys using async/await yet? Any suggestions for learning materials for async rust other than the standard stuff?
      - mholt 6y ago
        What functionality did you need that Caddy didn't have?
        anderspitman 6y ago
        Hey Matt! I'm referring to the ability to change the Caddy config from an API that is itself proxied through Caddy. Here's the issue which you very helpful in[0]. Ultimately I realized that most of what I needed from Caddy was really just certmagic, which has worked flawlessly since I got it set up. Plus I need the final product to compile into a single binary. Since my custom reverse proxy only took a few lines of code, I haven't worried too much about it. But there are a few features which I'll have to integrate eventually. If I end up seeing myself headed down the path of making a full-fledged reverse proxy, I'll reconsider trying to implement my project as a Caddy plugin. [0]: https://github.com/caddyserver/caddy/issues/3754 https://github.com/caddyserver/caddy/issues/3754
- eptcyka 6y ago
  Unrelated, but could you please expand on how firecracker fits within your stack?
  - tptacek 6y ago
    You could describe our job as "taking Dockerfiles from customers and running them globally"; the way we actually "run" Docker images is to convert them to root filesystems and run them inside Firecracker. Firecracker is the core of our stack.
    - riyakhanna1983 6y ago
      Why not just run Docker containers natively?
      - mrkurt 6y ago
        We have a whole post about workload isolation that answers this: https://fly.io/blog/sandboxing-and-workload-isolation/ https://fly.io/blog/sandboxing-and-workload-isolation/ The tldr is: Docker containers don't offer enough isolation for multi tenant systems. They're also very slow to boot, compared to a Firecracker VM.
        riyakhanna1983 6y ago
        But does FC not incur high I/O overhead at runtime?
        mrkurt 6y ago
        The flippant answer is "it doesn't really matter, safety usually wins over performance". But also, we run Firecrackers on our own physical servers and performance is quite good (even network + disk performance).
    - k__ 6y ago
      I find it rather curious that the cloud-native crowd tries to sell us containers, but cloud providers themselves use VMs. Like, if not even AWS, Cloudflare, and fly.io use containers, how can K8s be native in any way? I mean, that makes even Lambda, which runs on Firecracker VMs, more native than K8s.

Yep! We have a "Firecracker that accepts QUIC" running with this. People usually want HTTP + TLS handled for them, though. So when we ship QUIC + HTTP3 as a first class feature, we'll terminate QUIC and give people whatever their app process can accept.

dochtman 6y ago
Any insight into what QUIC/H3 stack you'll be using for the proxy?
- jeromegn 6y ago
  To be determined. We're hoping to contribute and use what's going to come out of hyper's h3 efforts (we use Rust for our reverse-proxy). There's not much there yet though: https://github.com/hyperium/h3 https://github.com/hyperium/h3 We're not in a huge hurry to support QUIC / H3 given its current adoption. However, our users' apps will be able to support it once UDP is fully launched, if they want to.
  - anderspitman 6y ago
    Are you using a custom reverse proxy? For a recent project I started with Caddy but ended up needing some functionality it didn't have, and didn't need most of what it did have. I'm currently using a custom proxy layer, but I'm concerned I might end up having to implement more than I want to (I know I'll at least need gzip). Curious what your experience at fly has been with this.
    - mrkurt 6y ago
      We are! It's Rust + Hyper. It is a _lot_ of work, but that's because we're trying to proxy arbitrary client traffic to arbitrary backends AND give them geo load balancing. Writing proxies is fun. Highly recommended.
      - ignoramous 6y ago
        another stupid question, but can't help it: golang seems like a popular choice among network developers. Any reason that made fly.io choose Rust over golang for the proxy?
        mrkurt 6y ago
        Because of JavaScript. Really! We settled on Rust back when we were building a JS runtime into our proxy. It's a great language for hosting v8. When we realized our customers just wanted to run any ol' code, and not be stuck in a v8 isolate, we extracted the proxy bits and kept them around. I think you could build our proxy just fine in Go. One really nice thing about Rust, though, is the Hyper HTTP library. It's _much_ more flexible than Go's built in HTTP package, which turns out to be really useful for our particular product.
      - anderspitman 6y ago
        Cool, thanks! I was actually just playing with Hyper for a few hours last night. Are you guys using async/await yet? Any suggestions for learning materials for async rust other than the standard stuff?
    - mholt 6y ago
      What functionality did you need that Caddy didn't have?
      - anderspitman 6y ago
        Hey Matt! I'm referring to the ability to change the Caddy config from an API that is itself proxied through Caddy. Here's the issue which you very helpful in[0]. Ultimately I realized that most of what I needed from Caddy was really just certmagic, which has worked flawlessly since I got it set up. Plus I need the final product to compile into a single binary. Since my custom reverse proxy only took a few lines of code, I haven't worried too much about it. But there are a few features which I'll have to integrate eventually. If I end up seeing myself headed down the path of making a full-fledged reverse proxy, I'll reconsider trying to implement my project as a Caddy plugin. [0]: https://github.com/caddyserver/caddy/issues/3754 https://github.com/caddyserver/caddy/issues/3754
eptcyka 6y ago
Unrelated, but could you please expand on how firecracker fits within your stack?
- tptacek 6y ago
  You could describe our job as "taking Dockerfiles from customers and running them globally"; the way we actually "run" Docker images is to convert them to root filesystems and run them inside Firecracker. Firecracker is the core of our stack.
  - riyakhanna1983 6y ago
    Why not just run Docker containers natively?
    - mrkurt 6y ago
      We have a whole post about workload isolation that answers this: https://fly.io/blog/sandboxing-and-workload-isolation/ https://fly.io/blog/sandboxing-and-workload-isolation/ The tldr is: Docker containers don't offer enough isolation for multi tenant systems. They're also very slow to boot, compared to a Firecracker VM.
      - riyakhanna1983 6y ago
        But does FC not incur high I/O overhead at runtime?
        mrkurt 6y ago
        The flippant answer is "it doesn't really matter, safety usually wins over performance". But also, we run Firecrackers on our own physical servers and performance is quite good (even network + disk performance).
  - k__ 6y ago
    I find it rather curious that the cloud-native crowd tries to sell us containers, but cloud providers themselves use VMs. Like, if not even AWS, Cloudflare, and fly.io use containers, how can K8s be native in any way? I mean, that makes even Lambda, which runs on Firecracker VMs, more native than K8s.