11 ms·

> Really it's hard to point to any popular open-source tools that fully support HTTP/3: rollout has barely even started. > This seems contradictory. What's goi

• kstrauser 2y ago
  I'm not an expert on HTTP/3, but vehemently disagree about IPv6. It removes tons of overhead and cruft, making it delightful for datacenter work. That, and basically guaranteeing you don't have to deal with the company you just acquired having deployed their accounts with the same 10/16 subnet your own company uses.
  • cogman10 2y ago
    The problem I keep running into is that IPv6 support in common infrastructure is somewhat lacking. It's always a headache to learn that some container orchestration system doesn't support IPv6. Or an http client. Or a DNS resolver. Or whatever. Not to mention the supreme annoyance I have that, to this day, my ISP still does not have IPv6 addressing available.
    • frollogaston 2y ago
      Yep, it's tragic because it all stems from unforced differences vs ipv4. The design was reasonable, but with perfect hindsight, it needed to be different. They needed to keep the existing /32s and just make the address field bigger, despite the disadvantages. "Everywhere but nowhere" is sorta how I'd describe ipv6. Most hardware and lower-level software supports it, so obviously it wasn't impossible to support a new protocol, but it's not being used.
      • JoshTriplett 2y ago
        > Yep, it's tragic because it all stems from unforced differences vs ipv4. The design was reasonable, but with perfect hindsight, it needed to be different. They needed to keep the existing /32s and just make the address field bigger, despite the disadvantages. Exactly. I would love to have seen the world in which that happened, and where all the other parts of IPv6 were independently proposed (and likely many of them rejected as unwanted).
        • frollogaston 2y ago
          The main problem wasn't all the smaller features but one big one in particular that can't be split into smaller pieces, the new addressing scheme. They wanted to replace all the existing addresses, which meant replacing all the routes. Besides the difficulty of that by itself, it automatically meant that the v6 versions of DNS, DHCP, NAT, etc wouldn't support v4, rather it'd be a totally separate stack. There were also some small things. And routers often having bad defaults for v6, which btw, would not even be a concern if they left the big thing alone.
          • JoshTriplett 2y ago
            > Besides the difficulty of that by itself, it automatically meant that the v6 versions of DNS, DHCP, NAT, etc wouldn't support v4, rather it'd be a totally separate stack. Sure, "make the addresses bigger" would have required providing DHCPv6, DNS AAAA records, and various other protocol updates for protocols that embedded IP addresses. And making changes to the protocol header at the same time (e.g. removing the redundant checksum) were also a good idea. It didn't require pushing SLAAC instead of DHCP. It didn't require recommending (though fortunately not requiring) IPsec for all IPv6 stacks. It didn't require changing the address syntax to use colons, causing pain for all protocols that used `IP:port` or similar. It didn't require mandating link-local addresses for every interface. It didn't require adding a mandatory address-collision-detection mechanism. And I'm sure I'm forgetting a few.
            • frollogaston 2y ago
    • p_l 2y ago
      Major reason for that is BSD Sockets and their leaky abstraction that results in hardcoding protocol details in application code. For a good decade a lot of software had to be slowly patched in every place that made a socket to add v6 support, and sometimes multiple times because getaddrinfo didn't reach everyone early enough.
      • throwaway2037 2y ago
        > results in hardcoding protocol details in application code Are you suggesting that this could have been implemented a different way? Example: IP could be negotiated to upgrade from v4 to v6? I am curious about your ideas.
        • hamandcheese 2y ago
          There isn't much need for many applications to know or care what IP protocol they are speaking, they are all just writing bytes to a TCP stream. I think the parent is saying that existing socket abstractions meant that these applications still had to be "upgraded" to support IPv6 whereas it could/should have been handled entirely by the OS with better socket APIs.
        • tsimionescu 2y ago
          I think in principle, an application didn't need to know the exact format of an IP address, even if connecting directly to an IP. A simple idea that could have made application code much more IP-agnostic would have been for SOCK_ADDR_IN to take the IP in string format, not as a four-byte value. That way, lots of application code would not need to even be recompiled to move from a 4 byte IPv4 address to a 16 byte IPv6 address, whereas today they not only need to be recompiled, they need to be changed at the source level to use a new type that allows for both. Of course, code that operates on packets, in the TCP/IP stack of the OS would have still needed to be rewritten. But that is far less code than "every application that opens a socket". Of course, this only applies to code that uses IPs only to open connections. There's lots of application code that does more things with IPs, such as parsing, displaying, validating etc. All of this code would still need to be rewritten to accept IPv6 addresses (and its much more complex string representations), that part is inevitable.
    • afiori 2y ago
      I wonder if something like HTTP connection upgrade would have been possible with ipv6-ipv4, maybe something like imagine Machine 1 with ips 1.1.1.1 and 11::11 and machine 2 with ips 2.2.2.2 and 22::22. When machine 2 receives a packet from 1.1.1.1 at 2.2.2.2 it sends a ipv6 ping-like packet to the ipv4-mapped address ::ffff:1.1.1.1 saying something like "hey you can also contact me at 22::22 and if machine 1 undertands then it can try to use the new address for the following packets. I can see how it would be hard to secure this operation.
  • cowsandmilk 2y ago
    For those building on AWS with VPC per service and using PrivateLink for connections between services, the whole IP conflict problem just evaporates. Admittedly, you’re paying some premiums to Amazon for that convenience.
  • einpoklum 2y ago
    > It removes tons of overhead and cruft Can you elaborate? Here, or with links? What kinds of overhead and cruft?
    • kstrauser 2y ago
      * The header is a fixed length. * You can't fragment packets. * The redundant checksum header was removed. * No more private addressing (unless you're a glutton for punishment). * No more NAT (see above). * Simpler routing. * Doesn't require DHCP. It benefits hugely from the lessons learned with IPv4.
      • tsimionescu 2y ago
        IPv6 packets can still be fragmented, but only at the source. IPv4 fragmentation has only worked this way in practice for a long time. Private addressing is still needed with IPv6, it's a crucial part of how address allocation works, and it's the only way to reliably connect to a client-like IPv6 device on the local network, since its public IP address will change all the time for privacy reasons, assuming it respects best practices. Routing is only simpler if the ISPs actually hand out the large prefixes they are supposed to. Not all of them do. DHCP is still required for many use cases. So now you have two solutions for handing out addresses, and you need to figure out when to use SLAAC and when to use DHCP. This is strictly more complex than IPv4, not simpler. SLAAC is mostly just unnecessary cruft, a cute little simple path for limited use cases, but it can never replace DHCPv6 for all use cases (e.g. for subnets smaller than a /64, for communicating additional information like a local DNS server or NTP server, for complex network topologies, for server machines etc).
      • cyberax 2y ago
        > * No more private addressing (unless you're a glutton for punishment). OK. So what happens if your ISP connection goes down? Your router will detect this and withdraw the ISP's prefix. So now you can't print because your printer doesn't have an address. Good luck. > * Doesn't require DHCP. This turns out to be a problem, as you can't easily see what's going on in the network.
      • 2y ago
  • yyyk 2y ago
    >That, and basically guaranteeing you don't have to deal with the company you just acquired having deployed their accounts with the same 10/16 subnet your own company uses. I always found that to be a desperate talking point. 'Prepare your network for the incredibly rare event where you intend to integrate directly' (didn't anyone hear of network segmentation?). It makes a lot more sense to worry about the ISP unilaterally changing your prefix - something that can only happen in IPv6.
    • bigstrat2003 2y ago
      > It makes a lot more sense to worry about the ISP unilaterally changing your prefix - something that can only happen in IPv6. ISPs unilaterally change your DHCP address on IPv4 all the time. And in any situation where you would have a static address for IPv4, your ISP should have no problem giving you a static v6 prefix. This argument makes no sense at all.
      • shawabawa3 2y ago
        Your ISP provided ipv4 has no relation to your internal private network space My understanding is that for some bizarre reason this is not usually the case with IPv6
• aoeusnth1 2y ago
  What kind of overhead? I’m curious if there’s data about this because I hadn’t heard that 1.1 was better for the data center.
  • Karrot_Kream 2y ago
    QUIC/HTTP3 relies on TLS. If you already have some encrypted transport, like an Istio/Envoy service mesh with mutual TLS, or Zerotier/Tailscale/Wireguard style encrypted overlay network, then there are no benefits to using HTTP3. Moreover native crypto libraries tend do a better job handling encryption anyway so rather than wasting cycles doing crypto in Go or Node it makes more sense to let the service mesh or the overlay handle encryption and let your app just respond to clear requests.
    • dathinab 2y ago
      > let the service mesh or the overlay handle encryption which could use HTTP/2 or HTTP/3 and HTTP/2 for the localhost (or unix socket) gateway<->app step to provide e.g. WebTransport support
      • Karrot_Kream 2y ago
        Sure I was responding to the context as I understood it here which was listening on HTTP/3 as an application rather than a service mesh layer. HTTP/3 can definitely be a choice for service mesh or some sort of overlay. Personally if I were setting up a new cloud/DC today I'd probably just use ZeroTier (or Tailscale) and let the overlay deal with encryption while I just have my sources and destinations do IP based filtering.
• urban_alien 2y ago
  Ok, HTTP/3 is mobile centric. But why not fallback to HTTP/2 in all other cases?
  • jallmann 2y ago
    http/1 traffic is a lot easier to inspect
  • tsimionescu 2y ago
    HTTP/2 is still mostly implemented only over TLS, and that can mean significant and completely useless overhead if the server-LB connection is already encrypted using some VPN solution like WireGuard.
• Varriount 2y ago
  I really have to agree with the "easier to debug" part. I one time had to debug a particularly nasty networking issue that was causing HTTP connections to just "stop" midway through sending data. Turned out to be a confusion mismatch between routers and allowed packet sizes. It would have been so much worse with a non-plaintext protocol.
• cogman10 2y ago
  Totally agree. Most of the benefit of HTTP 2/3 comes from minimizing TCP connections between app->lb. Once you are past the lb the benefits are dubious at best. Most application frameworks that I've dealt with have limited capabilities to handle concurrent requests, so it becomes a minor issue to have 100+ connections between the app and the lb. On the flipside, apps talking to the LB can create all sorts of headaches if they have even modest sized pools. 20 TCP connections from 100 different apps and you are already looking at hard to handle TCP flooding.
• frollogaston 2y ago
  Speaking of gRPC, it's unfortunate that they went all-in on HTTP/2. Should have made it work over HTTP/1.1. I know others made it work, but it wasn't first-party. Maybe it could've been more popular than JSON-over-HTTP by now.
  • koakuma-chan 2y ago
    No reason to choose HTTP/1.1 over HTTP/2.
    • frollogaston 2y ago
      The reason is that a lot of things don't support /2, ironically including Google's own AppEngine. Also Heroku.
    • otabdeveloper4 2y ago
      Nobody needs HTTP/2. It's just Google doing embrace-extend-extinguish of public domain protocols.
• raggi 2y ago
  HTTP/3 is not a mobile centric technology. Yes there was a lot of discussion of packet pacing and it's implications for mobile in early presentations on QUIC but that's not the same as "centric", that's one application of the behavior. Improved congestion control, reduced control plane cost and removal of head of line blocking behaviors have significant value in data center networks as well. How often do you work with services where they have absolutely atrocious tail latencies and wide gaps between median and mean latencies? how often is that a side effect of http/tcp semantics? IPv6 is the same deal, I sort of understand where the confusion comes from around QUIC because so much was discussed about mobile early on, and it just got parroted heavily in the rumor mill, but IPv6? that long predates the mobile explosion, and again, it helps as an application, but ascribing it as the only application because of it's applicability somewhere else doesn't hold up to basic scrutiny. The largest data centers these days are pushing up against a whole v4 IP class (I know, classes are dead, sorta) in hardware addressable compute units - a trend that is not slowing. We did this with quic data center side: https://tailscale.com/blog/living-in-the-future#the-c10k-problem-is-now-the-c10000k-problem https://tailscale.com/blog/living-in-the-future#the-c10k-pro... and while it might be slightly crazy in and of itself, it's far more practical with multiplexing than with a million excessively sized buffers competing over pools and so on. There is absolutely value to quic and ipv6 in the data center, perhaps it's not so useful for traditionally shaped and sized LAMP stacks, but you can absolutely make great use of these at scale and in modern architectures, and they open a lot of doors/relax constraints in the design space. This also doesn't mean everyone needs to reach for them, but I don't think they should be discarded or ascribed limited purpose so blithely.
  • djha-skin 2y ago
    I will acknowledge that truly massive datacenter deployments can and do use these technologies to good effect, but I haven't worked at any of these kinds of places in the last fifteen years and I suspect many (most?) of my colleagues haven't either. Anything smaller than a /8, they usually don't add much and just get in the way more often than not.
• jiggawatts 2y ago
  > pass back HTTP 1.1 ... to the backing service. Sure, but now you've lost some of the benefits of HTTP/3, such as the header compression and less head-of-line blocking. To some degree the load balancer can solve this by using multiple parallel HTTP 1.1 streams, but in practice I've seen pretty bad results in many common scenarios.
  • djha-skin 2y ago
    No one cares about those "benefits" _on a gigabit line_. The head of your line is not blocked at such speeds, believe you me. Same thing with compression. Like, why. Other than to make it harder to debug?
    • jiggawatts 2y ago
      I had head-of-line-blocking issues recently on a 10 Gbps data centre link! HTTP client packages often use a small, fixed number of connections per domain. So if you have two servers talking to each other and there's slow requests mixed in with short RPCs, the latter can sit in a queue for tens of seconds.
• shmerl 2y ago
  How is IPv6 not suitable?
• karel-3d 2y ago
  How is IPv6 mobile centric? It's from the 90s
  • skissane 2y ago
    The protocol isn’t but its deployment is - real-world deployment is predominantly mobile. That’s nothing to do with the inherent technical features of the protocol, it is a consequence of market history
• KaiserPro 2y ago
  HTTP3 is patch that unfucked some stupid design choices from HTTP2[1] However IPv6 is perfectly suited to the datacentre. So long as you have properly infrastructure setup (ie properly functioning DNS) IPv6 is a godsend for simplifying medium scale infra. In fact, if you want to get close to a million hosts, you need ipv6. [1] Me and http/2 have beef, TCP multiplexing was always going to be a bad idea, but because idealism got in the way of testing