16 ms·

HTTP/2 is a good example of how to handle textual-to-binary transition. The original HTTP/1 was textual (if a bit convoluted), and that helped it to become a l

by dexen 6y ago

HTTP/2 is a good example of how to handle textual-to-binary transition. The original HTTP/1 was textual (if a bit convoluted), and that helped it to become a lingua franca; helped cooperation and data interchange between applications proliferate; everybody was able to proverbially "scratch his itch". Gradually tooling grew up around it too. The HTTP/2 is binary, and also quite complex - however most of that is hidden behind the already established tooling, and from developer's perspective, the protocol is seen as "improved HTTP/1 with optional extras". The protocol appears textual for all developmental intents and purposes - because the transition was handled (mostly) smoothly in the tooling. The key APIs remained the same for quick, mundane use cases - and got extended for advanced use cases. There's a lesson in that having been a success. Unpopular opinion warning: contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level - which hampered its ability to be seamlessly employed in applications.

gambler 6y ago
>however most of that is hidden behind the already established tooling ...and so everyone without Google-level funding is stuck with this tooling. Thus, control over the protocol that millions of people could use to communicate with one another directly (by making websites and possibly servers) is ceded to a handful of centralized authorities that can handle the complexity and also happen to benefit from the new features. I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. There were no special libraries involved, so it was super-easy to understand what's going on. We're moving away from able to do things of this sort without special tooling and almost no one seems to notice or care.
- espadrine 6y ago
  > I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. That is still possible in the exact same way. But a toy is just a toy. All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. The result? Sure, it is a binary stream of random-looking bits. Yet to me, what matters about text protocols is not the ASCII encoding. It is the ability to read and edit the raw representation. As long as your protocol has an unambiguous one-to-one textual representation with two-way conversion, I can inspect it and modify it with no headache. An outstanding example of that is WASM, which converts to and from WAT: https://en.wikipedia.org/wiki/WebAssembly#Code_representation https://en.wikipedia.org/wiki/WebAssembly#Code_representatio...
  - LocalH 6y ago
    >All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. I reject the notion that encryption should be mandatory for all websites. It should be best practice, especially for a "modern" website with millions of users, but we don't need every single website encrypted.
    - miohtama 6y ago
      While I agree with you, it is best to be on the safe side. The damage from having a wrong website unencrypted could be massive vs. cost of simply encrypting everything. Demanding 100% encryption is an extra layer to protect against human mistakes.
      - LocalH 6y ago
        Demanding 100% encryption also locks out some retrocomputing hardware that had existing browsers in the early Internet days. Not all sites need encryption. Where it's appropriate, most certainly. HTTPS should be the overwhelming standard. But there is a place for HTTP, and there should always be. Same for other unencrypted protocols. Unencrypted FTP still has a place.
        SahAssar 6y ago
        HTTP/FTP certainly have their place, but that is not on the open internet. For retro computing and otherwise special cases a proxy on the local network can do HTTP->HTTPS conversion.
        Wowfunhappy 6y ago
        It's unfortunate that there doesn't seem to be a turn-key solution for this at the moment. I'm currently using Squid so I can use web-enabled applications on an older version of OS X, and it's great, but figuring out how to set it up took a solid day of work (partly because their documentation isn't very good), and the result will only work on macOS. Mitmproxy is much easier to set up, but too heavy for 24/7 use. Ideally this would be a DDWRT package, or maybe a Raspberry Pi image, all preconfigured and ready to go...
- tasogare 6y ago
  If you care about protocol simplicity and their afferant implementation costs, then the continuously creeping Web platform it a few magnitudes worse in this respect.
- SahAssar 6y ago
  Node has had a built in HTTP server since v0.1.17, are you sure those examples didn't use that? Because if they did then it was the same in those examples as it is now. Source: https://nodejs.org/api/http.html#http_class_http_server https://nodejs.org/api/http.html#http_class_http_server
- derefr 6y ago
  > ...and so everyone without Google-level funding is stuck with this tooling. ...no? It's actually pretty easy to write an HTTP/2 client library yourself. There are tons and tons of implementations of HTTP/2; at this point, nearly as many as there are of HTTP/1. Presuming you're familiar with the spec, you can code one yourself in an evening or two. (I'm in the Elixir ecosystem myself. We have https://hex.pm/packages/kadabra https://hex.pm/packages/kadabra — which is, apparently, 2408LOC at present. That's without stripping comments/trivial closing-token lines/etc, because Elixir doesn't have a good sloccount utility.) The only thing that could potentially get in the way of HTTP/2 implementation, is a language not having good support for parsing/generating binary data. Languages like Javascript or Python — i.e. languages where you have to deal with binary data as if it were a type of string, where the tools for dealing with bytes and with codepoints are all mushed together and confused — struggle with binary protocols of all kinds. People do nevertheless write binary protocols for these languages. But that's why people don't tend to think of these as "backend server languages", and instead use languages like Go or Erlang or even C—as in these languages, there are first-class "array/slice of bytes" types, and highly-efficient operations for manipulating them (with strict, predictable "bits are bits" semantics), which make writing binary-protocol libraries a breeze.
  - schoen 6y ago
    > The only thing that could potentially get in the way of HTTP/2 implementation, is a language not having good support for parsing/generating binary data. Languages like Javascript or Python — i.e. languages where you have to deal with binary data as if it were a type of string, where the tools for dealing with bytes and with codepoints are all mushed together and confused — struggle with binary protocols of all kinds. People do nevertheless write binary protocols for these languages. Python explicitly changed that between Python 2 and Python 3 (not without considerable pain for its users -- the fact that open("/dev/urandom").read(10) makes Python 3 crash¹ was what put me off of learning it for some years, for example). This isn't to say that Python users or documentation have all made the switch, but modern Python is very capable of distinguishing bytes and character set codepoints, and typically insists that programmers do so in most contexts. ¹ This crash turns out to be extremely easy to fix, by adding the file mode "rb" to the open() call, but Python 2 programmers wouldn't have expected to have to do that.
contravariant 6y ago
After all I've read on HTTP/2 I'm still not entirely sure what problem it is trying to solve.
- nbm 6y ago
  The main benefit is multiplexing - being able to use the same connection for multiple transactions at the same time. This can have benefits in finding and keeping the congestion window at its calculated maximum size, reduce connection-related start-up, as well as overcome waiting for a currently-used connection to be free if you have a max connection per server model. The other potential benefits were priorities and server-initiated push, but both I’d say largely went unused and/or were too much trouble to use. Priorities were redesigned in HTTP 3 - more at https://blog.cloudflare.com/adopting-a-new-approach-to-http-prioritization/ https://blog.cloudflare.com/adopting-a-new-approach-to-http-... - and Chrome recently decided push in HTTP 2 wasn’t worth keeping around - https://www.ctrl.blog/entry/http2-push-chromium-deprecation.html https://www.ctrl.blog/entry/http2-push-chromium-deprecation.... HTTP 2’s main problem is head-of-line blocking in TCP - basically, if you lose a packet, you wait until you get that packet and acknowledge a maximum amount of packets thereafter - slowing the connection down. With multiplexing, this means that a bunch of in-flight transactions, as well as potentially future ones, are blocked at the same time. With multiple TCP connections, you don’t have this problem of a dropped packet affecting multiple transactions. HTTP 3 has many more benefits - basically, all the benefits of multiplexing without the head of line blocking (instead, only that stream is affected), as well as ability to negotiate alternative congestion control algorithms when client TCP stacks don’t support newer ones - or come with bad defaults. And the future is bright for non-HTTP and non-reliable streams as well over QUIC, the transport HTTP 3 is built on.
  - contravariant 6y ago
    Right, all this kind of feels as if HTTP/2 is trying to solve transport layer problems in the application layer. Especially if you leave out the server initiated push. I can't really pretend to know much about this but I can't say I'm surprised that this causes problems when the underlying transport-layer protocol is trying to solve the same problem. So is it correct to view HTTP/3 as basically taking a step back and just running HTTP over a different transport-layer protocol (QUIC)? (If so I think the name is a bit confusing, HTTP over QUIC would be much clearer)
bullen 6y ago
HTTP/2.0 has TCP head-of-line issues, in practice that nullifies it's usefulness! HTTP/1.1 is much more balanced and simple, and as I said all over this topic the bottleneck is elsewhere!
rectang 6y ago
> Unpopular opinion warning: contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level - which hampered its ability to be seamlessly employed in applications. Unpopular? The gratuitous breaking of backwards compatibility by IPv6, inflicting staggering inefficiencies felt directly or indirectly by all internet users, should be a canonical case study by now. It should be taught to all engineering students as a cautionary tale: never, ever do this.
- convolvatron 6y ago
  i'm fairly sympathetic here - except part of the blame should really be on the socket layer and resolver interface. if they had been a bit better at modelling multiprotocol networks, this kind of transition would have been easier.
- Jenda_ 6y ago
  I would like to read about a better design that should be implemented instead - can you share some links? I cannot imagine how compatibility can be achieved given that address fields in IPv4 are fixed 32-bit.
  - toast0 6y ago
    I strongly believe IPv6 would be more fully deployed if they had only extended the address fields, and left everything else the same. Things like Neighbor Discovery Protocol don't have to exist; ARP would work fine; the protocol already contemplates different length addresses, it would just need a constant assigned for IPv6.
cesarb 6y ago
> contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level Unfortunately, it was not possible for IPv6 to maintain backward compatibility with IPv4 at the API level. That's because the IPv4 API was not textual; it was binary, with fixed-size 32-bit fields everywhere. What they did was the next best thing: they made the IPv6 API able to also use IPv4 addresses, so that new programs can use a single API for both IPv4 and IPv6.
- fulafel 6y ago
  The API compatibility is pretty far down the list of bottlenecks with IPv6. There was some churn related to it 20 years ago.
- m463 6y ago
  > Unfortunately, it was not possible for IPv6... Given hindsight, I think there is a ton of coulda-woulda-shoulda in bumpy transitions like ipv4 -> ipv6 or python 2->3

by dexen 6y ago

gambler 6y ago

>however most of that is hidden behind the already established tooling ...and so everyone without Google-level funding is stuck with this tooling. Thus, control over the protocol that millions of people could use to communicate with one another directly (by making websites and possibly servers) is ceded to a handful of centralized authorities that can handle the complexity and also happen to benefit from the new features. I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. There were no special libraries involved, so it was super-easy to understand what's going on. We're moving away from able to do things of this sort without special tooling and almost no one seems to notice or care.

espadrine 6y ago
> I remember how Node when it was just rising in popularity was usually demonstrated by writing a primitive HTTP server that served a "hello world" HTTP page. That is still possible in the exact same way. But a toy is just a toy. All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. The result? Sure, it is a binary stream of random-looking bits. Yet to me, what matters about text protocols is not the ASCII encoding. It is the ability to read and edit the raw representation. As long as your protocol has an unambiguous one-to-one textual representation with two-way conversion, I can inspect it and modify it with no headache. An outstanding example of that is WASM, which converts to and from WAT: https://en.wikipedia.org/wiki/WebAssembly#Code_representation https://en.wikipedia.org/wiki/WebAssembly#Code_representatio...
- LocalH 6y ago
  >All websites should encrypt their content with TLS. In fact, all protocols should encrypt their communications. I reject the notion that encryption should be mandatory for all websites. It should be best practice, especially for a "modern" website with millions of users, but we don't need every single website encrypted.
  - miohtama 6y ago
    While I agree with you, it is best to be on the safe side. The damage from having a wrong website unencrypted could be massive vs. cost of simply encrypting everything. Demanding 100% encryption is an extra layer to protect against human mistakes.
    - LocalH 6y ago
      Demanding 100% encryption also locks out some retrocomputing hardware that had existing browsers in the early Internet days. Not all sites need encryption. Where it's appropriate, most certainly. HTTPS should be the overwhelming standard. But there is a place for HTTP, and there should always be. Same for other unencrypted protocols. Unencrypted FTP still has a place.
      - SahAssar 6y ago
        HTTP/FTP certainly have their place, but that is not on the open internet. For retro computing and otherwise special cases a proxy on the local network can do HTTP->HTTPS conversion.
        Wowfunhappy 6y ago
        It's unfortunate that there doesn't seem to be a turn-key solution for this at the moment. I'm currently using Squid so I can use web-enabled applications on an older version of OS X, and it's great, but figuring out how to set it up took a solid day of work (partly because their documentation isn't very good), and the result will only work on macOS. Mitmproxy is much easier to set up, but too heavy for 24/7 use. Ideally this would be a DDWRT package, or maybe a Raspberry Pi image, all preconfigured and ready to go...
tasogare 6y ago
If you care about protocol simplicity and their afferant implementation costs, then the continuously creeping Web platform it a few magnitudes worse in this respect.
SahAssar 6y ago
Node has had a built in HTTP server since v0.1.17, are you sure those examples didn't use that? Because if they did then it was the same in those examples as it is now. Source: https://nodejs.org/api/http.html#http_class_http_server https://nodejs.org/api/http.html#http_class_http_server
derefr 6y ago
> ...and so everyone without Google-level funding is stuck with this tooling. ...no? It's actually pretty easy to write an HTTP/2 client library yourself. There are tons and tons of implementations of HTTP/2; at this point, nearly as many as there are of HTTP/1. Presuming you're familiar with the spec, you can code one yourself in an evening or two. (I'm in the Elixir ecosystem myself. We have https://hex.pm/packages/kadabra https://hex.pm/packages/kadabra — which is, apparently, 2408LOC at present. That's without stripping comments/trivial closing-token lines/etc, because Elixir doesn't have a good sloccount utility.) The only thing that could potentially get in the way of HTTP/2 implementation, is a language not having good support for parsing/generating binary data. Languages like Javascript or Python — i.e. languages where you have to deal with binary data as if it were a type of string, where the tools for dealing with bytes and with codepoints are all mushed together and confused — struggle with binary protocols of all kinds. People do nevertheless write binary protocols for these languages. But that's why people don't tend to think of these as "backend server languages", and instead use languages like Go or Erlang or even C—as in these languages, there are first-class "array/slice of bytes" types, and highly-efficient operations for manipulating them (with strict, predictable "bits are bits" semantics), which make writing binary-protocol libraries a breeze.
- schoen 6y ago
  > The only thing that could potentially get in the way of HTTP/2 implementation, is a language not having good support for parsing/generating binary data. Languages like Javascript or Python — i.e. languages where you have to deal with binary data as if it were a type of string, where the tools for dealing with bytes and with codepoints are all mushed together and confused — struggle with binary protocols of all kinds. People do nevertheless write binary protocols for these languages. Python explicitly changed that between Python 2 and Python 3 (not without considerable pain for its users -- the fact that open("/dev/urandom").read(10) makes Python 3 crash¹ was what put me off of learning it for some years, for example). This isn't to say that Python users or documentation have all made the switch, but modern Python is very capable of distinguishing bytes and character set codepoints, and typically insists that programmers do so in most contexts. ¹ This crash turns out to be extremely easy to fix, by adding the file mode "rb" to the open() call, but Python 2 programmers wouldn't have expected to have to do that.

contravariant 6y ago

After all I've read on HTTP/2 I'm still not entirely sure what problem it is trying to solve.

nbm 6y ago
The main benefit is multiplexing - being able to use the same connection for multiple transactions at the same time. This can have benefits in finding and keeping the congestion window at its calculated maximum size, reduce connection-related start-up, as well as overcome waiting for a currently-used connection to be free if you have a max connection per server model. The other potential benefits were priorities and server-initiated push, but both I’d say largely went unused and/or were too much trouble to use. Priorities were redesigned in HTTP 3 - more at https://blog.cloudflare.com/adopting-a-new-approach-to-http-prioritization/ https://blog.cloudflare.com/adopting-a-new-approach-to-http-... - and Chrome recently decided push in HTTP 2 wasn’t worth keeping around - https://www.ctrl.blog/entry/http2-push-chromium-deprecation.html https://www.ctrl.blog/entry/http2-push-chromium-deprecation.... HTTP 2’s main problem is head-of-line blocking in TCP - basically, if you lose a packet, you wait until you get that packet and acknowledge a maximum amount of packets thereafter - slowing the connection down. With multiplexing, this means that a bunch of in-flight transactions, as well as potentially future ones, are blocked at the same time. With multiple TCP connections, you don’t have this problem of a dropped packet affecting multiple transactions. HTTP 3 has many more benefits - basically, all the benefits of multiplexing without the head of line blocking (instead, only that stream is affected), as well as ability to negotiate alternative congestion control algorithms when client TCP stacks don’t support newer ones - or come with bad defaults. And the future is bright for non-HTTP and non-reliable streams as well over QUIC, the transport HTTP 3 is built on.
- contravariant 6y ago
  Right, all this kind of feels as if HTTP/2 is trying to solve transport layer problems in the application layer. Especially if you leave out the server initiated push. I can't really pretend to know much about this but I can't say I'm surprised that this causes problems when the underlying transport-layer protocol is trying to solve the same problem. So is it correct to view HTTP/3 as basically taking a step back and just running HTTP over a different transport-layer protocol (QUIC)? (If so I think the name is a bit confusing, HTTP over QUIC would be much clearer)

bullen 6y ago

HTTP/2.0 has TCP head-of-line issues, in practice that nullifies it's usefulness! HTTP/1.1 is much more balanced and simple, and as I said all over this topic the bottleneck is elsewhere!

rectang 6y ago

> Unpopular opinion warning: contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level - which hampered its ability to be seamlessly employed in applications. Unpopular? The gratuitous breaking of backwards compatibility by IPv6, inflicting staggering inefficiencies felt directly or indirectly by all internet users, should be a canonical case study by now. It should be taught to all engineering students as a cautionary tale: never, ever do this.

convolvatron 6y ago
i'm fairly sympathetic here - except part of the blame should really be on the socket layer and resolver interface. if they had been a bit better at modelling multiprotocol networks, this kind of transition would have been easier.
Jenda_ 6y ago
I would like to read about a better design that should be implemented instead - can you share some links? I cannot imagine how compatibility can be achieved given that address fields in IPv4 are fixed 32-bit.
- toast0 6y ago
  I strongly believe IPv6 would be more fully deployed if they had only extended the address fields, and left everything else the same. Things like Neighbor Discovery Protocol don't have to exist; ARP would work fine; the protocol already contemplates different length addresses, it would just need a constant assigned for IPv6.

cesarb 6y ago

> contrast the success of HTTP/2 with the failure of IPv6 to maintain backward compatibility at the API level Unfortunately, it was not possible for IPv6 to maintain backward compatibility with IPv4 at the API level. That's because the IPv4 API was not textual; it was binary, with fixed-size 32-bit fields everywhere. What they did was the next best thing: they made the IPv6 API able to also use IPv4 addresses, so that new programs can use a single API for both IPv4 and IPv6.

fulafel 6y ago
The API compatibility is pretty far down the list of bottlenecks with IPv6. There was some churn related to it 20 years ago.
m463 6y ago
> Unfortunately, it was not possible for IPv6... Given hindsight, I think there is a ton of coulda-woulda-shoulda in bumpy transitions like ipv4 -> ipv6 or python 2->3