10 ms·

A lot of this sounds like transport layer instead of application layer. I'm curious why it ended up in HTTP instead of TCP. For example, you could specify tha

by thisrod 11y ago

A lot of this sounds like transport layer instead of application layer. I'm curious why it ended up in HTTP instead of TCP. For example, you could specify that ports 49152 to 50179 formed a block on a given host, and so on in groups of 1028. Ports in a block would have to connect to the same port on the other end, and would share a packet buffer. There could be some rules to let the network stack pass on data to a process in anticipation of a three way handshake completing, TLS could assume that new connections shared the same secrets as existing ones, and so on. That seems simpler, and it prevents the wheel from having to be reinvented for every new application protocol. An obvious reason not to do that is that TCP is too widely used for the effects of changes to be predicted. Is that the only reason?

gtirloni 11y ago
I think you're confusing the transport layer with the session and presentation layers. That being said, HTTP/2 is more than just a different way of opening connections and, yes, changing TCP in such fundamental ways would be impossible given the widespread adoption and burden a layer that is not expected to do all of that (when it should be flexible and enable more complex operations from the upper layers).
- netheril96 11y ago
  > I think you're confusing the transport layer with the session and presentation layers. There is no confusion. No one uses OSI model. So there are no session and presentation layers, at least in the minds of OP.
  - billyhoffman 11y ago
    Ok, so in this view what is TLS? It's not the "transport layer" but it also is application agnostic.
    - netheril96 11y ago
      Application layer. Everything above the transport layer is on application layer.
aidenn0 11y ago
It ended up in application layer because of http gateways and NATs. The reason why spdy was https only at google was because https is pretty much the only port that nearly all corporate gateways let through as a "black box" Google even did tests with http over SCTP[1] and found that it solved a lot of the problems that spdy did. It's well accepted that the transport-layer is the "right" layer to fix this in, but it would not allow as wide a deployment. 1: http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...
chetanahuja 11y ago
This is the major knock on HTTP2. It's entirely focused on hacking around transport layer issues while keeping TCP intact and has shied away from fixing real and serious problems with the HTTP protocol itself. Here's a quote from the HTTP2 FAQ: ( https://http2.github.io/faq/#can-http2-make-cookies-or-other-headers-better https://http2.github.io/faq/#can-http2-make-cookies-or-other... ) "In particular, we want to be able to translate from HTTP/1 to HTTP/2 and back with no loss of information. If we started “cleaning up” the headers (and most will agree that HTTP headers are pretty messy), we’d have interoperability problems with much of the existing Web." Pretty disappointing really. It took 15 years of haggling and we ended up with precisely the old protocol but now in a new, improved binary format with multiplexing.
- netheril96 11y ago
  > Pretty disappointing really. It took 15 years of haggling and we ended up with precisely the old protocol but now in a new, improved binary format with multiplexing. Yeah, we can have a perfect reimplementation of the widely used protocol to fix all of the problems at once. Just look at how successful IPv6 has been.
  - neeee 11y ago
    IPv6 doesn't fix most of the problems with IP, in fact it makes some worse (routing tables are now bigger because of bigger addresses and there can be more of them). If IPv6 actually did attempt to fix issues in IP it might have seen faster adoption.
    - richardwhiuk 11y ago
      Routing tables will be smaller due to aggregation - that's the major factor driving up IPv4 route tables.
  - nly 11y ago
    IPv6 is doing fine. I have IPv6 at home, and natively on all of my servers. According to my Openwrt AP stats, about 47% of all my household inbound traffic receiving is via IPv6 (over 8 days of basically just web browsing).
jballanc 11y ago
It's funny to think that a mere 20 or so years ago, ethernet wasn't even a foregone conclusion. The first time I connected a computer to a network, I had to make sure I was plugging into the ethernet and not the token ring. Now here we are pushing transport layer fixes into the application layer just because we don't want to have to explain to sysadmins how to reconfigure their firewalls?
- digi_owl 11y ago
  I think it is aimed more at home users that are stuck behind a NAT the ISP controls than sysadmins.
  - rkangel 11y ago
    So we're not fixing the transport layer due to limitations in the network layer that have been fixed, we're just not using them.
    - digi_owl 11y ago
      Pretty much. I find myself suspecting that we will see more and more sofware "fixes" piled on top to avoid doing mass hardware replacements...
      - a3n 11y ago
        Yet it can be done. Digital over the air TV for example.
        digi_owl 11y ago
        One thing when it is done at the consumers expense, another when the company has to cover it.
    - dtech 11y ago
      You mean IPv6? That suffers from exactly the same problem: ISP's just don't give a damn and don't roll it out. You cannot expect your average internet user to know about these things, so they get away with it.
cnvogel 11y ago
> I'm curious why it ended up in HTTP instead of TCP. ...or just run HTTP over SCTP. http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#Message-based_multi-streaming http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...
- acdha 11y ago
  Check out http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#Implementations http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...: It's not supported at all on Windows or OS X and the implementations everywhere else are far less tested than TCP. That's a large immediate problem, particularly since you have to update the kernel to fix it, and it also means that many, many intermediaries (home routers, proxies, corporate firewalls, etc.) have never been pushed to support it at all, let alone as well as commonly-used protocols. The good news is that the semantics are close enough that if the situation improves for both client support and, critically, intermediaries it would be relatively straightforward to migrate to a HTTP/2-over-SCTP hybrid if proved better in some way.
  - cnvogel 11y ago
    I'm completely aware of the fact that SCTP isn't in the native Windows networking stack, and I'll take your word that it's not in OSX. But as long as we put up all these Frankenstein solutions of handling things up-in-the-stack because firewall admins can't be bothered up upgrade or configure their stuff correctly, we are just not putting out enough pressure that this will get changed! And even if we now introduce this sad compromise that is HTTP/2, there also will be a lot of proxy/firewall appliances that block HTTP/2, and there will be the equivalent of the government-entity of large corporation that was stuck on IE4/WinXP until 2014 using an outdated web-browser or intranet server. So, maybe we should try to get incompatible protocols out much earlier and if they turn out to have merit, we could enable them on released products and have Chrome/Firefox put up a nagging reminder: "Your web-experience would be much improved (or: this premium content could be watched at higher resolution, or security to your banking website, or...) if your network infrastructure would support SCTP/IPv6/DNSsec/, please ask your ISP or Administrator".
hobohacker 11y ago
To answer your specific question about why not in TCP, it's not deployable. Here's why: * In order to get multiplexing and other features introduced with HTTP/2, you need to change protocol framing. However, this means that the protocol is no longer backwards compatible. There are many ways to roll out non-backwards-compatible changes, but for TCP, they were deemed unacceptable. For example, you could negotiate the protocol change via a TCP extension. However, TCP extensions are known not to be widely deployable (https://www.imperialviolet.org/binary/ecntest.pdf https://www.imperialviolet.org/binary/ecntest.pdf) over the public internet. You could use a specific port, but that doesn't traverse enough middleboxes on the public internet (http://www.ietf.org/mail-archive/web/tls/current/msg05593.html http://www.ietf.org/mail-archive/web/tls/current/msg05593.ht...). Yada yada. * More importantly, TCP is generally implemented in the OS. That means, updating a protocol requires OS updates. That means we need both server and client OS updates in order to speak the new protocol. If you look at the very sad numbers on Windows versions uptake, Android version uptake, etc, you'll understand why many people don't want to wait for all OSes to update in order to take advantage of new protocol features. To quote Roberto (http://www.ietf.org/mail-archive/web/tsvwg/current/msg12184.html http://www.ietf.org/mail-archive/web/tsvwg/current/msg12184....) in his response to IETF transport protocol folks asking why application protocol folks are hacking around the transport: """ I understand the points you make, and am sympathetic. I feel the same way when I see people abusing HTTP to provide async notifications, etc. The fact is, however, if it isn't deployed, no matter how nice it would theoretically be when it is, it isn't useful and people WILL work around the problem. That is true regardless of whether the problem is at the application layer (e.g. HTTP), or at the transport layer (e.g. TCP), or elsewhere. The primary motivation is to get things working, and making things that lack redundancy, or are elegant comes in at a distant second. Deployed is the most important feature, thus things which quickly move protocols and protocol changes from theoretical to deployed are by far the most important things. The longer this takes, the more likely that the work-around becomes standard practice, at which point we've all "lost" the game. -=R """ Basically, this is another instance of Linus's quote (https://lkml.org/lkml/2009/3/25/632 https://lkml.org/lkml/2009/3/25/632): "Theory and practice sometimes clash. And when that happens, theory loses. Every single time." Theoretically, it'd be far more elegant to fix this in the transport layer. But in practice, it doesn't work. Except, maybe if you implement on top of UDP (e.g. QUIC), since that would be in user-space and firewalls don't filter out UDP as much. For more extensive analysis of HTTP/2's deployability options, amongst other considerations, check out my post on HTTP/2 considerations: https://insouciant.org/tech/http-slash-2-considerations-and-tradeoffs/#Deployability https://insouciant.org/tech/http-slash-2-considerations-and-....
AndrewDucker 11y ago
STCP is what you want. Sadly, Microsoft have completely failed to support it, so it's not going to happen.
megaman821 11y ago
Out of all the options putting transport layer concerns in the HTTP protocol is my least favorite. Google built a multiplexed stream transport over UDP called QUIC. HTTP over QUIC gives you most of what HTTP2 does. Also there is a transport layer protocol called SCTP. Again, HTTP over SCTP gives you most of what HTTP2 does.
- MertsA 11y ago
  HTTP over SCTP give you more than what HTTP2 does. I just wish Google would have made SPDY only work over SCTP as a measure to push for widespread adoption. Imagine finally having a seamless connection for every application on mobile devices where it would switch from wifi to 4G without any serious latency or dropped connections.

by thisrod 11y ago

gtirloni 11y ago

I think you're confusing the transport layer with the session and presentation layers. That being said, HTTP/2 is more than just a different way of opening connections and, yes, changing TCP in such fundamental ways would be impossible given the widespread adoption and burden a layer that is not expected to do all of that (when it should be flexible and enable more complex operations from the upper layers).

netheril96 11y ago
> I think you're confusing the transport layer with the session and presentation layers. There is no confusion. No one uses OSI model. So there are no session and presentation layers, at least in the minds of OP.
- billyhoffman 11y ago
  Ok, so in this view what is TLS? It's not the "transport layer" but it also is application agnostic.
  - netheril96 11y ago
    Application layer. Everything above the transport layer is on application layer.

aidenn0 11y ago

It ended up in application layer because of http gateways and NATs. The reason why spdy was https only at google was because https is pretty much the only port that nearly all corporate gateways let through as a "black box" Google even did tests with http over SCTP[1] and found that it solved a lot of the problems that spdy did. It's well accepted that the transport-layer is the "right" layer to fix this in, but it would not allow as wide a deployment. 1: http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...

chetanahuja 11y ago

This is the major knock on HTTP2. It's entirely focused on hacking around transport layer issues while keeping TCP intact and has shied away from fixing real and serious problems with the HTTP protocol itself. Here's a quote from the HTTP2 FAQ: ( https://http2.github.io/faq/#can-http2-make-cookies-or-other-headers-better https://http2.github.io/faq/#can-http2-make-cookies-or-other... ) "In particular, we want to be able to translate from HTTP/1 to HTTP/2 and back with no loss of information. If we started “cleaning up” the headers (and most will agree that HTTP headers are pretty messy), we’d have interoperability problems with much of the existing Web." Pretty disappointing really. It took 15 years of haggling and we ended up with precisely the old protocol but now in a new, improved binary format with multiplexing.

netheril96 11y ago
> Pretty disappointing really. It took 15 years of haggling and we ended up with precisely the old protocol but now in a new, improved binary format with multiplexing. Yeah, we can have a perfect reimplementation of the widely used protocol to fix all of the problems at once. Just look at how successful IPv6 has been.
- neeee 11y ago
  IPv6 doesn't fix most of the problems with IP, in fact it makes some worse (routing tables are now bigger because of bigger addresses and there can be more of them). If IPv6 actually did attempt to fix issues in IP it might have seen faster adoption.
  - richardwhiuk 11y ago
    Routing tables will be smaller due to aggregation - that's the major factor driving up IPv4 route tables.
- nly 11y ago
  IPv6 is doing fine. I have IPv6 at home, and natively on all of my servers. According to my Openwrt AP stats, about 47% of all my household inbound traffic receiving is via IPv6 (over 8 days of basically just web browsing).

jballanc 11y ago

It's funny to think that a mere 20 or so years ago, ethernet wasn't even a foregone conclusion. The first time I connected a computer to a network, I had to make sure I was plugging into the ethernet and not the token ring. Now here we are pushing transport layer fixes into the application layer just because we don't want to have to explain to sysadmins how to reconfigure their firewalls?

digi_owl 11y ago
I think it is aimed more at home users that are stuck behind a NAT the ISP controls than sysadmins.
- rkangel 11y ago
  So we're not fixing the transport layer due to limitations in the network layer that have been fixed, we're just not using them.
  - digi_owl 11y ago
    Pretty much. I find myself suspecting that we will see more and more sofware "fixes" piled on top to avoid doing mass hardware replacements...
    - a3n 11y ago
      Yet it can be done. Digital over the air TV for example.
      - digi_owl 11y ago
        One thing when it is done at the consumers expense, another when the company has to cover it.
  - dtech 11y ago
    You mean IPv6? That suffers from exactly the same problem: ISP's just don't give a damn and don't roll it out. You cannot expect your average internet user to know about these things, so they get away with it.

cnvogel 11y ago

> I'm curious why it ended up in HTTP instead of TCP. ...or just run HTTP over SCTP. http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#Message-based_multi-streaming http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...

acdha 11y ago
Check out http://en.wikipedia.org/wiki/Stream_Control_Transmission_Protocol#Implementations http://en.wikipedia.org/wiki/Stream_Control_Transmission_Pro...: It's not supported at all on Windows or OS X and the implementations everywhere else are far less tested than TCP. That's a large immediate problem, particularly since you have to update the kernel to fix it, and it also means that many, many intermediaries (home routers, proxies, corporate firewalls, etc.) have never been pushed to support it at all, let alone as well as commonly-used protocols. The good news is that the semantics are close enough that if the situation improves for both client support and, critically, intermediaries it would be relatively straightforward to migrate to a HTTP/2-over-SCTP hybrid if proved better in some way.
- cnvogel 11y ago
  I'm completely aware of the fact that SCTP isn't in the native Windows networking stack, and I'll take your word that it's not in OSX. But as long as we put up all these Frankenstein solutions of handling things up-in-the-stack because firewall admins can't be bothered up upgrade or configure their stuff correctly, we are just not putting out enough pressure that this will get changed! And even if we now introduce this sad compromise that is HTTP/2, there also will be a lot of proxy/firewall appliances that block HTTP/2, and there will be the equivalent of the government-entity of large corporation that was stuck on IE4/WinXP until 2014 using an outdated web-browser or intranet server. So, maybe we should try to get incompatible protocols out much earlier and if they turn out to have merit, we could enable them on released products and have Chrome/Firefox put up a nagging reminder: "Your web-experience would be much improved (or: this premium content could be watched at higher resolution, or security to your banking website, or...) if your network infrastructure would support SCTP/IPv6/DNSsec/, please ask your ISP or Administrator".

hobohacker 11y ago

To answer your specific question about why not in TCP, it's not deployable. Here's why: * In order to get multiplexing and other features introduced with HTTP/2, you need to change protocol framing. However, this means that the protocol is no longer backwards compatible. There are many ways to roll out non-backwards-compatible changes, but for TCP, they were deemed unacceptable. For example, you could negotiate the protocol change via a TCP extension. However, TCP extensions are known not to be widely deployable (https://www.imperialviolet.org/binary/ecntest.pdf https://www.imperialviolet.org/binary/ecntest.pdf) over the public internet. You could use a specific port, but that doesn't traverse enough middleboxes on the public internet (http://www.ietf.org/mail-archive/web/tls/current/msg05593.html http://www.ietf.org/mail-archive/web/tls/current/msg05593.ht...). Yada yada. * More importantly, TCP is generally implemented in the OS. That means, updating a protocol requires OS updates. That means we need both server and client OS updates in order to speak the new protocol. If you look at the very sad numbers on Windows versions uptake, Android version uptake, etc, you'll understand why many people don't want to wait for all OSes to update in order to take advantage of new protocol features. To quote Roberto (http://www.ietf.org/mail-archive/web/tsvwg/current/msg12184.html http://www.ietf.org/mail-archive/web/tsvwg/current/msg12184....) in his response to IETF transport protocol folks asking why application protocol folks are hacking around the transport: """ I understand the points you make, and am sympathetic. I feel the same way when I see people abusing HTTP to provide async notifications, etc. The fact is, however, if it isn't deployed, no matter how nice it would theoretically be when it is, it isn't useful and people WILL work around the problem. That is true regardless of whether the problem is at the application layer (e.g. HTTP), or at the transport layer (e.g. TCP), or elsewhere. The primary motivation is to get things working, and making things that lack redundancy, or are elegant comes in at a distant second. Deployed is the most important feature, thus things which quickly move protocols and protocol changes from theoretical to deployed are by far the most important things. The longer this takes, the more likely that the work-around becomes standard practice, at which point we've all "lost" the game. -=R """ Basically, this is another instance of Linus's quote (https://lkml.org/lkml/2009/3/25/632 https://lkml.org/lkml/2009/3/25/632): "Theory and practice sometimes clash. And when that happens, theory loses. Every single time." Theoretically, it'd be far more elegant to fix this in the transport layer. But in practice, it doesn't work. Except, maybe if you implement on top of UDP (e.g. QUIC), since that would be in user-space and firewalls don't filter out UDP as much. For more extensive analysis of HTTP/2's deployability options, amongst other considerations, check out my post on HTTP/2 considerations: https://insouciant.org/tech/http-slash-2-considerations-and-tradeoffs/#Deployability https://insouciant.org/tech/http-slash-2-considerations-and-....

AndrewDucker 11y ago

STCP is what you want. Sadly, Microsoft have completely failed to support it, so it's not going to happen.

megaman821 11y ago

Out of all the options putting transport layer concerns in the HTTP protocol is my least favorite. Google built a multiplexed stream transport over UDP called QUIC. HTTP over QUIC gives you most of what HTTP2 does. Also there is a transport layer protocol called SCTP. Again, HTTP over SCTP gives you most of what HTTP2 does.

MertsA 11y ago
HTTP over SCTP give you more than what HTTP2 does. I just wish Google would have made SPDY only work over SCTP as a measure to push for widespread adoption. Imagine finally having a seamless connection for every application on mobile devices where it would switch from wifi to 4G without any serious latency or dropped connections.