10 ms·

Why HTTP?

70 points by TimothyFitz 18y ago

jacquesm 18y ago
rsync uses it's own protocol (when run as a daemon), so does ssh and a whole slew of others the common element is that http can not provide these applications with the functionality that they seek. I think that the golden rule should be something like: If you can't do it on http you're free to roll your own but if you can do it over http then you really should.
- pmjordan 18y ago
  Interestingly, rsync can be used via an HTTP proxy, so presumably the protocol gets divided into HTTP requests. Doesn't seem too crazy really: HTTP supports partial downloads and uploads, and the rolling hashes can be easily be encoded in HTTP requests, too.
  - jacquesm 18y ago
    Yes, but the more useful options that rsync supports, such as all the file manipulation gear won't work via http so you'll end up with a subset of the functionality. interesting development: http://zsync.moria.org.uk/paper/ http://zsync.moria.org.uk/paper/
- whacked_new 18y ago
  > if you can do it over http then you really should There seems to be some wisdom here that I am not extracting successfully. It seems HTTP "can" in most cases. But where it can, do you really want to use it? Technically you could also write a db engine that reads over HTTP, but this must be a terrible idea; what would be the argument here that counters the rule of thumb?
  - jacquesm 18y ago
    can: - without getting really tortuous, but a little bit rethinking your problem should be fine - without sacrificing a large amount of speed or functionality - without compromising your design goals
    - whacked_new 18y ago
      Safe to say then, that it's a case by case and so, not so much a rule of thumb?
  - davidmathers 18y ago
    "Technically you could also write a db engine that reads over HTTP, but this must be a terrible idea" If you go to this url you can probably find some contact info to tell the NY Times people how terrible their idea is: http://code.nytimes.com/projects/dbslayer http://code.nytimes.com/projects/dbslayer
CalmQuiet 18y ago
OK, so I'm a noob about protocols. Can someone help me read between the lines? Does TF's defense/advocacy for HTTP reflect a "threat" to its universality? In any case, what protocols are developing as alternatives (besides the obvious: FTP, XMPP) - and for what purposes that HTTP cannot serve well?
- moe 18y ago
  Well, it seems the author drank a bit too much of the webservices kool-aid. HTTP is just one protocol amongst many. It was never meant to be "universal". The only reason why it's being abused for even the most unsuitable tasks nowadays is because when all you have is a hammer then everything starts to look like a thumb... There are many purposes where HTTP is not suitable at all. Realtime audio/video comes to mind - UDP is commonly used here because even the TCP latency is too much. Anything where you need a persistent, bi-directional connection is not a good match either. Anything where you need to push from server to client. In short: Anything that doesn't fit into the request/response paradigm. In fact, many consider HTTP in its current incarnation to not even be suitable for the interactivity that we expect of modern webapps. But ofcourse that doesn't stop the truly enlightened. Hence we got abominations like "Comet", a persistent stream-socket emulated on top of an inherently request/response-based protocol. Or RSS, which is nothing more than Usenet done really, really wrong - all lessons unlearned. Sorry, got a bit carried away. But you get the idea I guess.
  - iamwil 18y ago
    UDP and TCP aren't comparable to HTTP. HTTP is an application layer protocol, which rides on top of UDP and TCP, which are transport layer protocols. Be careful when you get carried away.
    - moe 18y ago
      Read my sentence again, I wrote: UDP is commonly used here because even the TCP latency is too much. I meant to say: They not only avoid HTTP - they even avoid TCP, too. Well, the phrasing was not ideal, I'll give you that.
axod 18y ago
OTOH, there are several things HTTP does ridiculously badly. It all depends on what you're building. Evaluate the case, and sure, if you need a more specific protocol, design one yourself. It's simple to layer on encryption, compression if you need those. Also the argument slightly reeks of the same "Everything knows how to deal with it!" that caused some people to use XML for everything regardless of how well it fit the job.
- skolor 18y ago
  The main point seemed to be that you wouldn't need to layer much, if anything, onto HTTP. That is not necessarily that much of an advantage, depending on your environment, but should be worth considering.
- abstractbill 18y ago
  God I hated those arguments so much. I especially hated being told I should be using xml "because the parser is already written" as if writing a parser was the difficult part of... well anything really ;-)
- EliAndrewC 18y ago
  HTTP isn't perfect, and there are plenty of times when a custom protocol makes sense. However, I think the author is arguing against the "everything needs its own custom protocol" mindset which is all to prevalent in the corporate world. In my work for USPS, I've encountered numerous custom protocols, most of which are far inferior to vanilla HTTP and which cause endless headaches due to poor documentation, debuggability, extensibility, bugginess, etc. So I agree with the author and my own approach to protocols is to start off with the assumption that I'm using HTTP and only use something else if there's a good reason why HTTP isn't appropriate (performance being the most common dealbreaker).
  - axod 18y ago
    You make it sound, as the author does, like HTTP is applicable to anything. It's not. It's a very specific request/response protocol, which suits some specific purposes. >> "start off with the assumption that I'm using HTTP" That's ridiculous. So if you're writing a multiplayer real-time game, you'd assume you're going to use HTTP?
jff 18y ago
Just do everything over 9p
ankhmoop 18y ago
I recently implemented an event logging server, intended to process a massive, concurrent number of event messages from a large clusters of systems. We choose to implement a CPU efficient binary protocol, automatic client-side queuing of outbound messages if a server failed, and automatic -client-side- fail-over to the next available server. The initial implementation, with no time spent on profiling/optimization, was able to receive and process 25,000 event messages per second from a single client, and scale up clients/cores relatively linearly. I can't even begin to fathom solving this problem with HTTP, or why the 'features' of HTTP listed (proxies, load balancers, web browsers, 'extensive hardware', etc) would be an improvement over the relatively simple and highly scalable single-tier implementation we created. Clearly, HTTP works well for some things, but "just use HTTP, your life will be simpler" is a naive axiom.

70 points by TimothyFitz 18y ago

rsync uses it's own protocol (when run as a daemon), so does ssh and a whole slew of others the common element is that http can not provide these applications with the functionality that they seek. I think that the golden rule should be something like: If you can't do it on http you're free to roll your own but if you can do it over http then you really should.

pmjordan 18y ago
Interestingly, rsync can be used via an HTTP proxy, so presumably the protocol gets divided into HTTP requests. Doesn't seem too crazy really: HTTP supports partial downloads and uploads, and the rolling hashes can be easily be encoded in HTTP requests, too.
- jacquesm 18y ago
  Yes, but the more useful options that rsync supports, such as all the file manipulation gear won't work via http so you'll end up with a subset of the functionality. interesting development: http://zsync.moria.org.uk/paper/ http://zsync.moria.org.uk/paper/
whacked_new 18y ago
> if you can do it over http then you really should There seems to be some wisdom here that I am not extracting successfully. It seems HTTP "can" in most cases. But where it can, do you really want to use it? Technically you could also write a db engine that reads over HTTP, but this must be a terrible idea; what would be the argument here that counters the rule of thumb?
- jacquesm 18y ago
  can: - without getting really tortuous, but a little bit rethinking your problem should be fine - without sacrificing a large amount of speed or functionality - without compromising your design goals
  - whacked_new 18y ago
    Safe to say then, that it's a case by case and so, not so much a rule of thumb?
- davidmathers 18y ago
  "Technically you could also write a db engine that reads over HTTP, but this must be a terrible idea" If you go to this url you can probably find some contact info to tell the NY Times people how terrible their idea is: http://code.nytimes.com/projects/dbslayer http://code.nytimes.com/projects/dbslayer

CalmQuiet 18y ago

OK, so I'm a noob about protocols. Can someone help me read between the lines? Does TF's defense/advocacy for HTTP reflect a "threat" to its universality? In any case, what protocols are developing as alternatives (besides the obvious: FTP, XMPP) - and for what purposes that HTTP cannot serve well?

moe 18y ago
Well, it seems the author drank a bit too much of the webservices kool-aid. HTTP is just one protocol amongst many. It was never meant to be "universal". The only reason why it's being abused for even the most unsuitable tasks nowadays is because when all you have is a hammer then everything starts to look like a thumb... There are many purposes where HTTP is not suitable at all. Realtime audio/video comes to mind - UDP is commonly used here because even the TCP latency is too much. Anything where you need a persistent, bi-directional connection is not a good match either. Anything where you need to push from server to client. In short: Anything that doesn't fit into the request/response paradigm. In fact, many consider HTTP in its current incarnation to not even be suitable for the interactivity that we expect of modern webapps. But ofcourse that doesn't stop the truly enlightened. Hence we got abominations like "Comet", a persistent stream-socket emulated on top of an inherently request/response-based protocol. Or RSS, which is nothing more than Usenet done really, really wrong - all lessons unlearned. Sorry, got a bit carried away. But you get the idea I guess.
- iamwil 18y ago
  UDP and TCP aren't comparable to HTTP. HTTP is an application layer protocol, which rides on top of UDP and TCP, which are transport layer protocols. Be careful when you get carried away.
  - moe 18y ago
    Read my sentence again, I wrote: UDP is commonly used here because even the TCP latency is too much. I meant to say: They not only avoid HTTP - they even avoid TCP, too. Well, the phrasing was not ideal, I'll give you that.

axod 18y ago

OTOH, there are several things HTTP does ridiculously badly. It all depends on what you're building. Evaluate the case, and sure, if you need a more specific protocol, design one yourself. It's simple to layer on encryption, compression if you need those. Also the argument slightly reeks of the same "Everything knows how to deal with it!" that caused some people to use XML for everything regardless of how well it fit the job.

skolor 18y ago
The main point seemed to be that you wouldn't need to layer much, if anything, onto HTTP. That is not necessarily that much of an advantage, depending on your environment, but should be worth considering.
abstractbill 18y ago
God I hated those arguments so much. I especially hated being told I should be using xml "because the parser is already written" as if writing a parser was the difficult part of... well anything really ;-)
EliAndrewC 18y ago
HTTP isn't perfect, and there are plenty of times when a custom protocol makes sense. However, I think the author is arguing against the "everything needs its own custom protocol" mindset which is all to prevalent in the corporate world. In my work for USPS, I've encountered numerous custom protocols, most of which are far inferior to vanilla HTTP and which cause endless headaches due to poor documentation, debuggability, extensibility, bugginess, etc. So I agree with the author and my own approach to protocols is to start off with the assumption that I'm using HTTP and only use something else if there's a good reason why HTTP isn't appropriate (performance being the most common dealbreaker).
- axod 18y ago
  You make it sound, as the author does, like HTTP is applicable to anything. It's not. It's a very specific request/response protocol, which suits some specific purposes. >> "start off with the assumption that I'm using HTTP" That's ridiculous. So if you're writing a multiplayer real-time game, you'd assume you're going to use HTTP?

jff 18y ago

Just do everything over 9p

ankhmoop 18y ago

I recently implemented an event logging server, intended to process a massive, concurrent number of event messages from a large clusters of systems. We choose to implement a CPU efficient binary protocol, automatic client-side queuing of outbound messages if a server failed, and automatic -client-side- fail-over to the next available server. The initial implementation, with no time spent on profiling/optimization, was able to receive and process 25,000 event messages per second from a single client, and scale up clients/cores relatively linearly. I can't even begin to fathom solving this problem with HTTP, or why the 'features' of HTTP listed (proxies, load balancers, web browsers, 'extensive hardware', etc) would be an improvement over the relatively simple and highly scalable single-tier implementation we created. Clearly, HTTP works well for some things, but "just use HTTP, your life will be simpler" is a naive axiom.