6 ms·

Why is this happening? Forgetting ubiquity of tooling, is there something technical about HTTP that makes it so attractive? I get JSON. People don't want to wr

by jolmg 8y ago

Why is this happening? Forgetting ubiquity of tooling, is there something technical about HTTP that makes it so attractive? I get JSON. People don't want to write text-parsers. But what's attractive about this abuse of HTTP? This isn't hypertext. Is it so complicated to work with a duplex-stream? Could a library alleviate the complications? Could it be that developers are more familiar with HTTP than with TCP and so TCP scares them even though it's simpler to use?

icebraining 8y ago
https://jmap.io/#why-use-https/json https://jmap.io/#why-use-https/json?
- jolmg 8y ago
  Everything mentioned there is because of its ubiquity, nothing about how the way HTTP operates makes it more suitable. What you link even says: > the encoding/transport part of JMAP [i.e. HTTP] is not core to its operation, so future specifications could easily add alternatives So HTTP is useless for this except for the fact that everyone uses it.
  - detaro 8y ago
    "possible to replace with alternatives" != "useless"
    - jolmg 8y ago
      By useless I meant that the use of HTTP doesn't contribute anything essential. Though maybe I could be wrong? But that's what I understood by "is not core to its operation". I understand it to just wrap the JSON in HTTP for the benefits that come from its ubiquity in use and not so much because of merits of the mechanisms described in the HTTP spec.
      - detaro 8y ago
        No, it's just the most widely supported protocol of that kind, so why not use it? Is there anything specific against it/how do you answer for another protocol the question of "why not HTTP, which is similar and has better support?"? (Caveat: I think you'd ideally want http/2, which isn't as widely supported) EDIT: actually, one point: you can talk JMAP over HTTP(S) directly from a browser without a translating server.
      - Arnt 8y ago
        That's right. That was one of the first ~five sentences that were spoken about JMAP. Ubiquiti is a feature. Not having to explain the caching model to people is a feature.
      - afiori 8y ago
        Which can be read also as: HTTP allows to embed arbitrary semantics with close to no friction. Which sounds like a really good point together with the tooling and ubiquity
jcranmer 8y ago
The honest reason boils down to "there's a lot more closing down of generic TCP than HTTP in firewalls and application privilege frameworks." Email has the other unfortunate property that it has arbitrary binary data (message bodies) mixed in with a UTF-8 control stream, which makes SMTP/IMAP/POP/NNTP especially annoying to implement in languages that draw very strong distinctions between binary and text strings, since you can't just wrap the TCP stream in a UTF-8 decoder/encoder and be done with it.
satyanash 8y ago
> Is it so complicated to work with a duplex-stream? It's not that duplex-stream is complicated, it's just that request/response semantics are much more suited for most use cases. Where duplex streams make sense, we now have websockets. Even then, on top of that you'll end up implementing stateful request/response semantics because you need some structure unless you're literally dumping binary data into the socket.
- afiori 8y ago
  Another difference is that the request/response pattern is much better suited for slow/unreliable connections and chaching.
zeroxfe 8y ago
> This isn't hypertext. That ship has long sailed. HTTP is no longer just hypertext -- it's a ubiquitous and well-understood transport for RPCs, with lots of tooling and infrastructure support across the Internet. I think at its core, HTTP abstracts away some of what you would want in a request-response protocol, but not really all of it. But _simply because of the ubiquity of tooling_, you get a whole lot of other stuff for free (load balancing, DDoS detection, in-browser debugging, etc.) > Is it so complicated to work with a duplex-stream? Could a library alleviate the complications? Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. IMO, your language's go-to HTTP library is that library that alleviates the complications. All this said, I personally think gRPC is the right approach for RPCs, and I hope to see broader adoption of it over the coming years.
- jolmg 8y ago
  > Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. Isn't it the same thing, but simpler and more flexible? You write a message, and then you read a reply, rinse and repeat. Could you expand on such things that are complicated with TCP that HTTP helps with?
  - jcelerier 8y ago
    > You write a message, and then you read a reply, rinse and repeat. well, no, you don't "read a message" with TCP, because it is a stream-oriented protocol. you read bytes that keep coming and wait until you have enough or reach a delimiter that you chose.
    - jolmg 8y ago
      Yes, that's what I meant. I would expect a good parser to hang until the message is complete as determined by the syntax of the protocol and return that, leaving the stream at the point the next message should start. I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more.
deleted 8y ago
[deleted]
vbezhenar 8y ago
1. Everyone does that, why should I do different thing? 2. I've heard about corporate firewalls which close everything and nobody has power to overcome that, so I'll tunnel everything via HTTP(S) and forget about those horrors. 3. HTTPS provide absolutely hassle-free encryption with reverse-proxies. I don't even need to think about it. Using TLS library sounds too complex and I must make some real effort to implement it. Rolling out my own encryption? Everyone will scream! 4. HTTP/1.1 protocol is really simple. There are some hidden gotchas in the corners, but basically it's headers, empty line and then either bytes or stream of chunks, that's all about it. So I can pass routing information (URL path), I can pass some metadata in headers and I can pass either a singular message or a streamed content. And with HTTP libraries in every language you'll get that functionality almost for free, both for server and client. 5. Theoretically you can leverage HTTP caching on various levels. Might be tricky, but certainly easier rather than implementing it from the scratch. And with things like Cloudflare you can't really build anything similar. 6. Browsers can only talk HTTP. So even if I don't need it know, if I would ever need to build web application, I'll have to provide HTTP bridge. And web applications are really common nowadays. You can't really make IMAP client in JavaScript running in browser. But you probably can do it with JMAP.
- afiori 8y ago
  Do you also have a list for why many people oppose HTTP (or better what is sometimes called "shoehorning everything in a HTTP shaped box")? I guess that the versioning and advanced features are a stark contrast with versionless TCP from the '80, but I am interested in hearing more opinions on the matter as I am ignorant of any dealbreaker regarding HTTP.
  - vbezhenar 8y ago
    The hardest thing with HTTP is to send server-initiated messages to client. There are multiple approaches (websockets, SSE, client polling) but they are not simple. Also HTTP carries some overhead. It's not significant, something like 50-100 bytes, but it might matter for a lot of tiny messages, I guess. Another factor is: those intermediate proxies might break your app. They might cache your responses even if you don't want that, so your application will act wierdly. Of course any intermediate proxy might break any protocol...
dragonwriter 8y ago
> Forgetting ubiquity of tooling, is there something technical about HTTP that makes it so attractive? Slightly different than ubiquity of tooling is firewall policies: HTTP(S) is the protocol most likely to get where it needs to without someone configuring something to allow it, so tunnelling everything over HTTP(S) is the path of least resistance to widespread use.
p_l 8y ago
I'll give a potentially flamebaity response... But it's because reality appears to say that OSI protocols were right and TCP was wrong. Plain TCP essentially emulates a full duplex serial port. Applications need more on top of that. Methods for framing, attaching Metadata, data flows, secure transport layers, etc. So HTTP which provides a modicum of those options ends up being default, because it's available. For comparison, OSI defined various components to handle those elements and a proper OSI stack should have then available to application developer. Another example would be Symbolics Lisp Machines (Genera) where the network layer provided a bunch of mid-layer protocols, like one which provided messages that were formed of basic typed data structures. So people would use those to quickly build custom protocols for their applications, thanks to not having to deal with low level details. Various Middleware also come to mind. TCP is not enough.
tracker1 8y ago
You can write a web front end that communicates with the mail server API directly.

by jolmg 8y ago

icebraining 8y ago

https://jmap.io/#why-use-https/json https://jmap.io/#why-use-https/json?

jolmg 8y ago
Everything mentioned there is because of its ubiquity, nothing about how the way HTTP operates makes it more suitable. What you link even says: > the encoding/transport part of JMAP [i.e. HTTP] is not core to its operation, so future specifications could easily add alternatives So HTTP is useless for this except for the fact that everyone uses it.
- detaro 8y ago
  "possible to replace with alternatives" != "useless"
  - jolmg 8y ago
    By useless I meant that the use of HTTP doesn't contribute anything essential. Though maybe I could be wrong? But that's what I understood by "is not core to its operation". I understand it to just wrap the JSON in HTTP for the benefits that come from its ubiquity in use and not so much because of merits of the mechanisms described in the HTTP spec.
    - detaro 8y ago
      No, it's just the most widely supported protocol of that kind, so why not use it? Is there anything specific against it/how do you answer for another protocol the question of "why not HTTP, which is similar and has better support?"? (Caveat: I think you'd ideally want http/2, which isn't as widely supported) EDIT: actually, one point: you can talk JMAP over HTTP(S) directly from a browser without a translating server.
    - Arnt 8y ago
      That's right. That was one of the first ~five sentences that were spoken about JMAP. Ubiquiti is a feature. Not having to explain the caching model to people is a feature.
    - afiori 8y ago
      Which can be read also as: HTTP allows to embed arbitrary semantics with close to no friction. Which sounds like a really good point together with the tooling and ubiquity

jcranmer 8y ago

The honest reason boils down to "there's a lot more closing down of generic TCP than HTTP in firewalls and application privilege frameworks." Email has the other unfortunate property that it has arbitrary binary data (message bodies) mixed in with a UTF-8 control stream, which makes SMTP/IMAP/POP/NNTP especially annoying to implement in languages that draw very strong distinctions between binary and text strings, since you can't just wrap the TCP stream in a UTF-8 decoder/encoder and be done with it.

satyanash 8y ago

> Is it so complicated to work with a duplex-stream? It's not that duplex-stream is complicated, it's just that request/response semantics are much more suited for most use cases. Where duplex streams make sense, we now have websockets. Even then, on top of that you'll end up implementing stateful request/response semantics because you need some structure unless you're literally dumping binary data into the socket.

afiori 8y ago
Another difference is that the request/response pattern is much better suited for slow/unreliable connections and chaching.

zeroxfe 8y ago

> This isn't hypertext. That ship has long sailed. HTTP is no longer just hypertext -- it's a ubiquitous and well-understood transport for RPCs, with lots of tooling and infrastructure support across the Internet. I think at its core, HTTP abstracts away some of what you would want in a request-response protocol, but not really all of it. But _simply because of the ubiquity of tooling_, you get a whole lot of other stuff for free (load balancing, DDoS detection, in-browser debugging, etc.) > Is it so complicated to work with a duplex-stream? Could a library alleviate the complications? Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. IMO, your language's go-to HTTP library is that library that alleviates the complications. All this said, I personally think gRPC is the right approach for RPCs, and I hope to see broader adoption of it over the coming years.

jolmg 8y ago
> Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. Isn't it the same thing, but simpler and more flexible? You write a message, and then you read a reply, rinse and repeat. Could you expand on such things that are complicated with TCP that HTTP helps with?
- jcelerier 8y ago
  > You write a message, and then you read a reply, rinse and repeat. well, no, you don't "read a message" with TCP, because it is a stream-oriented protocol. you read bytes that keep coming and wait until you have enough or reach a delimiter that you chose.
  - jolmg 8y ago
    Yes, that's what I meant. I would expect a good parser to hang until the message is complete as determined by the syntax of the protocol and return that, leaving the stream at the point the next message should start. I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more.

deleted 8y ago

[deleted]

vbezhenar 8y ago

1. Everyone does that, why should I do different thing? 2. I've heard about corporate firewalls which close everything and nobody has power to overcome that, so I'll tunnel everything via HTTP(S) and forget about those horrors. 3. HTTPS provide absolutely hassle-free encryption with reverse-proxies. I don't even need to think about it. Using TLS library sounds too complex and I must make some real effort to implement it. Rolling out my own encryption? Everyone will scream! 4. HTTP/1.1 protocol is really simple. There are some hidden gotchas in the corners, but basically it's headers, empty line and then either bytes or stream of chunks, that's all about it. So I can pass routing information (URL path), I can pass some metadata in headers and I can pass either a singular message or a streamed content. And with HTTP libraries in every language you'll get that functionality almost for free, both for server and client. 5. Theoretically you can leverage HTTP caching on various levels. Might be tricky, but certainly easier rather than implementing it from the scratch. And with things like Cloudflare you can't really build anything similar. 6. Browsers can only talk HTTP. So even if I don't need it know, if I would ever need to build web application, I'll have to provide HTTP bridge. And web applications are really common nowadays. You can't really make IMAP client in JavaScript running in browser. But you probably can do it with JMAP.

afiori 8y ago
Do you also have a list for why many people oppose HTTP (or better what is sometimes called "shoehorning everything in a HTTP shaped box")? I guess that the versioning and advanced features are a stark contrast with versionless TCP from the '80, but I am interested in hearing more opinions on the matter as I am ignorant of any dealbreaker regarding HTTP.
- vbezhenar 8y ago
  The hardest thing with HTTP is to send server-initiated messages to client. There are multiple approaches (websockets, SSE, client polling) but they are not simple. Also HTTP carries some overhead. It's not significant, something like 50-100 bytes, but it might matter for a lot of tiny messages, I guess. Another factor is: those intermediate proxies might break your app. They might cache your responses even if you don't want that, so your application will act wierdly. Of course any intermediate proxy might break any protocol...

dragonwriter 8y ago

> Forgetting ubiquity of tooling, is there something technical about HTTP that makes it so attractive? Slightly different than ubiquity of tooling is firewall policies: HTTP(S) is the protocol most likely to get where it needs to without someone configuring something to allow it, so tunnelling everything over HTTP(S) is the path of least resistance to widespread use.

p_l 8y ago

I'll give a potentially flamebaity response... But it's because reality appears to say that OSI protocols were right and TCP was wrong. Plain TCP essentially emulates a full duplex serial port. Applications need more on top of that. Methods for framing, attaching Metadata, data flows, secure transport layers, etc. So HTTP which provides a modicum of those options ends up being default, because it's available. For comparison, OSI defined various components to handle those elements and a proper OSI stack should have then available to application developer. Another example would be Symbolics Lisp Machines (Genera) where the network layer provided a bunch of mid-layer protocols, like one which provided messages that were formed of basic typed data structures. So people would use those to quickly build custom protocols for their applications, thanks to not having to deal with low level details. Various Middleware also come to mind. TCP is not enough.

tracker1 8y ago

You can write a web front end that communicates with the mail server API directly.