5 ms·

> This isn't hypertext. That ship has long sailed. HTTP is no longer just hypertext -- it's a ubiquitous and well-understood transport for RPCs, with lots of t

> This isn't hypertext. That ship has long sailed. HTTP is no longer just hypertext -- it's a ubiquitous and well-understood transport for RPCs, with lots of tooling and infrastructure support across the Internet. I think at its core, HTTP abstracts away some of what you would want in a request-response protocol, but not really all of it. But _simply because of the ubiquity of tooling_, you get a whole lot of other stuff for free (load balancing, DDoS detection, in-browser debugging, etc.) > Is it so complicated to work with a duplex-stream? Could a library alleviate the complications? Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. IMO, your language's go-to HTTP library is that library that alleviates the complications. All this said, I personally think gRPC is the right approach for RPCs, and I hope to see broader adoption of it over the coming years.

jolmg 7y ago
> Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. Isn't it the same thing, but simpler and more flexible? You write a message, and then you read a reply, rinse and repeat. Could you expand on such things that are complicated with TCP that HTTP helps with?
- jcelerier 7y ago
  > You write a message, and then you read a reply, rinse and repeat. well, no, you don't "read a message" with TCP, because it is a stream-oriented protocol. you read bytes that keep coming and wait until you have enough or reach a delimiter that you chose.
  - jolmg 7y ago
    Yes, that's what I meant. I would expect a good parser to hang until the message is complete as determined by the syntax of the protocol and return that, leaving the stream at the point the next message should start. I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more.
    - jcelerier 7y ago
      > I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more. Even if you have good parsing libraries, parser combinators, etc, you still need to write the grammar - which can be hard if you are not accustomed to that... years after I still find an occasional bug in grammars I've written in the past. Compare this to JSON over HTTP which ends up basically being var xhr = new XMLHttpRequest(); xhr.open('GET', "https://my/api.json", true); xhr.onreadystatechange = function(e) { var response = JSON.parse(xhr.responseText); } what other format allows to get objects over the network that easily, without any special libraries ?
- zeroxfe 7y ago
  When you're working with sockets there's lots to think about that you can simply ignore with an HTTP library -- connection setup and teardown, buffering, serialization, byte ordering etc. But more importantly, you'll have to build your own status/error handling, cache control, content encoding, TLS etc. (Simply, doing TLS correctly itself is not easy.) To be clear, I'm not saying that there's no place for socket programming -- there definitely is. I've spent a good part of my life doing obscure network programming on all kinds of platforms, so I try not to use TCP/UDP unless I have a really good reason. (It's kinda the same reason I won't use C or C++ -- too much to worry about for most general purposes.)
  - jolmg 7y ago
    Thanks for the reply. This gives me a better understanding of the issue here. > connection setup and teardown If there's no benefit in avoiding setting up and tearing down a connection with each request like one typically does with HTTP, then I can see how it might be an annoyance compared to making a single function call from a library that makes the request and returns the response. > buffering A parser should deal with that. It could be JSON parser. > serialization That's still a problem with HTTP. People typically use JSON for this and it doesn't depend on HTTP. > byte ordering parser/JSON's job. > your own status/error handling, cache control, content encoding That can be beneficial or it can be problematic. If you need such things, then yeah, it's cool to not have to think about making your own. However, if you don't or if your needs don't align with what HTTP provides, that can be problematic. For example, from a cursory search I see the following in the JMAP spec: > Implementors must take care to avoid inappropriate caching of the session object at the HTTP layer. Since the client should only refetch when it detects there is a change (via the sessionState property of an API response), it is RECOMMENDED to disable HTTP caching altogether, for example by setting `Cache-Control: no-cache, no-store, must-revalidate` on the response. So, HTTP provides caching, but not the caching mechanism that's needed, so it's recommended to explicitly disable it completely. With respect to status and error handling, it seems JMAP still had to define their own errors to be described in JSON[1], so not a perfect fit there either. On the content encoding, I'm not sure if JMAP would use something other than application/json, so it seems like something that wouldn't need to be specified if one didn't use HTTP. I know the discussion is not really about JMAP. I'm just using JMAP as an example on the above. > TLS etc. (Simply, doing TLS correctly itself is not easy.) I would hope that there's a library equivalent of the shell's: openssl s_client -connect $host:$port I don't know why a library would have to be much more complicated. Maybe I'm being naive, but I think that if one wouldn't specify special TLS parameters to an HTTP library, then there's no reason why one would need to if they skip the HTTP library and go straight to using a TLS library. IOW, I don't see why an HTTP library would be more helpful on setting up TLS than using a TLS library directly. [1] https://jmap.io/spec-core.html#initial-jmap-error-codes-registry https://jmap.io/spec-core.html#initial-jmap-error-codes-regi...
- tracker1 7y ago
  Try adding TLS support, then debugging the message transport.

jolmg 7y ago

> Yes, TCP streams are very complicated to work with (in my experience) -- there are way too many failure modes and corner cases to consider. Isn't it the same thing, but simpler and more flexible? You write a message, and then you read a reply, rinse and repeat. Could you expand on such things that are complicated with TCP that HTTP helps with?

jcelerier 7y ago
> You write a message, and then you read a reply, rinse and repeat. well, no, you don't "read a message" with TCP, because it is a stream-oriented protocol. you read bytes that keep coming and wait until you have enough or reach a delimiter that you chose.
- jolmg 7y ago
  Yes, that's what I meant. I would expect a good parser to hang until the message is complete as determined by the syntax of the protocol and return that, leaving the stream at the point the next message should start. I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more.
  - jcelerier 7y ago
    > I'm getting the feeling that the popularity of JSON and HTTP come by good part from the lack of good parsing libraries in many languages. People generally only know how to work with what regexes are able to handle and nothing more. Even if you have good parsing libraries, parser combinators, etc, you still need to write the grammar - which can be hard if you are not accustomed to that... years after I still find an occasional bug in grammars I've written in the past. Compare this to JSON over HTTP which ends up basically being var xhr = new XMLHttpRequest(); xhr.open('GET', "https://my/api.json", true); xhr.onreadystatechange = function(e) { var response = JSON.parse(xhr.responseText); } what other format allows to get objects over the network that easily, without any special libraries ?
zeroxfe 7y ago
When you're working with sockets there's lots to think about that you can simply ignore with an HTTP library -- connection setup and teardown, buffering, serialization, byte ordering etc. But more importantly, you'll have to build your own status/error handling, cache control, content encoding, TLS etc. (Simply, doing TLS correctly itself is not easy.) To be clear, I'm not saying that there's no place for socket programming -- there definitely is. I've spent a good part of my life doing obscure network programming on all kinds of platforms, so I try not to use TCP/UDP unless I have a really good reason. (It's kinda the same reason I won't use C or C++ -- too much to worry about for most general purposes.)
- jolmg 7y ago
  Thanks for the reply. This gives me a better understanding of the issue here. > connection setup and teardown If there's no benefit in avoiding setting up and tearing down a connection with each request like one typically does with HTTP, then I can see how it might be an annoyance compared to making a single function call from a library that makes the request and returns the response. > buffering A parser should deal with that. It could be JSON parser. > serialization That's still a problem with HTTP. People typically use JSON for this and it doesn't depend on HTTP. > byte ordering parser/JSON's job. > your own status/error handling, cache control, content encoding That can be beneficial or it can be problematic. If you need such things, then yeah, it's cool to not have to think about making your own. However, if you don't or if your needs don't align with what HTTP provides, that can be problematic. For example, from a cursory search I see the following in the JMAP spec: > Implementors must take care to avoid inappropriate caching of the session object at the HTTP layer. Since the client should only refetch when it detects there is a change (via the sessionState property of an API response), it is RECOMMENDED to disable HTTP caching altogether, for example by setting `Cache-Control: no-cache, no-store, must-revalidate` on the response. So, HTTP provides caching, but not the caching mechanism that's needed, so it's recommended to explicitly disable it completely. With respect to status and error handling, it seems JMAP still had to define their own errors to be described in JSON[1], so not a perfect fit there either. On the content encoding, I'm not sure if JMAP would use something other than application/json, so it seems like something that wouldn't need to be specified if one didn't use HTTP. I know the discussion is not really about JMAP. I'm just using JMAP as an example on the above. > TLS etc. (Simply, doing TLS correctly itself is not easy.) I would hope that there's a library equivalent of the shell's: openssl s_client -connect $host:$port I don't know why a library would have to be much more complicated. Maybe I'm being naive, but I think that if one wouldn't specify special TLS parameters to an HTTP library, then there's no reason why one would need to if they skip the HTTP library and go straight to using a TLS library. IOW, I don't see why an HTTP library would be more helpful on setting up TLS than using a TLS library directly. [1] https://jmap.io/spec-core.html#initial-jmap-error-codes-registry https://jmap.io/spec-core.html#initial-jmap-error-codes-regi...
tracker1 7y ago
Try adding TLS support, then debugging the message transport.