15 ms·
Building Protocols with HTTP
- ForHackernews 8y agoI really wish people would stop trying to cram everything into an HTTP-shaped box.
- sp4c3m0nk3y 8y agoI agree. What is wrong with creating new protocols?
- abricot 8y agoAnything outside of port 80/443 will be blocked at some point.
- nineteen999 8y agoBlocked by who? Are you talking within some specific context, eg. corporate networks? Because HTTP/HTTPS are both TCP based protocols with some very limiting characteristics for many applications. There are a whole lot of applications out there in widespread use that depend on UDP at the very least, eg. most multiplayer network games, just to use the simplest and most common example I can think of. Not everything can (or should) be stuffed into a HTTP transaction.
- gsich 8y agoYes, the blocking problem mostly affects corporate networks or shady wlan access points. Game developers don't really have a problem with using UDP/TCP on random ports.
- nineteen999 8y agoSure, but there's a bit of a stretch from there to the parent posters claim "anything outside of port 80/443 will be blocked at some point" as a justification not to develop new, non-HTTP based protocols.
- gsich 8y agoOf course, it's just a sloppy excuse.
- ori_b 8y agoOnly if nobody uses those ports. I want as much as possible to break for people who block those ports, and engage in other user-hostile actions.
- figgis 8y agoFamiliarity. I can either work with a protocol that has been around, in some way shape or form, for almost 30 years. That has matured, has a myriad of information relating to edge cases, is familiar to new and old developers, and is quick to develop for. Or I can create my own protocol which has none of those benefits besides possibly quick development.
- ori_b 8y ago> Familiarity. I can either work with a protocol that has been around, in some way shape or form, for almost 30 years. Yeah, using TCP directly is great, isn't it?
- gsich 8y agoStupid people block stuff. That's why (somewhat less stupid) people try to tunnel everything via HTTP.
- zaarn 8y agoMiddleboxes tend to not play well with non-HTTP content these days. Not long ago this was also the case for non-UDP/TCP protocols, this is even more cemented. The only times I know that non-HTTP works is video games and that is sorta hit-or-miss on one ISP I frequent, they block protocols they don't know for "security". Making it HTTPS is an easy way to get around any potential firewall problems and have it work easily in corporate settings.
- tptacek 8y agoYou can wish that all you want, but it isn't going to happen. HTTP is an extremely effective way to deploy new protocols; it is effectively the richest protocol design toolkit available universally to developers, and has the added benefit of playing well with middleboxes. Expect more "cramming" in the future, not less.
- tptacek 8y agoThere is some pretty dubious and I think wishful stuff in here, like "applications should align their usage closely as possible with web browsers", or the idea that ad-hoc protocols specified over HTTP should use links rather than fixed URLs. Does anyone have the backstory on this I-D? Anybody can write one, right?
- totalthrowaway 8y agoHe's not quite anyone. https://en.wikipedia.org/wiki/Mark_Nottingham https://en.wikipedia.org/wiki/Mark_Nottingham Perhaps this is meant as a public service, just to gather best practices. If you want to build your own thing, please go ahead, but don't call it HTTP.
- tptacek 8y agoFair enough, re Nottingham. Thanks for the link, that neatly answers my question. I strongly disagree that ignoring these suggested BCPs disqualifies an application as "HTTP". As far as I'm concerned, if a typical middlebox will reliably pass messages in it, it's HTTP.
- m0llusk 8y agoHTTP was intended to support web browsers, so there may be value in keeping compatibility. Application development, debugging, and scripting can all potentially be well served by having standard HTTP interfaces for interactions because this enables application operations to be performed or spoofed using web browsers.
- heipei 8y agoI can't say this trend surprises me. Years back I was an IETF meeting when people were already mentioning that the old saying "IP over everything, everything over IP" was slowly being phase out for "HTTP over everything, everything over HTTP". The thing to realize here is how much middleboxes sit between a client and a server, boxes which are aware of higher layers than just IP. It's impossible today to deploy an Internet-wide layer-4 protocol other than TCP or UDP, and increasingly it will be harder and harder to deploy a truly universal L7 protocol but HTTP. That's my limited view of this issue, how things like IPv6 might affect it I have no idea.
- Immortalin 8y agoBuilding a protocol over HTTP has the added benefit of making security easy. HTTP+TLS has a lot more eyeballs than protocols on lower levels. Most protocols on layer 3 and below weren't designed with security as a priority. If you are building a domain specific protocol, it's a lot easier to build on top of one which already has a good security story backed by a triumvirate of internet unicorns, than trying to re-invent the whole public key dance on a homebrewed setup.
- paulddraper 8y agoSSL/TLS with other protocols have a lot of usage too. SFTP/FTPS, SSH, SMTPS, etc.
- BTeam 8y agoSSH is a cryptographic network protocol that doesn't use SSL/TLS.
- paulddraper 8y agoMy bad. I should have said SSL/TLS/SSH. * SFTP/FTPS (SSH) * SSH (SSH) * SMTPS (SSL, TLS)
- 8y ago
- Ajedi32 8y ago> Applications that use HTTP are encouraged to allow an arbitrary URL to be used as that entry point. For example, rather than specifying "the initial document is at "/foo/v1", they should allow a deployment to use any URL as the entry point for the application. Is this really saying that, for example, I shouldn't have an API specification that says "to fetch a list of users", send a GET to `/api/v3/users`? What's the alternative?
- paulddraper 8y agoHypermedia [1] As The Engine Of Application State (HATEOAS) https://en.wikipedia.org/wiki/HATEOAS https://en.wikipedia.org/wiki/HATEOAS I not particularly a fan, but it's fairly common. It was popularized as one of the pillars of the REST paradigm. [1] https://en.wikipedia.org/wiki/Hypermedia https://en.wikipedia.org/wiki/Hypermedia [2] https://en.wikipedia.org/wiki/HATEOAS https://en.wikipedia.org/wiki/HATEOAS
- nickik 8y agoIs it actually common? Its talked about a lot, however in practice it is not used much from what I can tell.
- paulddraper 8y agoFor starters, HATEOAS it's used as the foundation of the most common networked applications in the world: web browsers. Paypal uses it for their API. But yeah, it's kind of like functional programming: the ideas are super common and well understood. But by the numbers, the less pure imperative programming paradigm is overwhelmingly more commonplace.
- JoshTriplett 8y agoAPI servers may need to co-exist, by mounting them into a hierarchy. You can certainly have `/api/v3/users`, but you should have enough configurability to support mounting that under `/yourapp/api/v3/users` so that someone doesn't need to rewrite URLs in a proxy.
- 8y ago
- deleted 8y ago[deleted]
- PinkMilkshake 8y agoIt's the next logical step to me. We have the Link, Internet, and Transport layers pretty well sorted. The Application layer is the next in line and it seems likely the web is going to win. A good thing about the web winning(and possibly part of the reason it's winning)is it has a lot of the next(and possibly final) layer already standardized. I'm not sure what you'd call it, let's say the Semantic Layer, all our human shit. (HTML, XML, and all the data formats, etc).
- rixed 8y agoAs I regard HTTP to be one of the main hindrance in todays IT industry I feel it is my duty to add my voice to those trying to warn against the now decade long trend to wrap everything into HTTP. Things that HTTP makes more complicated: 1. It's a verbose protocol with more often than not a really bad ratio payload over metadata; 2. Client authentication is hard; 3. Server authentication is expensive; 4. It's bad at streaming; 5. It does not help a bit for RPC (no support for fan-out, quotas, fail-overs, retries, ...). Building an infrastructure on HTTP makes it so unreliable that I came to consider one of the reason for Google technical superiority is that they early ditched HTTP completely out and used stubby for everything instead. Maybe they themselves view it as their secret sauce, which would explain why they've never published a comparative review (that I'm aware of)? Many people I've spoken to about this have expressed surprise that there is no HTTP flowing on Google's veins. To many, removing HTTP is like removing the solid ground; but to a former googler, loosing access to stubby and having to work with HTTP again is a painful experience that requires quite some time to accept.
- ori_b 8y ago> they early ditched HTTP completely out and used stubby for everything instead. Yes, when I worked there, it was interesting to realize that they take HTTP, and immediately convert it to protobuf.
- icebraining 8y agoSo Google made gRPC (which uses HTTP) just to sabotage their competition? Man, that's devious. By the way, server (and client, if needed) auth is implemented by the TLS layer. Do you propose using something else to secure other protocols?
- rixed 8y agoThey definitively pushed HTTP2 to help their clients move away from HTTP. My guess is they cannot make stubby public without disclosing too much valuable advantage and probably also knowing that this can only effectively run on their own DC at that point. At least, gRPC is easy to unwrap from HTTP (which is likely the very first step of processing on their side). Regarding protobuf, I'm not a big fan but at least that's not JSON (another awful tech that the industry is taking for granted). Also: TLS is slow, and last time I asked for the best practice to do client auth on HTTPS the general consensus seamed to be "don't do it".
- ori_b 8y ago"No".
- bullen 8y agoHere is an example of how you can do this: https://github.com/tinspin/fuse https://github.com/tinspin/fuse To use HTTPS, HTTP/2.0 or WebSockets is really not clever options when you can do secure, simple and more performant stuff in clear text over HTTP/1.1