17 ms·

Why does gRPC insist on trailers?

324 points by strzalek 4y ago

phendrenad2 4y ago
- upbeat_general 4y ago
  If you read the article you’d find out the opposite. That Chrome continues to lack support for trailers.
- e____g 4y ago
  This is very much not it. The Stubby team decided early on to use trailers and was combative and obstinate when the Chrome and GFE (reverse proxy) teams tried to explain why doing so would be a bad idea. The use of trailers in gRPC originates in the hubris of one team, not a conspiracy by Google.
deleted 4y ago
[deleted]
remram 4y ago
> As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. WTF is this? Those are different layer protocols. WebSocket can run on top of HTTP/2. It's like saying TLS is technically superior to TCP, or IP is superior to copper cables. Reference: https://www.rfc-editor.org/rfc/rfc8441.html https://www.rfc-editor.org/rfc/rfc8441.html
- anyfoo 4y ago
  I’m not a web developer, but that RFC, which talks about bootstrapping, talks about using the CONNECT method to “transition” to the WebSockets protocol. Which matches what I thought the CONNECT method does: Switch to a protocol that is not HTTP? But I only skimmed the introduction, did I miss something?
  - blibble 4y ago
    normally uses GET and the Upgrade header, not CONNECT
    - sveiss 4y ago
      That’s for HTTP/1.1, where WebSockets are really a completely different protocol which “hijacks” the underlying TCP or TCP+TLS stream from HTTP via the Upgrade request. HTTP/2 has its own concept of streams, so WebSockets can run over a single HTTP/2 stream, and the linked RFC describes extending the CONNECT method to take over a single stream.
- deleted 4y ago
  [deleted]
- jjtheblunt 4y ago
  > WebSocket can run on top of HTTP/2 Isn't "websocket" just a standard tcp socket, whose specification to instantiate it was born in a comparatively ephemeral HTTP (of whatever version) request, and which outlives the request, so isn't on top of anything other than tcp?
  - wmf 4y ago
    No, despite the name WebSockets are not plain TCP.
    - jjtheblunt 4y ago
      They're an http protocol for setting one up, you mean? First sentence at this article: https://en.wikipedia.org/wiki/WebSocket https://en.wikipedia.org/wiki/WebSocket
      - jhugo 4y ago
        No, they're more than that. After you upgrade to WebSocket you still have to speak the WebSocket protocol over TCP. It includes message framing — with different defined message types, masking, ping/pong, an extension system (including for example optional per-message compression), ...
      - deleted 4y ago
        [deleted]
  - aaaaaaaaaaab 4y ago
    TCP is a stream of bytes. WebSockets are message-based.
- rektide 4y ago
  For sure there are good reasons to abandon or find alternative resourceful protocols! But in general, http is & could be the de-facto really good resourceful protocol. It's already 90% there. Alas, the browser has been a major point of obstruction & difficulty & tension in making the most obvious most successful most clearly winning resourceful protocol at all better. The browser has sat on it's haunches & pissed around & prevented obvious & straightforward incremental growth that has happened everywhere else except the browser, such as with http trailers, such as with http2+ push. The browser has kept the de-facto resouceful protocol from developing. You dont have to believe in http as the way to see what an oppressive & stupid shitshow this is. Being able to enhance the de-facto protocol of the web better should be in everyones interest, in a way that doesnt prevent alternatives/offshoots. But right now only alternatives & offshoots have any traction, because the browsers have all shot doen & rejected doing anything to support modern http 2+ in any real capacity. Their http inplementations are all frozen in time.
- deleted 4y ago
  [deleted]
- jayd16 4y ago
  HTTP/2 provides features that websockets don't. Even if you were to use websockets over HTTP/2, you'd lose features like being able to multiplex requests _because_ it's a higher level protocol. Why is it wrong to say its better to use a more feature full and lower level protocol?
  - Thorrez 4y ago
    They provide different APIs. Websockets provide a bidirectional stream of messages. The messages in a given direction are always delivered in order. If they were to be suddenly reordered that would cause a lot of headaches. While the individual messages can't be multiplexed, different websocket streams over a single HTTP/2 connection can be multiplexed. I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript.
    - yesbabyyes 4y ago
      > I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript. I will never stop beating this drum: Server-Sent Events and EventSource! Simple to implement, scale, support. https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
      - phoboslab 4y ago
        Pardon my ignorance: Are messages sent through this guaranteed to arrive in order?
        roblabla 4y ago
        Yes. SSEs are just an infinite-length HTTP body formatted in a particular way. And like all HTTP bodies, they are guaranteed to arrive in the same order the bytes were sent.
- Matthias247 4y ago
  The idea here is that http provides something like request/response semantics, methods, path, status code, etc - which are all also useful for gRPC. Websockets provide none of that - they are just message streams. Websockets over http/2 are a new thing, and haven’t even been available at the time gRPC incepted.
  - stefan_ 4y ago
    That is their whole point. That is why they exist. Dumb reliable pipe, please keep your silly semantics away.
    - latch 4y ago
      Hardly dumb. They include a variable length, payload masking, their own layer of fragmentation, different message type, validation (txt vs bin) and an extension framework.
  - the_mitsuhiko 4y ago
    I'm not sure if websockets over HTTP/2 are actually a new thing. Firefox implemented support years ago but it was disabled almost immediately afterwards because it doesn't work with proxies and has been disabled still. I think the only engine implementing it is Chrome. As far as I know for HTTP/3 there is no way to use websockets yet.
- kevinmgranger 4y ago
  That's in a section specifically about picking the right transport. Per your example, it's like saying "TLS is technically superior to TCP, because it means our protocol can offload encryption and authentication to it".
joe_guy 4y ago
I had never heard of HTTP trailers. So FYI > The Trailer response header allows the sender to include additional fields at the end of chunked messages in order to supply metadata that might be dynamically generated while the message body is sent, such as a message integrity check, digital signature, or post-processing status. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Tr...
- k__ 4y ago
  Haven't headers and trailers been renamed in the recent past?
  - deleted 4y ago
    [deleted]
  - akshayshah 4y ago
    Yes, slightly - RFC 9110 ("HTTP Semantics") calls them "header fields" and "trailer fields," and it calls "headers" and "trailers" colloquialisms. In a nod to gRPC-style usage, the section on trailer fields even says, "Trailer fields can be useful for supplying...post-processing status information." https://www.rfc-editor.org/rfc/rfc9110.html#header.fields https://www.rfc-editor.org/rfc/rfc9110.html#header.fields https://www.rfc-editor.org/rfc/rfc9110.html#trailer.fields https://www.rfc-editor.org/rfc/rfc9110.html#trailer.fields
- lloeki 4y ago
  I suppose even fewer people have heard that Transfer-Encoding: chunked supports chunk extensions, which allows one to supply arbitrary metadata without trailers. https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1 https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1 Ever went to some site that generates compressed downloads or database exports on the fly, got no progress bar as a result, and were severely annoyed by that lack of feedback? I was, so I used chunk extensions to submit a draft to emit progress information dynamically: https://datatracker.ietf.org/doc/html/draft-lnageleisen-http-chunked-progress-00 https://datatracker.ietf.org/doc/html/draft-lnageleisen-http... As noted at the end of the draft, this could be generalized and extended to have additional capabilities such as in flight integrity checks, or whatever you can think of.
jeffbee 4y ago
Author doesn't support the case for grpc being a "failure". I wonder by what measure. It's certainly pretty popular.
- deleted 4y ago
  [deleted]
- plorkyeran 4y ago
  Being able to use gRPC in browsers was an explicit goal of the project, and that is impossible. gRPC-Web has to use a modified version of the protocol that has a limited feature set and performs worse for the reasons described in the article.
  - ikiris 4y ago
    Lets call the next version gRPC Send. then the vp can "leave" after a year or so, the project can get scrapped, and we can go back to something decent XD
  - __alexs 4y ago
    The lack of trailers support is not what means gRPC-Web needs to exist. It would be trivial for gRPC to support trailers-in-body like gRPC-Web does as a work around. The main problem for gRPC in the browser is the lack of HTTP/2 framing for messages which means gRPC-Web has to invent it's own framing format to make streams work. My experience in the early days of gRPC is that they seemed fairly unwilling to consider any need for an easy upgrade path for existing people using HTTP/1.1 at all. The author touches on this at the end: > Focus on customers. Despite locking horns with other orgs, our team had a more critical problem: we didn’t listen to early customer feedback. I'm glad they realise it now because lots of us warned them about this at the time.
- quietbritishjim 4y ago
  I was going to post a similar comment, but looking back at the post I realised that the author is upfront about what they consider the original point of gRPC to be: > gRPC was reared by two parents trying to solve similar problems: > 1. The Stubby team. They had just begun the next iteration of their RPC system... > 2. The API team. ... serving (all) public APIs at Google ... [this is not said explicitly but presumably the vast majority of API clients are web based] As you say, gPRC is very popular at server messaging, but I suppose it can never be an API solution. So, even if gRPC is successful in general, it was not successful at its original goal (as far as this author is concerned).
wmf 4y ago
Is it still the case that Google Chrome can't support Google gRPC?
- jxi 4y ago
  It requires a translation proxy: https://github.com/grpc/grpc-web https://github.com/grpc/grpc-web
- akshayshah 4y ago
  To be fair, it's also true that Firefox doesn't expose trailers to the fetch API. To the Chrome and gRPC team's credit, the public Chrome issue actually contains a somewhat substantive back-and-forth; the corresponding Firefox issue has virtually no discussion. https://bugzilla.mozilla.org/show_bug.cgi?id=1339096 https://bugzilla.mozilla.org/show_bug.cgi?id=1339096
criticaltinker 4y ago
Relevant post from a few days ago: Connect-Web: TypeScript library for calling RPC servers from web browsers https://news.ycombinator.com/item?id=32345670 https://news.ycombinator.com/item?id=32345670 I’m curious if anyone knows how Google internally works around the lack of support for gRPC in the browser? Perhaps gRPC is not used for public APIs? The lack of browser support in the protobuf and gRPC ecosystem was quite surprising and one of the biggest drawbacks noted by my team while evaluating various solutions.
- tombl 4y ago
  Looks like they have a separate protocol[0] for web compat, and they use a proxy to translate. [0]: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md
- kleton 4y ago
  Google internally doesn't have browser grpc clients. It's for service to service rpc and also exposed on googleapis.com apis to third party callers.
- skybrian 4y ago
  Back in the day, it wasn't used for private API's either. Different teams had come up with different ways of encoding protobuf-style messages as JSON for web apps. For the best browser-side performance, usually you want to use browser's native JSON.parse() API call and this doesn't really let you use unmodified protobufs. In particular, you can't use 64-bit ints since that's not a native JavaScript type. Meanwhile, server-side folks will use 64-bit ints routinely. So if the server-side folks decided on 64-bit integer ID's, you need workarounds like encoding them as strings on the wire. JavaScript has BigInt now, but still doesn't natively support decoding 64-bit integers from JSON. It didn't seem like the gRPC folks understood the needs of web developers very well.
  - akshayshah 4y ago
    Is decoding performance typically a problem for web UIs? The lackluster performance of binary protobuf decoding in browsers (and unmarshaling BigInts from JSON) seems much less problematic than (1) using a 200 for unary error responses, (2) choosing a wire format that's _always_ opaque to the network inspector tab, and (3) having really poor generated code. > It didn't seem like the gRPC folks understood the needs of web developers very well. Agreed. Being fair to the team that designed the protocol, though, it seems like browsers weren't in scope at the time.
game-of-throws 4y ago
> Why Do We Need Trailers At All? The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive.
- adev_ 4y ago
  > The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive. It did not to me. I would rephrase the argumentation as: - In HTTP/2 we thought to be smart by multiplexing multiple HTTP transactions over a single TCP connection. - Shit we realized later that HTTP/1.1 did not necessitate trailers because they could abort the connection and we can not afford to do that anymore, we are multiplexed now.... Shit, we do need trailers now. -> - That is currently a good example of good intentions inducing complexity. And complexity inducing even more complexity for free. - HTTP/2 is now rolled over the world and everybody has to deal with that - Still HTTP/2 suffers of several problems completely ignored by the blog post (Like the Head-of-Line blocking problem: it is not solved in HTTP/2). The result is now QUIC + HTTP/3 and we all start over again.
  - xyzzyz 4y ago
    > Like the Head-of-Line blocking problem: it is not solved in HTTP/2 How so? It is not solved on the network level (due to its use of TCP), but it is solved on application layer: slow response to one request is not blocking other requests.
    - adev_ 4y ago
      > How so? It is not solved on the network level (due to its use of TCP), but it is solved on application layer: slow response to one request is not blocking other requests. Like you said, it does not solve the network level: - in HTTP/2, A packet drop will slow down every HTTP transaction. - In HTTP/1 with a session pool (The usual web browser way of doing thing), it will slow down only one over X (X, size of the pool). This has been found to be a big problem over (unreliable) mobile network and that makes HTTP/2 sometimes even worst than HTTP/1. http://nl.cs.montana.edu/lab/publications/Goel_H2_extended.pdf http://nl.cs.montana.edu/lab/publications/Goel_H2_extended.p...
kiriberty 4y ago
Great article. I really like the points in "Lessons for Designers" section. Applicable for software engineering in general as well.
throwaway29303 4y ago
Interesting read. As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. Additionally, WebSockets suffers from the same head-of-line blocking problem HTTP/1.1 does. Not really a fair comparison. WebSockets is essentially a bidirectional stream of bytes without any verbs[0] or anything fancy. WebSockets is more like a fancy CONNECT. And speaking of bidirectional stream of bytes... HTTP/2 also suffers from head-of-the-line blocking as well, it uses TCP as its substrate, after all. QUIC, however, despite sharing some ideas from TCP, it seems to ameliorate this by resorting to multipaths[1]. It remains to be seen if indeed this is going to be beneficial however. [0] - unless you count its opcode field as something similar to HTTP verbs but if so it'd resemble more TCP than HTTP, I think [1] - https://datatracker.ietf.org/doc/html/draft-ietf-quic-multipath-02 https://datatracker.ietf.org/doc/html/draft-ietf-quic-multip...
jxi 4y ago
I was so excited for gRPC when it came out because it meant having strongly typed APIs and auto-generated clients, but two things made it horrible to use: requiring http/2 (so you couldn’t use most load balancers at the time) and the generated clients were unpleasant to use (you couldn’t just return an object to serialize, you had to conform to their streaming model).
- mcfedr 4y ago
  Checkout Twirp, you get the good parts, protobufs and generated code, but its supports regular http, by no including streaming. https://github.com/twitchtv/twirp https://github.com/twitchtv/twirp
thamer 4y ago
A few years ago I worked on a service that had to stream data out using protobuf messages, in a single request that could potentially transfer several gigabytes of data. At the HTTP level it was chunked, but above that I used a protobuf message that contained data plus a checksum of that data, with the last message of the stream containing no data but a checksum of the entire dataset (a flag was included to differentiate between the message types). This simple design led us to find several bugs in clients of this API (e.g. messages dropped or processed twice), and gave us a way to avoid some of the issues mentioned in this article. Even if you don't use HTTP trailers, you can still use them one layer above and benefit from similar guarantees.
- vidarh 4y ago
  Inserting metadata in the protobuf itself seems like the obvious, simple solution to avoid having to depend on what the transport layer supports. Just defining a message to provide the metadata they wanted to insert in trailers would have avoided a whole lot of pain.
twiss 4y ago
> Whether it’s because I was wrong, or failed to make the argument [for HTTP trailers support], I strongly suspect organizational boundaries had a substantial effect. The Area Tech Leads of Cloud also failed to convince their peers in Chrome, and as a result, trailers were ripped out [from the WHATWG fetch specification]. FWIW, I personally think it's a good thing that other teams within Google don't have too much of an "advantage" for getting features into Chrome, compared to other web developers, however, I also think it's very unfortunate that a single Chrome engineer gets to decide not only that it shouldn't be implemented in Chromium, but that that also has the effect of it being removed from the specification. (The linked issue [1] was also opened by a Google employee.) Of course, you might reasonably argue that, without consensus among the browsers to implement a feature, having it in the spec is useless. But nevertheless, with Chromium being an open source project, I think it would be better if it had a more democratic process of deciding which features should be supported (without, of course, requiring Google specifically to implement them, but also without, ideally, giving Google the power to veto them). [1]: https://github.com/whatwg/fetch/issues/772 https://github.com/whatwg/fetch/issues/772
- modeless 4y ago
  It's clear that the "single engineer" thing is a lie. Many engineers commented on the Chrome issue with opposing viewpoints, and even the original post describes it being escalated to tech leads on both sides, getting more people involved. I guarantee if it was only one person standing alone opposed to trailers then they would have been overruled. As you say, it's a good thing that Chrome resists adding the pet features of every other Google team to the web.
  - guenthert 4y ago
    Jeebus. Just because it's not true doesn't mean it's a lie.
    - modeless 4y ago
      It's a lie if OP knows it isn't true. And OP knows it isn't true because they explicitly describe a lot more than one person being involved on both sides.
chucky_z 4y ago
From my perspective, I think the biggest issue with gRPC is it using HTTP/2. I understand that there’s a lot of reasons to say “No, HTTP/2 is far superior to HTTP/1.1.” However, in terms of proxying _outside Google_ HTTP/2 has lagged, and continues to lags at the L7 proxy layer. I recently performed a lot of high-throughput proxying comparing HAProxy, Traefik, and Envoy. HTTP/1.1 outperformed HTTP/2 (even H2C) by a pretty fair margin. Enough that if gRPC used HTTP/1.1 we could use noticeably less hardware. I could see this holding true even with a service mesh.
- deleted 4y ago
  [deleted]
- thayne 4y ago
  Also, http/2 over cleartext is not very well supported by a lot of things. Which is probably a good thing when going over the open internet. But it means you have to deal with setting up certificates even if just developing locally, and makes it more difficult to use for IPC on a single host.
  - mort96 4y ago
    My preferred setup is to have an unencrypted service running on 127.0.0.1 (so not publicly available), and then have nginx in front to handle certificates. Lets me do all certificate stuff across all virtual hosts in one place. HTTP/2 makes this impossible due to its ridiculous TLS requirement, so I, and everyone who does it the way I do, must keep using HTTP/1.1 forever. It's my belief that requiring TLS for HTTP/2 is what killed the protocol. It just causes too much friction during both development and deployment, for little to no (or negative) performance gain.
    - simiones 4y ago
      > My preferred setup is to have an unencrypted service running on 127.0.0.1 (so not publicly available), Don't forget that JS from any webpage can access your 127.0.0.1 to various degrees. Depending on what types of requests exactly the server accepts, it may be somewhat unsafe for a machine with a browser.
AceJohnny2 4y ago
Offtopic, but: > However, Google is not one single company, but a collection of independent and distrusting companies. This is an important thing to keep in mind when considering the behavior of any large company.
- teaearlgraycold 4y ago
  As a Googler, it's worse. Even within one of the "companies" there is distrust in the chain of command.
  - Tyr42 4y ago
    Googler here. Yeah, it goes my boss, another guy I'm somewhat familiar with, then some cloud of VPs or something and then Sundar. I have no idea what those vp's are up to and not much faith in their decisions.
    - teaearlgraycold 4y ago
      When I joined I found it funny you can't DM Sundar. But you can DM SVPs. At least the SVP of my division is open on the chat app.
wonnage 4y ago
Trailers would be theoretically useful in a variety of HTML streaming-related cases if they actually had widespread support (but they don't): - sending down Server-Timing values for processing done after the headers are sent - updating the response status or redirecting after the headers are sent - deciding whether a response is cacheable after you've finished generating it All of these except the first one obviously break assumptions about HTTP and I'm not surprised they're unsupported. Firefox [1] actually supports the first case. The rest have workarounds, you can do a meta-refresh or a JS redirect, and you could simply not stream cacheable pages (assuming they'd generally be served from cache anyway). But it's still the case that frontend code generally likes to throw errors and trigger redirects in the course of rendering, rather than performing all that validation up front. That's sensible when you're rendering in a browser, but makes it hard to stream stuff with meaningful status codes.
fijiaarone 4y ago
Application layer encoding should not interfere in the protocol transport layer.
- jayd16 4y ago
  This is more about whether it is acceptable to push error checking to the application layer or not. Seems like the gRPC designers agreed with you and so they need trailers.
Matthias247 4y ago
> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure. I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trailers. The only thing that trailer support adds is the ability to send an additional late response code. That could have been added also without trailers. Just put another length prefixed block inside the body stream, and add a flag before that differentiates trailers from a message. Essentially protobuf (application messages) in protobuf (definition of the response body stream). I assume someone thought trailers would be a neat thing that is already part of the spec and can do the job. But the bet didn’t work out since browsers and most other HTTP libraries didn’t found them worthwhile enough to fully support.
- afc 4y ago
  He offers two facts that I think explain this well enough: > Additionally, they chose to keep Protobuf as the default wire format, but allow other encodings too. And: > Since streaming is a primary feature of gRPC, we often will not know the length of the response ahead of time. These make sense; you'd enable servers to start streaming back the responses directly as they were generating them, before the length of the response could be known. Not requiring servers to hold the entire response can have drastic latency and memory/performance impact for large responses.
  - Thorrez 4y ago
    This doesn't match what I see in the gRPC spec. It says every message must be length-prefixed. https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.... Disclaimer: I don't know much about gRPC.
    - akshayshah 4y ago
      I've spent a fair bit of time working with gRPC, and you're correct - gRPC's length-prefixing makes it easy to detect when individual messages are terminated early. You do still need some way to detect streams that terminate unexpectedly on message boundaries - perhaps you could rely on HTTP/2 EOS bits as evidence of an application-level success, but you need some equivalent of trailers to communicate the details of any errors that occur midway through the response stream anyways.
mdriley 4y ago
It seems like a lot of other technologies in this space have solved the listed problems while remaining compatible with browsers, load balancers, reverse proxies, etc. It was a product choice not to offer a fallback path when HTTP/2 was unavailable. That choice made gRPC impossible to deploy in a lot of real-world environments. What motivated that choice?
- lakomen 4y ago
  Can you name one or more?
  - mdriley 4y ago
    one relevant datapoint: https://devblogs.microsoft.com/dotnet/announcing-grpc-json-transcoding-for-dotnet/ https://devblogs.microsoft.com/dotnet/announcing-grpc-json-t...
alexcpn 4y ago
I use GRPC between micro services instead of REST and for that it is really great; All the deficiencies of REST - Non versioned, no typed goes away with GRPC and the protobuf is the official interface for all micro-services. No problems with this approach for over two years now; and also multi language support - We have Go and Java and Python and TypeScript micro-services now happily talking and getting new features and new interface methods updated. Maybe it was demise in the web space; but a jewel in micro-service space
- marcyb5st 4y ago
  This is more or less what stubby is/was for Google and so the original driving force in implementing it. Now, if you add a catch all service that translates the requests from the outside to Protobuffers and then forwards the translated requests to the correct service you have a GFE (Google Front-End) equivalent. Should you do it? Probably not as it's not just a dumb translation layer and it is extremely complex (e.g. needs to support streams which is non-trivial in this situation). For Google it's worth because this way you only have to handle protobuffers beyond the GFE layer.
- withinboredom 4y ago
  With GRPC, you lose the ability to introspect the data on-the-wire. You lose the ability to create optimized data formats for YOUR application (who said you have to use JSON?) Most people can’t implement REST correctly, so it has been a shitshow for the last 20 or so years, GRPC isn’t a magic bullet, it just forces you to solve problems (or helps you to solve them) that you should have been doing in the first place. You can do all of these things without GRPC, there is no power it grants you that can’t be done better or at all in your own libraries and specs.
  - ackfoobar 4y ago
    I suppose you mean "inspect" the data on-the-wire https://grpc.io/blog/wireshark/ https://grpc.io/blog/wireshark/ Wireshark can load proto files and decode the data for you. BTW, "The Internet is running in debug mode".
akshayshah 4y ago
The author also posted an interesting Twitter thread a few months ago [0], on the day my coworkers and I posted here about our gRPC-compatible RPC framework [1]. I was a bit afraid to read this post, but I shouldn't have been - the author's a class act, and he never called us out explicitly. There's not much written about what the gRPC team was _thinking_ when they wrote up the protocol, and this was a nice window into how contemporaneous changes to HTTP and the fetch API shaped their approach. Given my current work, the final section ("Lessons for Designers") really hit home. That said, I didn't follow the central argument - that you need HTTP trailers to detect incomplete protobuf messages. What's not mentioned in the blog post is that gRPC wraps every protobuf message in a 5-byte envelope, and the bulk of the envelope is devoted to specifying the length of the enclosed message. It's easy to detect prematurely terminated messages, because they don't contain the promised number of bytes. The author says, "[i]t’s not hard to imagine that trailers would be less of an issue, if the default encoding was JSON," because JSON objects are explicitly terminated by a closing } - but it seems to me that envelopes solve that problem neatly. With incomplete message detection handled, we're left looking for some mechanism to detect streams that prematurely terminate at a message boundary. (This is more likely than you might expect, since servers often crash at message boundaries.) In practice, gRPC implementations already buffer responses to unary RPCs. It's therefore easy to use the standard HTTP Content-Length header for unary responses. This covers the vast majority of RPCs with a simple, uncontroversial approach. Streaming responses do need some trailer-like mechanism, but not to detect premature termination - as long as we're restricting ourselves to HTTP/2, cleanly terminated streams always end with a frame with the end of stream bit set. Streaming does need some trailer-like mechanism to send the details of any errors that occur mid-stream, but there's no need to use HTTP trailers. As the author hints, there's some unused space in the message envelope - we can use one bit to flag the last message in the stream and use it for the end-of-stream metadata. This is, more or less, what the gRPC-Web protocol is. (Admittedly, it's probably a bad idea to rely on _every_ HTTP server and proxy on the internet handling premature termination correctly. We need some sort of trailer-like construct anyways, and the fact that it also improves robustness is a nice extra benefit.) So from the outside, it doesn't seem like trailers improve the robustness of most RPCs. Instead, it seems like the gRPC protocol prioritizes some abstract notion of cleanliness over simplicity in practice: by using the same wire protocol for unary and streaming RPCs, everyday request-response workloads take on all the complexity of streaming. Even for streaming responses, the practical difficulties of working with HTTP trailers have also been apparent for years; I'm shocked that more of the gRPC ecosystem hasn't followed .NET's lead and integrated gRPC-Web support into servers. (If I had to guess, it's difficult because many of Google's gRPC implementations include their own HTTP/2 transport - adding HTTP/1.1 support is a tremendous expansion in scope. Presumably the same applies to HTTP/3, once it's finalized.) Again, though, I appreciated the inside look into the gRPC team's thinking. It takes courage to discuss the imperfections of your own work, especially when your former coworkers are still supporting the project. gRPC is far from perfect, but the engineers working on it are clearly skilled, experienced, and generally decent people. Hats off to the author - personally, I hope to someday write code influential enough that a retrospective makes the front page of HN :) 0: https://twitter.com/CarlMastrangelo/status/1532256576274243584 https://twitter.com/CarlMastrangelo/status/15322565762742435... 1: https://news.ycombinator.com/item?id=31584555 https://news.ycombinator.com/item?id=31584555
rswail 4y ago
Personal opinion: RPC is a failed architectural style, independent of what serialization/marshalling of arguments is used. it failed with CORBA, it failed with ONC-RPC, it failed with Java RMI. Remote Procedure Calls attempt to abstract away the networked nature of the function and make it "look like" a local function call. That's Just Wrong. When two networked services are communicating, the network must be considered. REST relies on the media type, links and the limited verb set to define the resource and the state transfer operations to change the state of the resource. HTTP explicitly incorporates the networked nature of the server/client relationship, independent of, and irrespective of, the underlying server or client implementation. Media types, separated from the HTTP networking, define the format and serialization of the resource representation independent of the network. HTTP/REST doesn't really support streaming.
- akshayshah 4y ago
  That's true of CORBA, for sure. I'm not familiar with ONC-RPC or Java RMI. It's not true of gRPC. It's not "RPC" in any traditional sense - it's just a particular HTTP convention, and the clients reflect that. They're asynchronous, make deadlines and transport errors first-class concepts, and make it easy to work with HTTP headers (and trailers, as the article explains). Calling a gRPC API with a generated client often doesn't feel too different from using a client library for a REST API. It's definitely a verb-oriented style, as opposed to REST's noun orientation. That's sometimes a plus, and sometimes a minus; it's the same "Kingdom of Nouns" debate [0] that's been going on about Java-style OOP for years. 0: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...
  - rswail 4y ago
    The verb-oriented style is part of the problem. Too many verbs is the problem. Java/OOP problems are not the same as the REST style, which is entirely about Nouns. There's none of the Java ManagerFactoryManager problems. The generated client from an IDL that wraps the network protocol with a function call is also part of the problem. REST APIs that have function calls that aren't "Send Request and wait for Response" aren't REST. ("wait for response" doesn't imply synchronous implementation, but HTTP is a request/response oriented protocol).
kris-nova 4y ago
gRPC: protobuf and stubby for performance reasons, we’ve spared no expense.
lakomen 4y ago
To this day it's still not clear to me, as even if asked on their github issues there is no definite answer, Can one use nginx in front of a grpc serving backend if the client is a JS client in the broadest sense? This unanswered question is the main reason I'm still doing RESTful JSON.
jiggawatts 4y ago
Something that’s always bugged me about streaming protocols of this type is that they prevent processing pipelining. If trailers are used for things such as checksums, then the client must wait patiently for potentially gigabytes of data to stream to it before it can verify the data integrity and start processing it safely. If the data is sent chunked, then this is not an issue. The client can start decoding chunks as they arrive, each one with a separate checksum.
- crest 4y ago
  I would argue that RPC systems shouldn't be burdened down with features like this. If you want to exchange more date than either endpoint can buffer comfortably in memory split it up into several RPC messages e.g. create large object, define byte range, seal object. If the producer can precompute the length and hash you can simplify things by using content addressing to reference such objects and replicate them. If you can afford the overhead breaking them up into Merkle DAGs reasonably sized leaves is a good idea to allow validating and resuming partial transfers. It matters if your devices are connected via PCIe in a single chassis or mobile networks spread over half the world and datacenter optimised protocol won't be optimal for expensive, slow, unreliable links.
- jmillikin 4y ago
  This comment is mixing up a few concerns. 1. When transferring large amounts of data, the checksum for the full transfer can't be verified until all the data is received. If you want to (for example) download an Ubuntu ISO and verify its checksum before installing it, you'll have to buffer the data somewhere until the download finishes. 2. When transferring small amounts of data, such as individual chunks, the data integrity is (/ should be) automatically verified by the encryption layer[0] of the underlying transport. There's no point in putting a shasum into each chunk because if a bit gets flipped in transit then that chunk will never even arrive in your message handler. 3. In gRPC, chunking large data transfers is mandatory because the library will reject Protobuf messages larger than a few megabytes[1]. As the chunks of data arrive, they can be passed into a hash function at the same time as you're buffering them to disk. [0] gRPC supports running without encryption for local development, but obviously for real workloads you'd do end-to-end TLS. [1] IIRC the default for C++ and Go implementations of gRPC is 4 MiB, which can be overridden when the client/server is being initialized. For bulk data transfer there's also the Protobuf hard limit of 2GB[2] for variable-length data. [2] https://developers.google.com/protocol-buffers/docs/encoding https://developers.google.com/protocol-buffers/docs/encoding
AtNightWeCode 4y ago
Very nice post. Http2 did not solve the TCP HOL problems though. Not sure about the WS statement. On the other hand. Vanilla WS has never ended up in prod on any of my projects even if it has been implemented several times.
jenia2022 4y ago
I'm not getting it. Why is HTTP so inadequate for gRPC? A service app for example can open 1000 sockets with a server and simply multiplex that way.

324 points by strzalek 4y ago

phendrenad2 4y ago

upbeat_general 4y ago
If you read the article you’d find out the opposite. That Chrome continues to lack support for trailers.
e____g 4y ago
This is very much not it. The Stubby team decided early on to use trailers and was combative and obstinate when the Chrome and GFE (reverse proxy) teams tried to explain why doing so would be a bad idea. The use of trailers in gRPC originates in the hubris of one team, not a conspiracy by Google.

deleted 4y ago

[deleted]

remram 4y ago

> As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. WTF is this? Those are different layer protocols. WebSocket can run on top of HTTP/2. It's like saying TLS is technically superior to TCP, or IP is superior to copper cables. Reference: https://www.rfc-editor.org/rfc/rfc8441.html https://www.rfc-editor.org/rfc/rfc8441.html

anyfoo 4y ago
I’m not a web developer, but that RFC, which talks about bootstrapping, talks about using the CONNECT method to “transition” to the WebSockets protocol. Which matches what I thought the CONNECT method does: Switch to a protocol that is not HTTP? But I only skimmed the introduction, did I miss something?
- blibble 4y ago
  normally uses GET and the Upgrade header, not CONNECT
  - sveiss 4y ago
    That’s for HTTP/1.1, where WebSockets are really a completely different protocol which “hijacks” the underlying TCP or TCP+TLS stream from HTTP via the Upgrade request. HTTP/2 has its own concept of streams, so WebSockets can run over a single HTTP/2 stream, and the linked RFC describes extending the CONNECT method to take over a single stream.
deleted 4y ago
[deleted]
jjtheblunt 4y ago
> WebSocket can run on top of HTTP/2 Isn't "websocket" just a standard tcp socket, whose specification to instantiate it was born in a comparatively ephemeral HTTP (of whatever version) request, and which outlives the request, so isn't on top of anything other than tcp?
- wmf 4y ago
  No, despite the name WebSockets are not plain TCP.
  - jjtheblunt 4y ago
    They're an http protocol for setting one up, you mean? First sentence at this article: https://en.wikipedia.org/wiki/WebSocket https://en.wikipedia.org/wiki/WebSocket
    - jhugo 4y ago
      No, they're more than that. After you upgrade to WebSocket you still have to speak the WebSocket protocol over TCP. It includes message framing — with different defined message types, masking, ping/pong, an extension system (including for example optional per-message compression), ...
    - deleted 4y ago
      [deleted]
- aaaaaaaaaaab 4y ago
  TCP is a stream of bytes. WebSockets are message-based.
rektide 4y ago
For sure there are good reasons to abandon or find alternative resourceful protocols! But in general, http is & could be the de-facto really good resourceful protocol. It's already 90% there. Alas, the browser has been a major point of obstruction & difficulty & tension in making the most obvious most successful most clearly winning resourceful protocol at all better. The browser has sat on it's haunches & pissed around & prevented obvious & straightforward incremental growth that has happened everywhere else except the browser, such as with http trailers, such as with http2+ push. The browser has kept the de-facto resouceful protocol from developing. You dont have to believe in http as the way to see what an oppressive & stupid shitshow this is. Being able to enhance the de-facto protocol of the web better should be in everyones interest, in a way that doesnt prevent alternatives/offshoots. But right now only alternatives & offshoots have any traction, because the browsers have all shot doen & rejected doing anything to support modern http 2+ in any real capacity. Their http inplementations are all frozen in time.
deleted 4y ago
[deleted]
jayd16 4y ago
HTTP/2 provides features that websockets don't. Even if you were to use websockets over HTTP/2, you'd lose features like being able to multiplex requests _because_ it's a higher level protocol. Why is it wrong to say its better to use a more feature full and lower level protocol?
- Thorrez 4y ago
  They provide different APIs. Websockets provide a bidirectional stream of messages. The messages in a given direction are always delivered in order. If they were to be suddenly reordered that would cause a lot of headaches. While the individual messages can't be multiplexed, different websocket streams over a single HTTP/2 connection can be multiplexed. I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript.
  - yesbabyyes 4y ago
    > I think websockets also provides a feature that HTTP/2 doesn't: the ability to easily push data from the server to browser javascript. I will never stop beating this drum: Server-Sent Events and EventSource! Simple to implement, scale, support. https://developer.mozilla.org/en-US/docs/Web/API/Server-sent_events https://developer.mozilla.org/en-US/docs/Web/API/Server-sent... https://developer.mozilla.org/en-US/docs/Web/API/EventSource https://developer.mozilla.org/en-US/docs/Web/API/EventSource
    - phoboslab 4y ago
      Pardon my ignorance: Are messages sent through this guaranteed to arrive in order?
      - roblabla 4y ago
        Yes. SSEs are just an infinite-length HTTP body formatted in a particular way. And like all HTTP bodies, they are guaranteed to arrive in the same order the bytes were sent.
Matthias247 4y ago
The idea here is that http provides something like request/response semantics, methods, path, status code, etc - which are all also useful for gRPC. Websockets provide none of that - they are just message streams. Websockets over http/2 are a new thing, and haven’t even been available at the time gRPC incepted.
- stefan_ 4y ago
  That is their whole point. That is why they exist. Dumb reliable pipe, please keep your silly semantics away.
  - latch 4y ago
    Hardly dumb. They include a variable length, payload masking, their own layer of fragmentation, different message type, validation (txt vs bin) and an extension framework.
- the_mitsuhiko 4y ago
  I'm not sure if websockets over HTTP/2 are actually a new thing. Firefox implemented support years ago but it was disabled almost immediately afterwards because it doesn't work with proxies and has been disabled still. I think the only engine implementing it is Chrome. As far as I know for HTTP/3 there is no way to use websockets yet.
kevinmgranger 4y ago
That's in a section specifically about picking the right transport. Per your example, it's like saying "TLS is technically superior to TCP, because it means our protocol can offload encryption and authentication to it".

joe_guy 4y ago

I had never heard of HTTP trailers. So FYI > The Trailer response header allows the sender to include additional fields at the end of chunked messages in order to supply metadata that might be dynamically generated while the message body is sent, such as a message integrity check, digital signature, or post-processing status. https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Trailer https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Tr...

k__ 4y ago
Haven't headers and trailers been renamed in the recent past?
- deleted 4y ago
  [deleted]
- akshayshah 4y ago
  Yes, slightly - RFC 9110 ("HTTP Semantics") calls them "header fields" and "trailer fields," and it calls "headers" and "trailers" colloquialisms. In a nod to gRPC-style usage, the section on trailer fields even says, "Trailer fields can be useful for supplying...post-processing status information." https://www.rfc-editor.org/rfc/rfc9110.html#header.fields https://www.rfc-editor.org/rfc/rfc9110.html#header.fields https://www.rfc-editor.org/rfc/rfc9110.html#trailer.fields https://www.rfc-editor.org/rfc/rfc9110.html#trailer.fields
lloeki 4y ago
I suppose even fewer people have heard that Transfer-Encoding: chunked supports chunk extensions, which allows one to supply arbitrary metadata without trailers. https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1 https://datatracker.ietf.org/doc/html/rfc2616#section-3.6.1 Ever went to some site that generates compressed downloads or database exports on the fly, got no progress bar as a result, and were severely annoyed by that lack of feedback? I was, so I used chunk extensions to submit a draft to emit progress information dynamically: https://datatracker.ietf.org/doc/html/draft-lnageleisen-http-chunked-progress-00 https://datatracker.ietf.org/doc/html/draft-lnageleisen-http... As noted at the end of the draft, this could be generalized and extended to have additional capabilities such as in flight integrity checks, or whatever you can think of.

jeffbee 4y ago

Author doesn't support the case for grpc being a "failure". I wonder by what measure. It's certainly pretty popular.

deleted 4y ago
[deleted]
plorkyeran 4y ago
Being able to use gRPC in browsers was an explicit goal of the project, and that is impossible. gRPC-Web has to use a modified version of the protocol that has a limited feature set and performs worse for the reasons described in the article.
- ikiris 4y ago
  Lets call the next version gRPC Send. then the vp can "leave" after a year or so, the project can get scrapped, and we can go back to something decent XD
- __alexs 4y ago
  The lack of trailers support is not what means gRPC-Web needs to exist. It would be trivial for gRPC to support trailers-in-body like gRPC-Web does as a work around. The main problem for gRPC in the browser is the lack of HTTP/2 framing for messages which means gRPC-Web has to invent it's own framing format to make streams work. My experience in the early days of gRPC is that they seemed fairly unwilling to consider any need for an easy upgrade path for existing people using HTTP/1.1 at all. The author touches on this at the end: > Focus on customers. Despite locking horns with other orgs, our team had a more critical problem: we didn’t listen to early customer feedback. I'm glad they realise it now because lots of us warned them about this at the time.
quietbritishjim 4y ago
I was going to post a similar comment, but looking back at the post I realised that the author is upfront about what they consider the original point of gRPC to be: > gRPC was reared by two parents trying to solve similar problems: > 1. The Stubby team. They had just begun the next iteration of their RPC system... > 2. The API team. ... serving (all) public APIs at Google ... [this is not said explicitly but presumably the vast majority of API clients are web based] As you say, gPRC is very popular at server messaging, but I suppose it can never be an API solution. So, even if gRPC is successful in general, it was not successful at its original goal (as far as this author is concerned).

wmf 4y ago

Is it still the case that Google Chrome can't support Google gRPC?

jxi 4y ago
It requires a translation proxy: https://github.com/grpc/grpc-web https://github.com/grpc/grpc-web
akshayshah 4y ago
To be fair, it's also true that Firefox doesn't expose trailers to the fetch API. To the Chrome and gRPC team's credit, the public Chrome issue actually contains a somewhat substantive back-and-forth; the corresponding Firefox issue has virtually no discussion. https://bugzilla.mozilla.org/show_bug.cgi?id=1339096 https://bugzilla.mozilla.org/show_bug.cgi?id=1339096

criticaltinker 4y ago

Relevant post from a few days ago: Connect-Web: TypeScript library for calling RPC servers from web browsers https://news.ycombinator.com/item?id=32345670 https://news.ycombinator.com/item?id=32345670 I’m curious if anyone knows how Google internally works around the lack of support for gRPC in the browser? Perhaps gRPC is not used for public APIs? The lack of browser support in the protobuf and gRPC ecosystem was quite surprising and one of the biggest drawbacks noted by my team while evaluating various solutions.

tombl 4y ago
Looks like they have a separate protocol[0] for web compat, and they use a proxy to translate. [0]: https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-WEB.md
kleton 4y ago
Google internally doesn't have browser grpc clients. It's for service to service rpc and also exposed on googleapis.com apis to third party callers.
skybrian 4y ago
Back in the day, it wasn't used for private API's either. Different teams had come up with different ways of encoding protobuf-style messages as JSON for web apps. For the best browser-side performance, usually you want to use browser's native JSON.parse() API call and this doesn't really let you use unmodified protobufs. In particular, you can't use 64-bit ints since that's not a native JavaScript type. Meanwhile, server-side folks will use 64-bit ints routinely. So if the server-side folks decided on 64-bit integer ID's, you need workarounds like encoding them as strings on the wire. JavaScript has BigInt now, but still doesn't natively support decoding 64-bit integers from JSON. It didn't seem like the gRPC folks understood the needs of web developers very well.
- akshayshah 4y ago
  Is decoding performance typically a problem for web UIs? The lackluster performance of binary protobuf decoding in browsers (and unmarshaling BigInts from JSON) seems much less problematic than (1) using a 200 for unary error responses, (2) choosing a wire format that's _always_ opaque to the network inspector tab, and (3) having really poor generated code. > It didn't seem like the gRPC folks understood the needs of web developers very well. Agreed. Being fair to the team that designed the protocol, though, it seems like browsers weren't in scope at the time.

game-of-throws 4y ago

> Why Do We Need Trailers At All? The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive.

adev_ 4y ago
> The author convinced they're needed. But I wonder if some sort of error signaling should have been baked into `Transfer-Encoding: chunked` instead. It wouldn't have made sense in HTTP/1.1 since you can just close the connection. But in later HTTP versions with pipelined requests, I can see the use for bailing on one request while keeping the rest alive. It did not to me. I would rephrase the argumentation as: - In HTTP/2 we thought to be smart by multiplexing multiple HTTP transactions over a single TCP connection. - Shit we realized later that HTTP/1.1 did not necessitate trailers because they could abort the connection and we can not afford to do that anymore, we are multiplexed now.... Shit, we do need trailers now. -> - That is currently a good example of good intentions inducing complexity. And complexity inducing even more complexity for free. - HTTP/2 is now rolled over the world and everybody has to deal with that - Still HTTP/2 suffers of several problems completely ignored by the blog post (Like the Head-of-Line blocking problem: it is not solved in HTTP/2). The result is now QUIC + HTTP/3 and we all start over again.
- xyzzyz 4y ago
  > Like the Head-of-Line blocking problem: it is not solved in HTTP/2 How so? It is not solved on the network level (due to its use of TCP), but it is solved on application layer: slow response to one request is not blocking other requests.
  - adev_ 4y ago
    > How so? It is not solved on the network level (due to its use of TCP), but it is solved on application layer: slow response to one request is not blocking other requests. Like you said, it does not solve the network level: - in HTTP/2, A packet drop will slow down every HTTP transaction. - In HTTP/1 with a session pool (The usual web browser way of doing thing), it will slow down only one over X (X, size of the pool). This has been found to be a big problem over (unreliable) mobile network and that makes HTTP/2 sometimes even worst than HTTP/1. http://nl.cs.montana.edu/lab/publications/Goel_H2_extended.pdf http://nl.cs.montana.edu/lab/publications/Goel_H2_extended.p...

kiriberty 4y ago

Great article. I really like the points in "Lessons for Designers" section. Applicable for software engineering in general as well.

throwaway29303 4y ago

Interesting read. As an aside, HTTP/2 is technically superior to WebSockets. HTTP/2 keeps the semantics of the web, while WS does not. Additionally, WebSockets suffers from the same head-of-line blocking problem HTTP/1.1 does. Not really a fair comparison. WebSockets is essentially a bidirectional stream of bytes without any verbs[0] or anything fancy. WebSockets is more like a fancy CONNECT. And speaking of bidirectional stream of bytes... HTTP/2 also suffers from head-of-the-line blocking as well, it uses TCP as its substrate, after all. QUIC, however, despite sharing some ideas from TCP, it seems to ameliorate this by resorting to multipaths[1]. It remains to be seen if indeed this is going to be beneficial however. [0] - unless you count its opcode field as something similar to HTTP verbs but if so it'd resemble more TCP than HTTP, I think [1] - https://datatracker.ietf.org/doc/html/draft-ietf-quic-multipath-02 https://datatracker.ietf.org/doc/html/draft-ietf-quic-multip...

jxi 4y ago

I was so excited for gRPC when it came out because it meant having strongly typed APIs and auto-generated clients, but two things made it horrible to use: requiring http/2 (so you couldn’t use most load balancers at the time) and the generated clients were unpleasant to use (you couldn’t just return an object to serialize, you had to conform to their streaming model).

mcfedr 4y ago
Checkout Twirp, you get the good parts, protobufs and generated code, but its supports regular http, by no including streaming. https://github.com/twitchtv/twirp https://github.com/twitchtv/twirp

thamer 4y ago

A few years ago I worked on a service that had to stream data out using protobuf messages, in a single request that could potentially transfer several gigabytes of data. At the HTTP level it was chunked, but above that I used a protobuf message that contained data plus a checksum of that data, with the last message of the stream containing no data but a checksum of the entire dataset (a flag was included to differentiate between the message types). This simple design led us to find several bugs in clients of this API (e.g. messages dropped or processed twice), and gave us a way to avoid some of the issues mentioned in this article. Even if you don't use HTTP trailers, you can still use them one layer above and benefit from similar guarantees.

vidarh 4y ago
Inserting metadata in the protobuf itself seems like the obvious, simple solution to avoid having to depend on what the transport layer supports. Just defining a message to provide the metadata they wanted to insert in trailers would have avoided a whole lot of pain.

twiss 4y ago

> Whether it’s because I was wrong, or failed to make the argument [for HTTP trailers support], I strongly suspect organizational boundaries had a substantial effect. The Area Tech Leads of Cloud also failed to convince their peers in Chrome, and as a result, trailers were ripped out [from the WHATWG fetch specification]. FWIW, I personally think it's a good thing that other teams within Google don't have too much of an "advantage" for getting features into Chrome, compared to other web developers, however, I also think it's very unfortunate that a single Chrome engineer gets to decide not only that it shouldn't be implemented in Chromium, but that that also has the effect of it being removed from the specification. (The linked issue [1] was also opened by a Google employee.) Of course, you might reasonably argue that, without consensus among the browsers to implement a feature, having it in the spec is useless. But nevertheless, with Chromium being an open source project, I think it would be better if it had a more democratic process of deciding which features should be supported (without, of course, requiring Google specifically to implement them, but also without, ideally, giving Google the power to veto them). [1]: https://github.com/whatwg/fetch/issues/772 https://github.com/whatwg/fetch/issues/772

modeless 4y ago
It's clear that the "single engineer" thing is a lie. Many engineers commented on the Chrome issue with opposing viewpoints, and even the original post describes it being escalated to tech leads on both sides, getting more people involved. I guarantee if it was only one person standing alone opposed to trailers then they would have been overruled. As you say, it's a good thing that Chrome resists adding the pet features of every other Google team to the web.
- guenthert 4y ago
  Jeebus. Just because it's not true doesn't mean it's a lie.
  - modeless 4y ago
    It's a lie if OP knows it isn't true. And OP knows it isn't true because they explicitly describe a lot more than one person being involved on both sides.

chucky_z 4y ago

From my perspective, I think the biggest issue with gRPC is it using HTTP/2. I understand that there’s a lot of reasons to say “No, HTTP/2 is far superior to HTTP/1.1.” However, in terms of proxying _outside Google_ HTTP/2 has lagged, and continues to lags at the L7 proxy layer. I recently performed a lot of high-throughput proxying comparing HAProxy, Traefik, and Envoy. HTTP/1.1 outperformed HTTP/2 (even H2C) by a pretty fair margin. Enough that if gRPC used HTTP/1.1 we could use noticeably less hardware. I could see this holding true even with a service mesh.

deleted 4y ago
[deleted]
thayne 4y ago
Also, http/2 over cleartext is not very well supported by a lot of things. Which is probably a good thing when going over the open internet. But it means you have to deal with setting up certificates even if just developing locally, and makes it more difficult to use for IPC on a single host.
- mort96 4y ago
  My preferred setup is to have an unencrypted service running on 127.0.0.1 (so not publicly available), and then have nginx in front to handle certificates. Lets me do all certificate stuff across all virtual hosts in one place. HTTP/2 makes this impossible due to its ridiculous TLS requirement, so I, and everyone who does it the way I do, must keep using HTTP/1.1 forever. It's my belief that requiring TLS for HTTP/2 is what killed the protocol. It just causes too much friction during both development and deployment, for little to no (or negative) performance gain.
  - simiones 4y ago
    > My preferred setup is to have an unencrypted service running on 127.0.0.1 (so not publicly available), Don't forget that JS from any webpage can access your 127.0.0.1 to various degrees. Depending on what types of requests exactly the server accepts, it may be somewhat unsafe for a machine with a browser.

AceJohnny2 4y ago

Offtopic, but: > However, Google is not one single company, but a collection of independent and distrusting companies. This is an important thing to keep in mind when considering the behavior of any large company.

teaearlgraycold 4y ago
As a Googler, it's worse. Even within one of the "companies" there is distrust in the chain of command.
- Tyr42 4y ago
  Googler here. Yeah, it goes my boss, another guy I'm somewhat familiar with, then some cloud of VPs or something and then Sundar. I have no idea what those vp's are up to and not much faith in their decisions.
  - teaearlgraycold 4y ago
    When I joined I found it funny you can't DM Sundar. But you can DM SVPs. At least the SVP of my division is open on the chat app.

wonnage 4y ago

Trailers would be theoretically useful in a variety of HTML streaming-related cases if they actually had widespread support (but they don't): - sending down Server-Timing values for processing done after the headers are sent - updating the response status or redirecting after the headers are sent - deciding whether a response is cacheable after you've finished generating it All of these except the first one obviously break assumptions about HTTP and I'm not surprised they're unsupported. Firefox [1] actually supports the first case. The rest have workarounds, you can do a meta-refresh or a JS redirect, and you could simply not stream cacheable pages (assuming they'd generally be served from cache anyway). But it's still the case that frontend code generally likes to throw errors and trigger redirects in the course of rendering, rather than performing all that validation up front. That's sensible when you're rendering in a browser, but makes it hard to stream stuff with meaningful status codes.

fijiaarone 4y ago

Application layer encoding should not interfere in the protocol transport layer.

jayd16 4y ago
This is more about whether it is acceptable to push error checking to the application layer or not. Seems like the gRPC designers agreed with you and so they need trailers.

Matthias247 4y ago

> In this flow, what was the length of the /data resource? Since we don’t have a Content-Length, we are not sure the entire response came back. If the connection was closed, does it mean it succeeded or failed? We aren’t sure. I don’t get that argument. GRPC uses length prefixed protobuf messages. It is obvious for the peer if a complete message (inside a stream or single response) is received - with and without trailers. The only thing that trailer support adds is the ability to send an additional late response code. That could have been added also without trailers. Just put another length prefixed block inside the body stream, and add a flag before that differentiates trailers from a message. Essentially protobuf (application messages) in protobuf (definition of the response body stream). I assume someone thought trailers would be a neat thing that is already part of the spec and can do the job. But the bet didn’t work out since browsers and most other HTTP libraries didn’t found them worthwhile enough to fully support.

afc 4y ago
He offers two facts that I think explain this well enough: > Additionally, they chose to keep Protobuf as the default wire format, but allow other encodings too. And: > Since streaming is a primary feature of gRPC, we often will not know the length of the response ahead of time. These make sense; you'd enable servers to start streaming back the responses directly as they were generating them, before the length of the response could be known. Not requiring servers to hold the entire response can have drastic latency and memory/performance impact for large responses.
- Thorrez 4y ago
  This doesn't match what I see in the gRPC spec. It says every message must be length-prefixed. https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.md https://github.com/grpc/grpc/blob/master/doc/PROTOCOL-HTTP2.... Disclaimer: I don't know much about gRPC.
  - akshayshah 4y ago
    I've spent a fair bit of time working with gRPC, and you're correct - gRPC's length-prefixing makes it easy to detect when individual messages are terminated early. You do still need some way to detect streams that terminate unexpectedly on message boundaries - perhaps you could rely on HTTP/2 EOS bits as evidence of an application-level success, but you need some equivalent of trailers to communicate the details of any errors that occur midway through the response stream anyways.

mdriley 4y ago

It seems like a lot of other technologies in this space have solved the listed problems while remaining compatible with browsers, load balancers, reverse proxies, etc. It was a product choice not to offer a fallback path when HTTP/2 was unavailable. That choice made gRPC impossible to deploy in a lot of real-world environments. What motivated that choice?

lakomen 4y ago
Can you name one or more?
- mdriley 4y ago
  one relevant datapoint: https://devblogs.microsoft.com/dotnet/announcing-grpc-json-transcoding-for-dotnet/ https://devblogs.microsoft.com/dotnet/announcing-grpc-json-t...

alexcpn 4y ago

I use GRPC between micro services instead of REST and for that it is really great; All the deficiencies of REST - Non versioned, no typed goes away with GRPC and the protobuf is the official interface for all micro-services. No problems with this approach for over two years now; and also multi language support - We have Go and Java and Python and TypeScript micro-services now happily talking and getting new features and new interface methods updated. Maybe it was demise in the web space; but a jewel in micro-service space

marcyb5st 4y ago
This is more or less what stubby is/was for Google and so the original driving force in implementing it. Now, if you add a catch all service that translates the requests from the outside to Protobuffers and then forwards the translated requests to the correct service you have a GFE (Google Front-End) equivalent. Should you do it? Probably not as it's not just a dumb translation layer and it is extremely complex (e.g. needs to support streams which is non-trivial in this situation). For Google it's worth because this way you only have to handle protobuffers beyond the GFE layer.
withinboredom 4y ago
With GRPC, you lose the ability to introspect the data on-the-wire. You lose the ability to create optimized data formats for YOUR application (who said you have to use JSON?) Most people can’t implement REST correctly, so it has been a shitshow for the last 20 or so years, GRPC isn’t a magic bullet, it just forces you to solve problems (or helps you to solve them) that you should have been doing in the first place. You can do all of these things without GRPC, there is no power it grants you that can’t be done better or at all in your own libraries and specs.
- ackfoobar 4y ago
  I suppose you mean "inspect" the data on-the-wire https://grpc.io/blog/wireshark/ https://grpc.io/blog/wireshark/ Wireshark can load proto files and decode the data for you. BTW, "The Internet is running in debug mode".

akshayshah 4y ago

The author also posted an interesting Twitter thread a few months ago [0], on the day my coworkers and I posted here about our gRPC-compatible RPC framework [1]. I was a bit afraid to read this post, but I shouldn't have been - the author's a class act, and he never called us out explicitly. There's not much written about what the gRPC team was _thinking_ when they wrote up the protocol, and this was a nice window into how contemporaneous changes to HTTP and the fetch API shaped their approach. Given my current work, the final section ("Lessons for Designers") really hit home. That said, I didn't follow the central argument - that you need HTTP trailers to detect incomplete protobuf messages. What's not mentioned in the blog post is that gRPC wraps every protobuf message in a 5-byte envelope, and the bulk of the envelope is devoted to specifying the length of the enclosed message. It's easy to detect prematurely terminated messages, because they don't contain the promised number of bytes. The author says, "[i]t’s not hard to imagine that trailers would be less of an issue, if the default encoding was JSON," because JSON objects are explicitly terminated by a closing } - but it seems to me that envelopes solve that problem neatly. With incomplete message detection handled, we're left looking for some mechanism to detect streams that prematurely terminate at a message boundary. (This is more likely than you might expect, since servers often crash at message boundaries.) In practice, gRPC implementations already buffer responses to unary RPCs. It's therefore easy to use the standard HTTP Content-Length header for unary responses. This covers the vast majority of RPCs with a simple, uncontroversial approach. Streaming responses do need some trailer-like mechanism, but not to detect premature termination - as long as we're restricting ourselves to HTTP/2, cleanly terminated streams always end with a frame with the end of stream bit set. Streaming does need some trailer-like mechanism to send the details of any errors that occur mid-stream, but there's no need to use HTTP trailers. As the author hints, there's some unused space in the message envelope - we can use one bit to flag the last message in the stream and use it for the end-of-stream metadata. This is, more or less, what the gRPC-Web protocol is. (Admittedly, it's probably a bad idea to rely on _every_ HTTP server and proxy on the internet handling premature termination correctly. We need some sort of trailer-like construct anyways, and the fact that it also improves robustness is a nice extra benefit.) So from the outside, it doesn't seem like trailers improve the robustness of most RPCs. Instead, it seems like the gRPC protocol prioritizes some abstract notion of cleanliness over simplicity in practice: by using the same wire protocol for unary and streaming RPCs, everyday request-response workloads take on all the complexity of streaming. Even for streaming responses, the practical difficulties of working with HTTP trailers have also been apparent for years; I'm shocked that more of the gRPC ecosystem hasn't followed .NET's lead and integrated gRPC-Web support into servers. (If I had to guess, it's difficult because many of Google's gRPC implementations include their own HTTP/2 transport - adding HTTP/1.1 support is a tremendous expansion in scope. Presumably the same applies to HTTP/3, once it's finalized.) Again, though, I appreciated the inside look into the gRPC team's thinking. It takes courage to discuss the imperfections of your own work, especially when your former coworkers are still supporting the project. gRPC is far from perfect, but the engineers working on it are clearly skilled, experienced, and generally decent people. Hats off to the author - personally, I hope to someday write code influential enough that a retrospective makes the front page of HN :) 0: https://twitter.com/CarlMastrangelo/status/1532256576274243584 https://twitter.com/CarlMastrangelo/status/15322565762742435... 1: https://news.ycombinator.com/item?id=31584555 https://news.ycombinator.com/item?id=31584555

rswail 4y ago

Personal opinion: RPC is a failed architectural style, independent of what serialization/marshalling of arguments is used. it failed with CORBA, it failed with ONC-RPC, it failed with Java RMI. Remote Procedure Calls attempt to abstract away the networked nature of the function and make it "look like" a local function call. That's Just Wrong. When two networked services are communicating, the network must be considered. REST relies on the media type, links and the limited verb set to define the resource and the state transfer operations to change the state of the resource. HTTP explicitly incorporates the networked nature of the server/client relationship, independent of, and irrespective of, the underlying server or client implementation. Media types, separated from the HTTP networking, define the format and serialization of the resource representation independent of the network. HTTP/REST doesn't really support streaming.

akshayshah 4y ago
That's true of CORBA, for sure. I'm not familiar with ONC-RPC or Java RMI. It's not true of gRPC. It's not "RPC" in any traditional sense - it's just a particular HTTP convention, and the clients reflect that. They're asynchronous, make deadlines and transport errors first-class concepts, and make it easy to work with HTTP headers (and trailers, as the article explains). Calling a gRPC API with a generated client often doesn't feel too different from using a client library for a REST API. It's definitely a verb-oriented style, as opposed to REST's noun orientation. That's sometimes a plus, and sometimes a minus; it's the same "Kingdom of Nouns" debate [0] that's been going on about Java-style OOP for years. 0: http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom-of-nouns.html http://steve-yegge.blogspot.com/2006/03/execution-in-kingdom...
- rswail 4y ago
  The verb-oriented style is part of the problem. Too many verbs is the problem. Java/OOP problems are not the same as the REST style, which is entirely about Nouns. There's none of the Java ManagerFactoryManager problems. The generated client from an IDL that wraps the network protocol with a function call is also part of the problem. REST APIs that have function calls that aren't "Send Request and wait for Response" aren't REST. ("wait for response" doesn't imply synchronous implementation, but HTTP is a request/response oriented protocol).

kris-nova 4y ago

gRPC: protobuf and stubby for performance reasons, we’ve spared no expense.

lakomen 4y ago

To this day it's still not clear to me, as even if asked on their github issues there is no definite answer, Can one use nginx in front of a grpc serving backend if the client is a JS client in the broadest sense? This unanswered question is the main reason I'm still doing RESTful JSON.

jiggawatts 4y ago

Something that’s always bugged me about streaming protocols of this type is that they prevent processing pipelining. If trailers are used for things such as checksums, then the client must wait patiently for potentially gigabytes of data to stream to it before it can verify the data integrity and start processing it safely. If the data is sent chunked, then this is not an issue. The client can start decoding chunks as they arrive, each one with a separate checksum.

crest 4y ago
I would argue that RPC systems shouldn't be burdened down with features like this. If you want to exchange more date than either endpoint can buffer comfortably in memory split it up into several RPC messages e.g. create large object, define byte range, seal object. If the producer can precompute the length and hash you can simplify things by using content addressing to reference such objects and replicate them. If you can afford the overhead breaking them up into Merkle DAGs reasonably sized leaves is a good idea to allow validating and resuming partial transfers. It matters if your devices are connected via PCIe in a single chassis or mobile networks spread over half the world and datacenter optimised protocol won't be optimal for expensive, slow, unreliable links.
jmillikin 4y ago
This comment is mixing up a few concerns. 1. When transferring large amounts of data, the checksum for the full transfer can't be verified until all the data is received. If you want to (for example) download an Ubuntu ISO and verify its checksum before installing it, you'll have to buffer the data somewhere until the download finishes. 2. When transferring small amounts of data, such as individual chunks, the data integrity is (/ should be) automatically verified by the encryption layer[0] of the underlying transport. There's no point in putting a shasum into each chunk because if a bit gets flipped in transit then that chunk will never even arrive in your message handler. 3. In gRPC, chunking large data transfers is mandatory because the library will reject Protobuf messages larger than a few megabytes[1]. As the chunks of data arrive, they can be passed into a hash function at the same time as you're buffering them to disk. [0] gRPC supports running without encryption for local development, but obviously for real workloads you'd do end-to-end TLS. [1] IIRC the default for C++ and Go implementations of gRPC is 4 MiB, which can be overridden when the client/server is being initialized. For bulk data transfer there's also the Protobuf hard limit of 2GB[2] for variable-length data. [2] https://developers.google.com/protocol-buffers/docs/encoding https://developers.google.com/protocol-buffers/docs/encoding

AtNightWeCode 4y ago

Very nice post. Http2 did not solve the TCP HOL problems though. Not sure about the WS statement. On the other hand. Vanilla WS has never ended up in prod on any of my projects even if it has been implemented several times.

jenia2022 4y ago

I'm not getting it. Why is HTTP so inadequate for gRPC? A service app for example can open 1000 sockets with a server and simply multiplex that way.