3 ms·

gRPC: From service definition to wire format

158 points by latonz 8mo ago

tempest_ 8mo ago
I like the idea of grpc because I wanted the contract but I tried it on a small service and I think I would avoid it in the future. Too many rough edges and features I didnt really need. I was using it in Rust and python mainly (maybe it is better in Go?) but it had a whole bunch of google stuff in there I didnt need. - Configuring the python client with a json string that did not seem to have a documented schema - Error types that were overly general in some ways and overly specific in other ways - HAProxy couldn't easily health check the service There were a few others that I cant remember because it was ~5 years ago. I liked the idea of the contract and protobuf seemed easy to write but had no need for client side dns load balancing and the like and was not working in GoLang.
- kjuulh 8mo ago
  I think connect-rpc[0] strikes a good balance between normal http apis, and gRPC. It allows protobuf as json. So you could think of it as an opinionated http api spec. A health check would be just a call to an url /something.v1.MyService/MyMethod -d { "input": "something }. it works really well, and the tooling is pretty good, though it isn't that widely supported yet. Rust for one doesn't have an implementation. But I've been using it at work, and we basically haven't had any issues with it (go and typescript). But the good thing is that it can interoperate with normal grpc servers, etc. But that of course locks it into using the protobuf wireformat, which is part of the trouble ;) 0: https://connectrpc.com/ https://connectrpc.com/
  - jamesu 8mo ago
    Using connectrpc was a pretty refreshing experience for me. Implementing a client for the HTTP stuff at least is pretty easy! I was able to implement a basic runner for forgejo using the protobuf spec for the runner + libcurl within a few days.
  - selfmodruntime 8mo ago
    +1 for connect-rpc. After go-micro‘s maintainer went off the rails, I ripped it all out and switched to connect.
    - smnscu 8mo ago
      > After go-micro‘s maintainer went off the rails What do you mean by this? Genuinely curious, as someone who's followed that project in the past.
      - asim 8mo ago
        He probably means when I took VC funding in 2019 and started to rip apart the framework to try build a platform and business. The 2-3 years after were very chaotic. My goal was never to serve the community but instead leverage it to build a business. Ultimately that failed. The truth is it's very difficult to sustain open source. Go-micro was never the end goal. It was always a stepping stone to a platform e.g microservices PaaS. A lot of hard lessons learned along the way. Now with Copilot and AI I'm able to go back and fix a lot of issues but nothing will fix trust with a community or the passage of time. People move on. It served a certain purpose at certain time. Note: The company behind connect-rpc raised $100m but for more of a build system around protobuf as opposed to the rpc framework but this was my thinking as well. The ability to raise $10-20m would create the space to build the platform off the back of the success of the framework.
        smnscu 8mo ago
        Obligatory "this is why I love HN" but even for that standard, this is is an incredibly open account, thank you for the insight and sorry it hasn't seemed to pan out quite how you hoped. Still sounds like you got your bag, built something cool, and have your "micro" share of Internet legacy, so not too bad eh?
- dewey 8mo ago
  I've only enjoyed using Protobuf + gRPC after we've started using https://buf.build https://buf.build. Before that it was always a pain with Makefiles building obscure commands, then developers having different versions of the Protobuf compiler installed and all kinds of paper cuts like that. Now it's just "buf generate", every developer has the exact same settings defined in the repo and on the frontend side we are just importing the generated Typescript client and have all the types instantly available there. Also nice to have a hosted documentation to link people to. My experience is mostly with Go, Python and TS.
  - PessimalDecimal 8mo ago
    buf.build sounds interesting as a middle ground for using protos without going all-in on the Bazel build ecosystem.
- neRok 8mo ago
  > - Configuring the python client with a json string that did not seem to have a documented schema I'm far from an expert, yet I came to believe that what you've described is basically "code smell". And the smell probably comes from seemingly innocuous things like enum's. And you wondered if the solution was using Go, but no, it isn't. I was actually Go at the time myself (this was a few years ago, and I used Twirp instead of Protobuf) - but I realised that RDBMS > "Server(Go)" layer had quirks, and then the "Server(Go)" > "API(JS)" had other quirks -- and so I realised that you may as well "splat" out every attribute/relationship. Because ultimately, that's the problem... Eg: is it a null field, or undefined, or empty, or false, or [], or {}? ... [] == my valentines day inbox. :P
allanrbo 8mo ago
For those situations where you need just a little bit of protobuf in your project, and don't want to bother with the whole proto ecosystem of codegen and deps: https://github.com/allanrbo/pb.py https://github.com/allanrbo/pb.py
est 8mo ago
> The contract-first philosophy gRPC/protobuf is largely a Google cult. I've seen too projects with complex business logic simply give up and embed JSON strings inside pb. Like WTF...? Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly. As it occured to me, gRPC could optionally just serve those .proto files in the initial h2 handshake on the wire. It add just few kilobytes but solves a big problem.
- jcgrillo 8mo ago
  I personally really like gRPC and protobufs. I think they strike a good balance between a number of indirectly competing objectives. However I completely agree with your observation that as soon as you move beyond a single source of truth for the .proto files it all goes to shit. I've seen some horrible things--generated code being committed to version control and copied between repos, .proto files duplicated and manually kept up to date (or not). Both had hilarious failure modes. There is no viable synchronization mechanism except to ensure that each .proto file is defined in exactly one place, that each time someone touches a .proto file all the downstream dependencies on that file are updated--everyone who consumes any code generated from that .proto--and that for every such change clients are deployed before servers. Usually these invariants are maintained by meatspace protocols which invariably fail.
  - jeffbee 8mo ago
    I don't see why any of that would be necessary. There are simple rules for protobuf compatibility and people only need to follow them. Never re-use a field number to mean something else. Never change the type of a field. That's it. Those are the only rules. If you follow them you don't have to think about any of that stuff that you mentioned.
    - jcgrillo 8mo ago
      Absolutely! Forward and backward compatibility are one of the wonderful things about protobufs. And that all goes wrong when you try to define the interface in more than one place. EDIT: also, although the wire protocol may tolerate unknown or missing data, almost always the application doesn't. EDIT AGAIN: I'm not saying this is how it should be just that this is the low energy state the socio-technical system seems to arrive at over time. So ideally it should be simple but due to imperfect decisions it gets horribly complicated over time.
cyberax 8mo ago
Protobuf is good, but it's not perfect. The handling of "oneof" fields is weird and the Python bindings were written by drunk squirrels, enums are strange, etc. gRPC is terrible, but ConnectRPC allows sane integration of PB with regular browser clients. Buf.build also has a lot of helpful tools, like backwards compatibility checking. But it's not worse than other alternatives like Thrift. And waaaaaaaaaayyyyyy better than OpenAPI monstrosities.
pm90 8mo ago
My gripe with grpc is that it doesn’t play super well with kubernetes services… you have to take a little bit of care, you need to understand how k8s services work and you have to understand how load balancing in grpc works. Ideally I would want to use protobuf as an interchange format, and a “dumb” http server that understands that. That being said… once you do configure it properly it can be a powerful tool. The complexity though is usually not worth it unless you’re at a certain scale.
- whs 8mo ago
  I wrote gRPC xDS server for Kubernetes that is configuration-free. Basically just load xDS client library into your code, then use xds:///servicename.namespace (unlike DNS, namespace is always required). It should be as lightweight and scales in similar way to the cluster DNS. My company run this exact code in production since it was created in 2022. We probably have several times more than 1000 rps gRPC requests running internally including over the public internet for hybrid cloud connectivity. That being said, gRPC's xDS client is not always bugs-free. https://github.com/wongnai/xds https://github.com/wongnai/xds
- DaGardner 8mo ago
  Have you checked out https://connectrpc.com/ https://connectrpc.com/ We are using this for the basic HTTP server/client compatibility. And that means we can also use from web a context without any proxy setup.
- jeffbee 8mo ago
  If you just want to send a protobuf to a host:port there's no reason you can't do that with gRPC. Client load balancers are something you can optionally layer on top.
aanet 8mo ago
I’m old enough to remember the days of CORBA (Orbix, Iona, BEA, anyone ??) and its IDL, the IDL compiler, stubs and other props and doodads. Ah yes, and the registry as well, where the services registered themselves, and the discovery mechanisms. gRPC (a very Googly thing) took it all, hook line and sinker and made it URLesque Can’t recall how the ORB overhead has been resolved in gRPCs
- speedbird 8mo ago
  And still I found it easier to use than gRPC.
- jeffbee 8mo ago
  There is nothing analogous to the ORB in gRPC, so your complaint seems hallucinated. Callers are entirely responsible for figuring out where their services are. gRPC holds no opinions whatsoever about how your servers are named or how their lifecycles are managed. There is not a "bus".

158 points by latonz 8mo ago

I like the idea of grpc because I wanted the contract but I tried it on a small service and I think I would avoid it in the future. Too many rough edges and features I didnt really need. I was using it in Rust and python mainly (maybe it is better in Go?) but it had a whole bunch of google stuff in there I didnt need. - Configuring the python client with a json string that did not seem to have a documented schema - Error types that were overly general in some ways and overly specific in other ways - HAProxy couldn't easily health check the service There were a few others that I cant remember because it was ~5 years ago. I liked the idea of the contract and protobuf seemed easy to write but had no need for client side dns load balancing and the like and was not working in GoLang.

kjuulh 8mo ago
I think connect-rpc[0] strikes a good balance between normal http apis, and gRPC. It allows protobuf as json. So you could think of it as an opinionated http api spec. A health check would be just a call to an url /something.v1.MyService/MyMethod -d { "input": "something }. it works really well, and the tooling is pretty good, though it isn't that widely supported yet. Rust for one doesn't have an implementation. But I've been using it at work, and we basically haven't had any issues with it (go and typescript). But the good thing is that it can interoperate with normal grpc servers, etc. But that of course locks it into using the protobuf wireformat, which is part of the trouble ;) 0: https://connectrpc.com/ https://connectrpc.com/
- jamesu 8mo ago
  Using connectrpc was a pretty refreshing experience for me. Implementing a client for the HTTP stuff at least is pretty easy! I was able to implement a basic runner for forgejo using the protobuf spec for the runner + libcurl within a few days.
- selfmodruntime 8mo ago
  +1 for connect-rpc. After go-micro‘s maintainer went off the rails, I ripped it all out and switched to connect.
  - smnscu 8mo ago
    > After go-micro‘s maintainer went off the rails What do you mean by this? Genuinely curious, as someone who's followed that project in the past.
    - asim 8mo ago
      He probably means when I took VC funding in 2019 and started to rip apart the framework to try build a platform and business. The 2-3 years after were very chaotic. My goal was never to serve the community but instead leverage it to build a business. Ultimately that failed. The truth is it's very difficult to sustain open source. Go-micro was never the end goal. It was always a stepping stone to a platform e.g microservices PaaS. A lot of hard lessons learned along the way. Now with Copilot and AI I'm able to go back and fix a lot of issues but nothing will fix trust with a community or the passage of time. People move on. It served a certain purpose at certain time. Note: The company behind connect-rpc raised $100m but for more of a build system around protobuf as opposed to the rpc framework but this was my thinking as well. The ability to raise $10-20m would create the space to build the platform off the back of the success of the framework.
      - smnscu 8mo ago
        Obligatory "this is why I love HN" but even for that standard, this is is an incredibly open account, thank you for the insight and sorry it hasn't seemed to pan out quite how you hoped. Still sounds like you got your bag, built something cool, and have your "micro" share of Internet legacy, so not too bad eh?
dewey 8mo ago
I've only enjoyed using Protobuf + gRPC after we've started using https://buf.build https://buf.build. Before that it was always a pain with Makefiles building obscure commands, then developers having different versions of the Protobuf compiler installed and all kinds of paper cuts like that. Now it's just "buf generate", every developer has the exact same settings defined in the repo and on the frontend side we are just importing the generated Typescript client and have all the types instantly available there. Also nice to have a hosted documentation to link people to. My experience is mostly with Go, Python and TS.
- PessimalDecimal 8mo ago
  buf.build sounds interesting as a middle ground for using protos without going all-in on the Bazel build ecosystem.
neRok 8mo ago
> - Configuring the python client with a json string that did not seem to have a documented schema I'm far from an expert, yet I came to believe that what you've described is basically "code smell". And the smell probably comes from seemingly innocuous things like enum's. And you wondered if the solution was using Go, but no, it isn't. I was actually Go at the time myself (this was a few years ago, and I used Twirp instead of Protobuf) - but I realised that RDBMS > "Server(Go)" layer had quirks, and then the "Server(Go)" > "API(JS)" had other quirks -- and so I realised that you may as well "splat" out every attribute/relationship. Because ultimately, that's the problem... Eg: is it a null field, or undefined, or empty, or false, or [], or {}? ... [] == my valentines day inbox. :P

allanrbo 8mo ago

For those situations where you need just a little bit of protobuf in your project, and don't want to bother with the whole proto ecosystem of codegen and deps: https://github.com/allanrbo/pb.py https://github.com/allanrbo/pb.py

est 8mo ago

> The contract-first philosophy gRPC/protobuf is largely a Google cult. I've seen too projects with complex business logic simply give up and embed JSON strings inside pb. Like WTF...? Everything was good in the begining, as long as everyone submits their .proto to a centralized repo. Once the one team starts to host their own, things get broken quickly. As it occured to me, gRPC could optionally just serve those .proto files in the initial h2 handshake on the wire. It add just few kilobytes but solves a big problem.

jcgrillo 8mo ago
I personally really like gRPC and protobufs. I think they strike a good balance between a number of indirectly competing objectives. However I completely agree with your observation that as soon as you move beyond a single source of truth for the .proto files it all goes to shit. I've seen some horrible things--generated code being committed to version control and copied between repos, .proto files duplicated and manually kept up to date (or not). Both had hilarious failure modes. There is no viable synchronization mechanism except to ensure that each .proto file is defined in exactly one place, that each time someone touches a .proto file all the downstream dependencies on that file are updated--everyone who consumes any code generated from that .proto--and that for every such change clients are deployed before servers. Usually these invariants are maintained by meatspace protocols which invariably fail.
- jeffbee 8mo ago
  I don't see why any of that would be necessary. There are simple rules for protobuf compatibility and people only need to follow them. Never re-use a field number to mean something else. Never change the type of a field. That's it. Those are the only rules. If you follow them you don't have to think about any of that stuff that you mentioned.
  - jcgrillo 8mo ago
    Absolutely! Forward and backward compatibility are one of the wonderful things about protobufs. And that all goes wrong when you try to define the interface in more than one place. EDIT: also, although the wire protocol may tolerate unknown or missing data, almost always the application doesn't. EDIT AGAIN: I'm not saying this is how it should be just that this is the low energy state the socio-technical system seems to arrive at over time. So ideally it should be simple but due to imperfect decisions it gets horribly complicated over time.

cyberax 8mo ago

Protobuf is good, but it's not perfect. The handling of "oneof" fields is weird and the Python bindings were written by drunk squirrels, enums are strange, etc. gRPC is terrible, but ConnectRPC allows sane integration of PB with regular browser clients. Buf.build also has a lot of helpful tools, like backwards compatibility checking. But it's not worse than other alternatives like Thrift. And waaaaaaaaaayyyyyy better than OpenAPI monstrosities.

pm90 8mo ago

My gripe with grpc is that it doesn’t play super well with kubernetes services… you have to take a little bit of care, you need to understand how k8s services work and you have to understand how load balancing in grpc works. Ideally I would want to use protobuf as an interchange format, and a “dumb” http server that understands that. That being said… once you do configure it properly it can be a powerful tool. The complexity though is usually not worth it unless you’re at a certain scale.

whs 8mo ago
I wrote gRPC xDS server for Kubernetes that is configuration-free. Basically just load xDS client library into your code, then use xds:///servicename.namespace (unlike DNS, namespace is always required). It should be as lightweight and scales in similar way to the cluster DNS. My company run this exact code in production since it was created in 2022. We probably have several times more than 1000 rps gRPC requests running internally including over the public internet for hybrid cloud connectivity. That being said, gRPC's xDS client is not always bugs-free. https://github.com/wongnai/xds https://github.com/wongnai/xds
DaGardner 8mo ago
Have you checked out https://connectrpc.com/ https://connectrpc.com/ We are using this for the basic HTTP server/client compatibility. And that means we can also use from web a context without any proxy setup.
jeffbee 8mo ago
If you just want to send a protobuf to a host:port there's no reason you can't do that with gRPC. Client load balancers are something you can optionally layer on top.

aanet 8mo ago

I’m old enough to remember the days of CORBA (Orbix, Iona, BEA, anyone ??) and its IDL, the IDL compiler, stubs and other props and doodads. Ah yes, and the registry as well, where the services registered themselves, and the discovery mechanisms. gRPC (a very Googly thing) took it all, hook line and sinker and made it URLesque Can’t recall how the ORB overhead has been resolved in gRPCs

speedbird 8mo ago
And still I found it easier to use than gRPC.
jeffbee 8mo ago
There is nothing analogous to the ORB in gRPC, so your complaint seems hallucinated. Callers are entirely responsible for figuring out where their services are. gRPC holds no opinions whatsoever about how your servers are named or how their lifecycles are managed. There is not a "bus".