4 ms·

Can people share their experience and success stories with gRPC and/or GraphQL? I've tried both in the past and found that APIs are rarely as static as typed A

Can people share their experience and success stories with gRPC and/or GraphQL? I've tried both in the past and found that APIs are rarely as static as typed API systems require. Many APIs have fields that are required conditionally based on the value of other fields. Other times the schema is dynamic and workarounds are required like stuffing things into a JSON object and making it a "string" in the API. This added a bunch of work to marshall it in and out of a custom workaround format. In both cases (gRPC and GraphQL) we've run into so many issues that the technology was abandoned and we fell back to plain JSON REST.

cobookman 8y ago
You can have conditional protobuf fields / messages. Checkout: - Any: https://developers.google.com/protocol-buffers/docs/proto3#any https://developers.google.com/protocol-buffers/docs/proto3#a... - OneOf: https://developers.google.com/protocol-buffers/docs/proto3#oneof https://developers.google.com/protocol-buffers/docs/proto3#o... I believe Any is the equivalent to having a proto in format of: ``` message MyMessage { string type = 1; bytes data = 2; } ```
- buckhx 8y ago
  Also FieldMasks and the View patterns can be very useful for the other issues that were raised https://cloud.google.com/apis/design/design_patterns#partial_response https://cloud.google.com/apis/design/design_patterns#partial...
yashap 8y ago
Have used both, and don’t like either. I actually DO like very statically typed schemas, but you can easily do this with json over REST, and using IDLs to generate SDKs. For gRPC vs. REST, I just don’t see the benefits. Performance wins of gRPC are extremely minimal for most backends, ”push based” workflows can easily be handled with webhooks (or message queues), so why am I throwing out extremely mature technology (and tooling/middleware!) that every dev is very familiar with? For GraphQL, reducing client/datacentre round trips is nice, but can also easily be achieved with RESTful “coordination” services, that serve the front end specifically, and do nothing but stringing together calls to backend services. Why force people to learn another complex technology for that? “Returning smaller payloads” makes minimal impact in practice, and GraphQL requests are MASSIVE compared to REST requests, so you often end up with a performance loss, not gain. Also, GraphQL requests are way harder to glance at in the browser, you can’t just look for GET /dog/123, you have to peek into all the request bodies. And again, all the great REST middleware stops working, because you’re doing RPC. Basically I see them both as extra complexity with little payoff.
- cheapsteak 8y ago
  Curious to know if you would count "ecosystem" as a point towards either? Afaik there isn't a standardized way of creating statically typed schemas for REST. A standard statically typed schema like GraphQL's allows a community to create and maintain tooling around it. A person would create"[some tool] for GraphQL" instead of "[some tool] for [specific api]"
  - bcheung 8y ago
    There's Open API Specification (Swagger). It's more documentation than strict protocol enforcement though.
  - bcheung 8y ago
    In theory, yes, but in the case of GraphQL, I found it was very difficult to generate schemas dynamically. Factoring out common fields was not possible. Common fields in schemas had to be duplicated and there was no concept of an include/mixin/extends/etc. Best technique was generating plaintext strings of GraphQL programmatically (losing any editor or type checking support). Dropping down to the AST layer of GraphQL is a bit more programmatic but not really an ideal level of abstraction to work at. It was too verbose and low level. REST integrates well with native JS constructs (it's just data) so it's much easier to compose and use metaprogramming techniques.
- zbentley 8y ago
  > For GraphQL, reducing client/datacentre round trips is nice, but can also easily be achieved with RESTful “coordination” services, that serve the front end specifically, and do nothing but stringing together calls to backend services. I mean . . . that's all a GraphQL "server" (or parser/aggregator endpoint) is. > Why force people to learn another complex technology for that? Because most later-stage 'RESTful “coordination” services' like you describe probably already include: 1) a DSL of some kind for expressing what data you like; 2) significant complexity for expressing grouping or deduplication logic; 3) complex requirements for coordinated backend services so that their "raw" data can be rolled up un an efficient and uniform way by the coordinator. That's complex, and new to most people--pretty much everyone who starts work on it, if you built it yourself. GraphQL queries and aggregation layers are similarly complex, but new to fewer people, and better supported. I think there are some big problems with GraphQL as a concept/technology (I am currently pushing back against its adoption in my area), but the above are not the right criticisms.
buckhx 8y ago
We've been using gRPC for a new web project and it has been a good experience so far. The codegen in JS is pretty verbose and can have touch ergonomics, but not the end of the world. We haven't ran into troubles with the type system, but we leverage OneOf's and Views a decent amount. The thing we don't have a great solution for right now is how to fetch referenced types efficiently (eg. stream of posts with users referenced by a foreign key), but we're not tackling that until performance of the resource fetches becomes a problem and evaluating whether to cache values client side or design a solution at the API layer.
helper 8y ago
We've been using gRPC since late 2015 for communication between IOT devices and cloud hosted servers. We chose gRPC for the following reasons: bi-directional streaming support; TCP connection multiplexing; mutual authentication with client TLS certs; a large company backing it so that it would likely continue to be supported in the future. In other words gRPC fit our use case nicely and if it didn't exist at the time we would have built our own crappy protocol ourselves. I think gRPC is great for server-to-server communication and use cases where you need bi-directional streaming. I would not use it for an externally facing API. I would not recommend it for browser-to-server communication unless you are already using gRPC other places in your stack. If you want the IDL you can use protobufs over plain HTTP. GraphQL solves a very different problem and I would generally not consider it as an alternative to gRPC.
- be_erik 8y ago
  Agreed, I think most people miss the point that most protobuf implementations support JSON and/or binary across the wire. We flip an accept flag for the frontend and we get well structured JSON.
be_erik 8y ago
We use both, gRPC for backend services and a GraphQL server that aggregates and communicates with those services for the frontend. Because the backend is all described with .proto files its trivial to autogenerate most of the GraphQL server that serves the fronted. Tends to be the best of both worlds, IMO.
zbentley 8y ago
> I've tried both in the past and found that APIs are rarely as static as typed API systems require. Many APIs have fields that are required conditionally based on the value of other fields. Other times the schema is dynamic and workarounds are required like stuffing things into a JSON object and making it a "string" in the API. There are big advantages to strict, mostly-static, clearly typed APIs, but you're right that adoption is often tricky. The problem isn't the nature of typed APIs, or missing capabilities in the data transfer formats/communication libraries themselves. The problem is tooling. Put another way: the tipping point (between untyped/whatever-you-like JSON and something structured like gRPC) is not how easy it is to use already-existent strict/typed APIs, but how easy it is to add, deploy, and change them. API and client developers will stop pushing back on a structured format, or adding workarounds (like stuffing random JSON into string fields) if the tooling that supports changing API definitions is excellent: if they can easily change the minimum of code or type definitions to make the data-"shape" (schema) changes they want, and immediately be able to exercise their changes across multiple backend APIs and frontend clients (with obvious and predictable behaviors around versioning and backwards/forwards compatibility), and can have confidence that they can deploy those changes to real users in a seamless and low-touch way (e.g. no "everyone please update your endpoints to /api/v804.7/"), then resistance/workarounds to structured API patterns will disappear. There's some expertise/discipline involved in getting those things right, but they need to be easy first. The companies that have really gotten huge benefits from stricter, more uniform, generated and/or typed API definitions (for both internal service interactions and external clients) aren't all huge companies, they aren't all using the same tech stacks, and they aren't all in the same kinds of markets. The thing they have in common is a huge investment in good tooling around how to create, modify, and deploy cross-platform typed APIs.

cobookman 8y ago

You can have conditional protobuf fields / messages. Checkout: - Any: https://developers.google.com/protocol-buffers/docs/proto3#any https://developers.google.com/protocol-buffers/docs/proto3#a... - OneOf: https://developers.google.com/protocol-buffers/docs/proto3#oneof https://developers.google.com/protocol-buffers/docs/proto3#o... I believe Any is the equivalent to having a proto in format of: ``` message MyMessage { string type = 1; bytes data = 2; } ```

buckhx 8y ago
Also FieldMasks and the View patterns can be very useful for the other issues that were raised https://cloud.google.com/apis/design/design_patterns#partial_response https://cloud.google.com/apis/design/design_patterns#partial...

yashap 8y ago

Have used both, and don’t like either. I actually DO like very statically typed schemas, but you can easily do this with json over REST, and using IDLs to generate SDKs. For gRPC vs. REST, I just don’t see the benefits. Performance wins of gRPC are extremely minimal for most backends, ”push based” workflows can easily be handled with webhooks (or message queues), so why am I throwing out extremely mature technology (and tooling/middleware!) that every dev is very familiar with? For GraphQL, reducing client/datacentre round trips is nice, but can also easily be achieved with RESTful “coordination” services, that serve the front end specifically, and do nothing but stringing together calls to backend services. Why force people to learn another complex technology for that? “Returning smaller payloads” makes minimal impact in practice, and GraphQL requests are MASSIVE compared to REST requests, so you often end up with a performance loss, not gain. Also, GraphQL requests are way harder to glance at in the browser, you can’t just look for GET /dog/123, you have to peek into all the request bodies. And again, all the great REST middleware stops working, because you’re doing RPC. Basically I see them both as extra complexity with little payoff.

cheapsteak 8y ago
Curious to know if you would count "ecosystem" as a point towards either? Afaik there isn't a standardized way of creating statically typed schemas for REST. A standard statically typed schema like GraphQL's allows a community to create and maintain tooling around it. A person would create"[some tool] for GraphQL" instead of "[some tool] for [specific api]"
- bcheung 8y ago
  There's Open API Specification (Swagger). It's more documentation than strict protocol enforcement though.
- bcheung 8y ago
  In theory, yes, but in the case of GraphQL, I found it was very difficult to generate schemas dynamically. Factoring out common fields was not possible. Common fields in schemas had to be duplicated and there was no concept of an include/mixin/extends/etc. Best technique was generating plaintext strings of GraphQL programmatically (losing any editor or type checking support). Dropping down to the AST layer of GraphQL is a bit more programmatic but not really an ideal level of abstraction to work at. It was too verbose and low level. REST integrates well with native JS constructs (it's just data) so it's much easier to compose and use metaprogramming techniques.
zbentley 8y ago
> For GraphQL, reducing client/datacentre round trips is nice, but can also easily be achieved with RESTful “coordination” services, that serve the front end specifically, and do nothing but stringing together calls to backend services. I mean . . . that's all a GraphQL "server" (or parser/aggregator endpoint) is. > Why force people to learn another complex technology for that? Because most later-stage 'RESTful “coordination” services' like you describe probably already include: 1) a DSL of some kind for expressing what data you like; 2) significant complexity for expressing grouping or deduplication logic; 3) complex requirements for coordinated backend services so that their "raw" data can be rolled up un an efficient and uniform way by the coordinator. That's complex, and new to most people--pretty much everyone who starts work on it, if you built it yourself. GraphQL queries and aggregation layers are similarly complex, but new to fewer people, and better supported. I think there are some big problems with GraphQL as a concept/technology (I am currently pushing back against its adoption in my area), but the above are not the right criticisms.

buckhx 8y ago

We've been using gRPC for a new web project and it has been a good experience so far. The codegen in JS is pretty verbose and can have touch ergonomics, but not the end of the world. We haven't ran into troubles with the type system, but we leverage OneOf's and Views a decent amount. The thing we don't have a great solution for right now is how to fetch referenced types efficiently (eg. stream of posts with users referenced by a foreign key), but we're not tackling that until performance of the resource fetches becomes a problem and evaluating whether to cache values client side or design a solution at the API layer.

helper 8y ago

We've been using gRPC since late 2015 for communication between IOT devices and cloud hosted servers. We chose gRPC for the following reasons: bi-directional streaming support; TCP connection multiplexing; mutual authentication with client TLS certs; a large company backing it so that it would likely continue to be supported in the future. In other words gRPC fit our use case nicely and if it didn't exist at the time we would have built our own crappy protocol ourselves. I think gRPC is great for server-to-server communication and use cases where you need bi-directional streaming. I would not use it for an externally facing API. I would not recommend it for browser-to-server communication unless you are already using gRPC other places in your stack. If you want the IDL you can use protobufs over plain HTTP. GraphQL solves a very different problem and I would generally not consider it as an alternative to gRPC.

be_erik 8y ago
Agreed, I think most people miss the point that most protobuf implementations support JSON and/or binary across the wire. We flip an accept flag for the frontend and we get well structured JSON.

be_erik 8y ago

We use both, gRPC for backend services and a GraphQL server that aggregates and communicates with those services for the frontend. Because the backend is all described with .proto files its trivial to autogenerate most of the GraphQL server that serves the fronted. Tends to be the best of both worlds, IMO.

zbentley 8y ago

> I've tried both in the past and found that APIs are rarely as static as typed API systems require. Many APIs have fields that are required conditionally based on the value of other fields. Other times the schema is dynamic and workarounds are required like stuffing things into a JSON object and making it a "string" in the API. There are big advantages to strict, mostly-static, clearly typed APIs, but you're right that adoption is often tricky. The problem isn't the nature of typed APIs, or missing capabilities in the data transfer formats/communication libraries themselves. The problem is tooling. Put another way: the tipping point (between untyped/whatever-you-like JSON and something structured like gRPC) is not how easy it is to use already-existent strict/typed APIs, but how easy it is to add, deploy, and change them. API and client developers will stop pushing back on a structured format, or adding workarounds (like stuffing random JSON into string fields) if the tooling that supports changing API definitions is excellent: if they can easily change the minimum of code or type definitions to make the data-"shape" (schema) changes they want, and immediately be able to exercise their changes across multiple backend APIs and frontend clients (with obvious and predictable behaviors around versioning and backwards/forwards compatibility), and can have confidence that they can deploy those changes to real users in a seamless and low-touch way (e.g. no "everyone please update your endpoints to /api/v804.7/"), then resistance/workarounds to structured API patterns will disappear. There's some expertise/discipline involved in getting those things right, but they need to be easy first. The companies that have really gotten huge benefits from stricter, more uniform, generated and/or typed API definitions (for both internal service interactions and external clients) aren't all huge companies, they aren't all using the same tech stacks, and they aren't all in the same kinds of markets. The thing they have in common is a huge investment in good tooling around how to create, modify, and deploy cross-platform typed APIs.