5 ms·

Rejoiner: Unified GraphQL schema from gRPC microservices

108 points by patrickaljord 9y ago

brango 9y ago
Looks interesting, but TBH I'm still waiting for full swagger spec generation. It should be easy to go from grpc->swagger to generate REST client libs. Unfortunately in practice the swagger generator has many holes in it making it useless in several use cases. So... without being able to generate swagger specs it looks like gRPC is no use for a large number of use cases. Shame really since it has promise but still seems half-baked.
- bpicolo 9y ago
  That's pretty much exactly what tools like https://github.com/grpc-ecosystem/grpc-gateway https://github.com/grpc-ecosystem/grpc-gateway are doing
  - brango 9y ago
    I know. It doesn't work well enough. If Google focussed on this I think gRPC would gain massive ground. But issues are left open for months without any response, etc. We've had to drop gRPC as a direct result of this.
    - jsjohnst 9y ago
      > But issues are left open for months without any response, etc. Code is open source, fork and then submit a PR?
      - brango 9y ago
        No one bothered replying to my issue and suggested fix. I'm not doing work without maintainers agreeing my approach. So no.
        ithkuil 9y ago
        Care to share a link to the issue / pr?
        jsjohnst 9y ago
        If you aren’t willing to put in any effort, then I guess the problem is you getting in your own way. Good luck!
        zapita 9y ago
        The person you're replying to clearly is willing to put in effort, but requires a minimum of guidance from the maintainers of the project. That's perfectly reasonable, and your passive-aggressive response is not justified.
- bognition 9y ago
  > It should be easy to go from grpc->swagger to generate REST client libs. I theory yes, but would you want to? In my experience I've found that building REST services is very different than building a GRPC service. The contract between the client and server are totally different. > gRPC is no use for a large number of use cases. Shame really since it has promise but still seems half-baked. gRPC is half-baked? You realize that the a large portion of the google backend is built on top of gRPC. Its an extremely well tested tool.
  - brango 9y ago
    I'd want to because grpc doesn't support javascript on the web. REST is the defacto standard. I am aware Google use gRPC in their SDKs and backends, none of which are JS web. So yes, I want to build microservices with gRPC but expose them with REST for legacy/external clients.
    - pjmlp 9y ago
      I imagine that could be done via native JS arrays or a WebAssembly module, just not sure if it would be an improvement versus REST(JSONP) from performance point of view.
    - anameaname 9y ago
      The problem with web is that browsers don't expose trailers. gRPC depends on having trailers to know if the RPC succeeded or failed, but neither Chrome nor Firefox, nor any other browsers surface this information. If this was exposed in the XHR or Fetch API, gRPC would work. It's browsers in this case that are the laggard (trailers have been part of the spec since HTTP/1.1).
      - muxator 9y ago
        Sorry, what is a trailer?
  - tejasmanohar 9y ago
    How much of the Google backend is actually built on top of gRPC? My (outsider's) understanding is gRPC was the "open-source version/rewrite" (typical Google) of Google's internal software, stubby. IIRC, _some_ of the Google Cloud public-facing APIs use it, but what else? As a proponent of gRPC, I can say that some of the libraries, tools, docs, etc. are definitely half-baked. There are plenty of severe issues in the (confusing codebase of) grpc-go that have been open for over 1.5y.
KenanSulayman 9y ago
> Note: This is not an official Google product.
- eugenhotaj 9y ago
  Yea not directly supported by Google, but being built by Google engineer(s).
  - thangngoc89 9y ago
    This is from the README of the project. I think the reason this was posted here because the original title of this thread was "Google released ..."
friendly_chap 9y ago
It seems like slowly but surely the industry is realizing the incredible productivity penalty of microservices/fas etc, and just generally any model compared to your good old ORM in a monolithic codebase approach. I still feel like we are way ahead with the concepts in (shameless plug alert) https://github.com/1backend/1backend https://github.com/1backend/1backend when it comes to the big picture of building microservices that actually build on each other, although it's statistically very likely this is just a personal opinion :)). Anyway, I'm generally happy that this space is advancing, as someone who spent most of his recent career in this space I almost gave up hope: simply there was not even a common vocabulary to properly discuss ideas, everybody just wanted microservices because that was the next cool thing, and looked absolutely perplexed when explained the pitfalls. GraphQL, this, and all the different kind of faas platforms popping up nowadays all help as a point of reference. I truly think we are on the verge of radically changing how we build applications soon - I think these technologies can be viewed as a new breed of web frameworks, a breed which has the distributed principles in its core.
- chuhnk 9y ago
  Remember there's a trade-off to everything and that systems naturally move towards being distributed as they scale. Not just technical systems but all systems in nature. There's always a singular starting point and the move to distributed systems is one where trade-offs are being made to solve specific problems. Taking this approach pragmatically leads to the adoption of microservices and functions at the right time. This library solves a specific problem at the query layer. It's definitely time we had something like this across microservices and functions. There have been other attempts but in unstructured ways. It's likely in the long term systems architectures are multilayered. Monoliths, microservices and functions along with graphql or other query models all within the same environment.
  - friendly_chap 9y ago
    Very nicely put. I agree it's an evolution, but I see a lot of companies waste a lot of time on the monolith -> services transition phase. Usually what happens is that they try a complete rewrite which turns out to be rather expensive (especially in opportunity cost). They should ideally just move the pain points (ie. performance problems) to services and optimize that, but the enthusiasm towards the shiny new way of doing things that will fix their problems is so high that they opt for the riskier move of rewriting all the things. That scenario is simply so dangerous and expensive that companies will be better off starting with a proper distributed framework - which doesn't really exists yet, but we are slowly getting there. Anyway, this is just my 2c but I have seen this pattern of full blown rewrite at a lot of places.
adamkl 9y ago
We were looking at an architecture exactly like this (GraphQL front end, backed by gRPC microservices), because we liked the strongly typed interfaces of gRPC, and the flexibility of GraphQL, but we ran into some technology issues (http/2 support in our PaaS) that made it a non-starter. We found that we could get the same benefits (minus the performance of gRPC) by just using GraphQL for all of our microservices, and dynamically stitching them together with Apollo’s graphql-tools[1]. We’re really high on the approach because it lets us develop independent services, each with a strongly typed interface, and then mix and match them into a single GraphQL interface on the fly. It’s been gaining a lot of traction internally at my company. We even put together a “create-react-app” for GraphQL so our teams can easily spin up new services. [1] https://www.apollographql.com/docs/graphql-tools/schema-stitching.html https://www.apollographql.com/docs/graphql-tools/schema-stit...
- Sirupsen 9y ago
  GraphQL for RPC is an interesting approach. What do you see as the disadvantages? It'd seem to me that not all services can be neatly represented as a 'graph', which gRPC works well for. Have you run into this?
  - the_duke 9y ago
    How you model a GraphQL schema is up to you. You can just treat it as a realitvely flat, rest-like schema that does not have much in terms of graph relations. The Graph in GraphQL is mostly optional.
    - adamkl 9y ago
      Exactly!
  - adamkl 9y ago
    Everybody focuses on the “graph” aspect of GraphQL (obviously), but at its core, GraphQL let’s you define methods with input types and return types in the same fashion as you would with any other RPC framework. If you define your methods and types independently of each other (i.e. without any sort of graph or relationship) I don’t see how it’s any different than SOAP/XML or gRPC. The bonus comes with graph/query capabilities that come layered on top. We’re basically exposing our canonical domain model (broken down into microservices by bounded context, and stitched back together to present a unified model), and at the same time, we should have no problem handling more RPC styled requests (e.g. sendNotificationEmail()) when necessary. This approach was a bit of a revelation for us, but we haven’t really seen anyone else out there doing the same thing, so I hope we aren’t barking up the wrong tree.

108 points by patrickaljord 9y ago

Looks interesting, but TBH I'm still waiting for full swagger spec generation. It should be easy to go from grpc->swagger to generate REST client libs. Unfortunately in practice the swagger generator has many holes in it making it useless in several use cases. So... without being able to generate swagger specs it looks like gRPC is no use for a large number of use cases. Shame really since it has promise but still seems half-baked.

bpicolo 9y ago
That's pretty much exactly what tools like https://github.com/grpc-ecosystem/grpc-gateway https://github.com/grpc-ecosystem/grpc-gateway are doing
- brango 9y ago
  I know. It doesn't work well enough. If Google focussed on this I think gRPC would gain massive ground. But issues are left open for months without any response, etc. We've had to drop gRPC as a direct result of this.
  - jsjohnst 9y ago
    > But issues are left open for months without any response, etc. Code is open source, fork and then submit a PR?
    - brango 9y ago
      No one bothered replying to my issue and suggested fix. I'm not doing work without maintainers agreeing my approach. So no.
      - ithkuil 9y ago
        Care to share a link to the issue / pr?
      - jsjohnst 9y ago
        If you aren’t willing to put in any effort, then I guess the problem is you getting in your own way. Good luck!
        zapita 9y ago
        The person you're replying to clearly is willing to put in effort, but requires a minimum of guidance from the maintainers of the project. That's perfectly reasonable, and your passive-aggressive response is not justified.
bognition 9y ago
> It should be easy to go from grpc->swagger to generate REST client libs. I theory yes, but would you want to? In my experience I've found that building REST services is very different than building a GRPC service. The contract between the client and server are totally different. > gRPC is no use for a large number of use cases. Shame really since it has promise but still seems half-baked. gRPC is half-baked? You realize that the a large portion of the google backend is built on top of gRPC. Its an extremely well tested tool.
- brango 9y ago
  I'd want to because grpc doesn't support javascript on the web. REST is the defacto standard. I am aware Google use gRPC in their SDKs and backends, none of which are JS web. So yes, I want to build microservices with gRPC but expose them with REST for legacy/external clients.
  - pjmlp 9y ago
    I imagine that could be done via native JS arrays or a WebAssembly module, just not sure if it would be an improvement versus REST(JSONP) from performance point of view.
  - anameaname 9y ago
    The problem with web is that browsers don't expose trailers. gRPC depends on having trailers to know if the RPC succeeded or failed, but neither Chrome nor Firefox, nor any other browsers surface this information. If this was exposed in the XHR or Fetch API, gRPC would work. It's browsers in this case that are the laggard (trailers have been part of the spec since HTTP/1.1).
    - muxator 9y ago
      Sorry, what is a trailer?
- tejasmanohar 9y ago
  How much of the Google backend is actually built on top of gRPC? My (outsider's) understanding is gRPC was the "open-source version/rewrite" (typical Google) of Google's internal software, stubby. IIRC, _some_ of the Google Cloud public-facing APIs use it, but what else? As a proponent of gRPC, I can say that some of the libraries, tools, docs, etc. are definitely half-baked. There are plenty of severe issues in the (confusing codebase of) grpc-go that have been open for over 1.5y.

KenanSulayman 9y ago

> Note: This is not an official Google product.

eugenhotaj 9y ago
Yea not directly supported by Google, but being built by Google engineer(s).
- thangngoc89 9y ago
  This is from the README of the project. I think the reason this was posted here because the original title of this thread was "Google released ..."

friendly_chap 9y ago

It seems like slowly but surely the industry is realizing the incredible productivity penalty of microservices/fas etc, and just generally any model compared to your good old ORM in a monolithic codebase approach. I still feel like we are way ahead with the concepts in (shameless plug alert) https://github.com/1backend/1backend https://github.com/1backend/1backend when it comes to the big picture of building microservices that actually build on each other, although it's statistically very likely this is just a personal opinion :)). Anyway, I'm generally happy that this space is advancing, as someone who spent most of his recent career in this space I almost gave up hope: simply there was not even a common vocabulary to properly discuss ideas, everybody just wanted microservices because that was the next cool thing, and looked absolutely perplexed when explained the pitfalls. GraphQL, this, and all the different kind of faas platforms popping up nowadays all help as a point of reference. I truly think we are on the verge of radically changing how we build applications soon - I think these technologies can be viewed as a new breed of web frameworks, a breed which has the distributed principles in its core.

chuhnk 9y ago
Remember there's a trade-off to everything and that systems naturally move towards being distributed as they scale. Not just technical systems but all systems in nature. There's always a singular starting point and the move to distributed systems is one where trade-offs are being made to solve specific problems. Taking this approach pragmatically leads to the adoption of microservices and functions at the right time. This library solves a specific problem at the query layer. It's definitely time we had something like this across microservices and functions. There have been other attempts but in unstructured ways. It's likely in the long term systems architectures are multilayered. Monoliths, microservices and functions along with graphql or other query models all within the same environment.
- friendly_chap 9y ago
  Very nicely put. I agree it's an evolution, but I see a lot of companies waste a lot of time on the monolith -> services transition phase. Usually what happens is that they try a complete rewrite which turns out to be rather expensive (especially in opportunity cost). They should ideally just move the pain points (ie. performance problems) to services and optimize that, but the enthusiasm towards the shiny new way of doing things that will fix their problems is so high that they opt for the riskier move of rewriting all the things. That scenario is simply so dangerous and expensive that companies will be better off starting with a proper distributed framework - which doesn't really exists yet, but we are slowly getting there. Anyway, this is just my 2c but I have seen this pattern of full blown rewrite at a lot of places.

adamkl 9y ago

We were looking at an architecture exactly like this (GraphQL front end, backed by gRPC microservices), because we liked the strongly typed interfaces of gRPC, and the flexibility of GraphQL, but we ran into some technology issues (http/2 support in our PaaS) that made it a non-starter. We found that we could get the same benefits (minus the performance of gRPC) by just using GraphQL for all of our microservices, and dynamically stitching them together with Apollo’s graphql-tools[1]. We’re really high on the approach because it lets us develop independent services, each with a strongly typed interface, and then mix and match them into a single GraphQL interface on the fly. It’s been gaining a lot of traction internally at my company. We even put together a “create-react-app” for GraphQL so our teams can easily spin up new services. [1] https://www.apollographql.com/docs/graphql-tools/schema-stitching.html https://www.apollographql.com/docs/graphql-tools/schema-stit...

Sirupsen 9y ago
GraphQL for RPC is an interesting approach. What do you see as the disadvantages? It'd seem to me that not all services can be neatly represented as a 'graph', which gRPC works well for. Have you run into this?
- the_duke 9y ago
  How you model a GraphQL schema is up to you. You can just treat it as a realitvely flat, rest-like schema that does not have much in terms of graph relations. The Graph in GraphQL is mostly optional.
  - adamkl 9y ago
    Exactly!
- adamkl 9y ago
  Everybody focuses on the “graph” aspect of GraphQL (obviously), but at its core, GraphQL let’s you define methods with input types and return types in the same fashion as you would with any other RPC framework. If you define your methods and types independently of each other (i.e. without any sort of graph or relationship) I don’t see how it’s any different than SOAP/XML or gRPC. The bonus comes with graph/query capabilities that come layered on top. We’re basically exposing our canonical domain model (broken down into microservices by bounded context, and stitched back together to present a unified model), and at the same time, we should have no problem handling more RPC styled requests (e.g. sendNotificationEmail()) when necessary. This approach was a bit of a revelation for us, but we haven’t really seen anyone else out there doing the same thing, so I hope we aren’t barking up the wrong tree.