4 ms·

I've been building API's for a long time, using gRPC, and HTTP/REST (we'll not go into CORBA or DCOM, because I'll cry). To that end, I've open sourced a Go lib

I've been building API's for a long time, using gRPC, and HTTP/REST (we'll not go into CORBA or DCOM, because I'll cry). To that end, I've open sourced a Go library for generating your clients and servers from OpenAPI specs (https://github.com/oapi-codegen/oapi-codegen https://github.com/oapi-codegen/oapi-codegen). I disagree with the way this article breaks down the options. There is no difference between OpenAPI and REST, it's a strange distinction. OpenAPI is a way of documenting the behavior of your HTTP API. You can express a RESTful API using OpenAPI, or something completely random, it's up to you. The purpose of OpenAPI is to have a schema language to describe your API for tooling to interpret, so in concept, it's similar to Protocol Buffer files that are used to specify gRPC protocols. gRPC is an RPC mechanism for sending protos back and forth. When Google open sourced protobufs, they didn't opensource the RPC layer, called "stubby" at Google, which made protos really great. gRPC is not stubby, and it's not as awesome, but it's still very efficient at transport, and fairly easy too extend and hook into. The problem is, it's a self-contained ecosystem that isn't as robust as mainstream HTTP libraries, which give you all kinds of useful middleware like logging or auth. You'll be implementing lots of these yourself with gRPC, particularly if you are making RPC calls across services implemented in different languages. To me, the problem with gRPC is proto files. Every client must be built against .proto files compatible with the server; it's not a discoverable protocol. With an HTTP API, you can make calls to it via curl or your own code without having the OpenAPI description, so it's a "softer" binding. This fact alone makes it easier to work with and debug.

physicles 2y ago
> There is no difference between OpenAPI and REST, it's a strange distinction. That threw me off too. What the article calls REST, I understand to be closer to HATEOAS. > I've open sourced a Go library for generating your clients and servers from OpenAPI specs As a maintainer of a couple pretty substantial APIs with internal and external clients, I'm really struggling to understand the workflow that starts with generating code from OpenAPI specs. Once you've filled in all those generated stubs, how can you then iterate on the API spec? The tooling will just give you more stubs that you have to manually merge in, and it'll get harder and harder to find the relevant updates as the API grows. This is why I created an abomination that uses go/ast and friends to generate the OpenAPI spec from the code. It's not perfect, but it's a 95% solution that works with both Echo and Gin. So when we need to stand up a new endpoint and allow the front end to start coding against it ASAP, the workflow looks like this: 1. In a feature branch, define the request and response structs, and write an empty handler that parses parameters and returns an empty response. 2. Generate the docs and send them to the front end dev. Now, most devs never have to think about how to express their API in OpenAPI. And the docs will always be perfectly in sync with the code.
- plorkyeran 2y ago
  HATEOAS is just REST as originally envisioned but accepting that the REST name has come to be attached to something different.
- mkleczek 2y ago
  > This is why I created an abomination that uses go/ast and friends to generate the OpenAPI spec from the code. This is against "interface first" principle and couples clients of your API to its implementation. That might be OK if the only consumer of the API is your own application as in that case API is really just an internal implementation detail. But even then - once you have to support multiple versions of your own client it becomes difficult not to break them.
  - physicles 2y ago
    I don't see why it couples clients to the implementation. Effectively, there's no difference between writing the code first and updating the OpenAPI spec, and updating the spec first and then doing some sort of code gen to update the implementation. The end state of the world is the same. In either case, modifications to the spec will be scrutinized to make sure there are no breaking changes.
    - Cthulhu_ 2y ago
      Yeah this is the way, I mean if the spec already exists it makes sense to go spec-first. I went spec-first last time I built an API because I find most generators to be imperfect or lacking features; going spec-first ensured that the spec was correct at least, and the implementations could do the workarounds (e.g. type conversions in Go) where necessary. That is, generate spec from code and your spec is limited to what can be expressed by the code, its annotations, and the support that the generator has. Most generators (to or from openapi) are imperfect and have to compromise on some features, which can lead to miscommunication between clients/servers.
  - jitl 2y ago
    OpenAPI spec being authored by a human or a machine, it can still be the same YAML at the end of the day, so why would one approach be more brittle / breaks your clients than the other?
mandevil 2y ago
There is a distinction between (proper) REST and what this blog calls "OpenAPI". But the thing is, almost no one builds a true, proper REST API. In practice, everyone uses the OpenAPI approach. The way that REST was defined by Roy Fielding in his 2000 Ph.D dissertation ("Architectural Styles and the Design of Network-based Software Architectures") it was supposed to allow a web-like exploring of all available resources. You would GET the root URL, and the 200 OK Response would provide a set of links that would allow you to traverse all available resources provided by the API (it was allowed to be hierarchical- but everything had to be accessible somewhere in the link tree). This was supposed to allow discoverability. In practice, everywhere I've ever worked over the past two decades has just used POST resource_name/resource_id/sub_resource/sub_resource_id/mutatation_type- or PUT resource_name/resource_id/sub_resource/sub_resource_id depending on how that company handled the idempotency issues that PUT creates- with all of those being magic URL's assembled by the client with knowledge of the structure (often defined in something like Swagger/OpenAPI), lacking the link-traversal from root that was a hallmark of Fielding's original work. Pedants (which let's face it, most of us are) will often describe what is done in practice as "RESTful" rather than "REST" just to acknowledge that they are not implementing Fielding's definition of REST.
- bborud 2y ago
  I tend to prefer RESTish rather than RESTful since RESTful almost suggests attempting to implement Fielding's ideas but not quite getting there. I think the subset of approaches that try and fail to implement Fielding's ideas is an order of magnitude (or two) smaller than those who go for something that is superficially similar, but has nothing to do with HATEOAS :-). REST is an interesting idea, but I don't think it is a practical idea. It is too hard to design tools and libraries that helps/encourages/forces the user implement HATEOAS sensibly, easily and consistently.
  - mandevil 2y ago
    While it is amazing for initial discovery to have everything presented for the developer's inspection, in production it ends up requiring too many network round-trips to actually traverse from root to /resource_name/resource_id/sub_resource_name/sub_resource_id, or an already verbose transaction (everything is serialized and deserialized into strings!) becomes gigantic if you if don't make it hierarchical and just drop every URL into the root response. This is why everyone just builds magic URL endpoints, and hopefully also includes a OpenAPI/Swagger documentation for them so the developer can figure it out. And then keeps the documentation up-to-date as they add new sub_resource endpoints!
TheGoodBarn 2y ago
Just chiming in to say we use oapi-codegen everyday and it’s phenomenal. Migrated away from Swaggo -> oapi during a large migration to be interface first for separating out large vertical slices and it’s been a godsend.
happyweasel 2y ago
Buggy/incomplete Openapi codegen for rust was a huge disappointment for me. At least with grpc some languages are first class citizens. Of course generated code has some uglyness. Kinda sad http2 traffic can be flaky due to bugs in network hardware.
arccy 2y ago
These days there's gprc reflection for discovery https://grpc.io/docs/guides/reflection/ https://grpc.io/docs/guides/reflection/
cpursley 2y ago
I'm piggybacking on the OpenAPI spec as well to generate a SQL-like query syntax along with generated types which makes working with any 3rd party API feel the same. What if you could query any ole' API like this?: Apipe.new(GitHun) |> from("search/repositories") |> eq(:language, "elixir") |> order_by(:updated) |> limit(1) |> execute() This way, you don't have to know about all the available gRPC functions or the 3rd party API's RESTful quirks while retaining built-in documenting and having access to types. https://github.com/cpursley/apipe https://github.com/cpursley/apipe I'm considering building a TS adapter layer so that you can just drop this into your JS/TS project like you would with Supabase: const { data, error } = await apipe.from('search/repositories').eq('language', 'elixir').order_by('updated').limit(1) Where this would run through the Elixir proxy which would do the heavy lifting like async, handle rate limits, etc.
Pooge 2y ago
> https://github.com/oapi-codegen/oapi-codegen https://github.com/oapi-codegen/oapi-codegen I'm using it for a small personal project! Works very well. Thank you!
deleted 2y ago
[deleted]
cyberax 2y ago
> To me, the problem with gRPC is proto files. Every client must be built against .proto files compatible with the server; it's not a discoverable protocol. That's not quite true. You can build an OpenAPI description based on JSON serialization of Protobufs and serve it via Swagger. The gRPC itself also offers built-in reflection (and a nice grpcurl utility that uses it!).

physicles 2y ago

> There is no difference between OpenAPI and REST, it's a strange distinction. That threw me off too. What the article calls REST, I understand to be closer to HATEOAS. > I've open sourced a Go library for generating your clients and servers from OpenAPI specs As a maintainer of a couple pretty substantial APIs with internal and external clients, I'm really struggling to understand the workflow that starts with generating code from OpenAPI specs. Once you've filled in all those generated stubs, how can you then iterate on the API spec? The tooling will just give you more stubs that you have to manually merge in, and it'll get harder and harder to find the relevant updates as the API grows. This is why I created an abomination that uses go/ast and friends to generate the OpenAPI spec from the code. It's not perfect, but it's a 95% solution that works with both Echo and Gin. So when we need to stand up a new endpoint and allow the front end to start coding against it ASAP, the workflow looks like this: 1. In a feature branch, define the request and response structs, and write an empty handler that parses parameters and returns an empty response. 2. Generate the docs and send them to the front end dev. Now, most devs never have to think about how to express their API in OpenAPI. And the docs will always be perfectly in sync with the code.

plorkyeran 2y ago
HATEOAS is just REST as originally envisioned but accepting that the REST name has come to be attached to something different.
mkleczek 2y ago
> This is why I created an abomination that uses go/ast and friends to generate the OpenAPI spec from the code. This is against "interface first" principle and couples clients of your API to its implementation. That might be OK if the only consumer of the API is your own application as in that case API is really just an internal implementation detail. But even then - once you have to support multiple versions of your own client it becomes difficult not to break them.
- physicles 2y ago
  I don't see why it couples clients to the implementation. Effectively, there's no difference between writing the code first and updating the OpenAPI spec, and updating the spec first and then doing some sort of code gen to update the implementation. The end state of the world is the same. In either case, modifications to the spec will be scrutinized to make sure there are no breaking changes.
  - Cthulhu_ 2y ago
    Yeah this is the way, I mean if the spec already exists it makes sense to go spec-first. I went spec-first last time I built an API because I find most generators to be imperfect or lacking features; going spec-first ensured that the spec was correct at least, and the implementations could do the workarounds (e.g. type conversions in Go) where necessary. That is, generate spec from code and your spec is limited to what can be expressed by the code, its annotations, and the support that the generator has. Most generators (to or from openapi) are imperfect and have to compromise on some features, which can lead to miscommunication between clients/servers.
- jitl 2y ago
  OpenAPI spec being authored by a human or a machine, it can still be the same YAML at the end of the day, so why would one approach be more brittle / breaks your clients than the other?

mandevil 2y ago

There is a distinction between (proper) REST and what this blog calls "OpenAPI". But the thing is, almost no one builds a true, proper REST API. In practice, everyone uses the OpenAPI approach. The way that REST was defined by Roy Fielding in his 2000 Ph.D dissertation ("Architectural Styles and the Design of Network-based Software Architectures") it was supposed to allow a web-like exploring of all available resources. You would GET the root URL, and the 200 OK Response would provide a set of links that would allow you to traverse all available resources provided by the API (it was allowed to be hierarchical- but everything had to be accessible somewhere in the link tree). This was supposed to allow discoverability. In practice, everywhere I've ever worked over the past two decades has just used POST resource_name/resource_id/sub_resource/sub_resource_id/mutatation_type- or PUT resource_name/resource_id/sub_resource/sub_resource_id depending on how that company handled the idempotency issues that PUT creates- with all of those being magic URL's assembled by the client with knowledge of the structure (often defined in something like Swagger/OpenAPI), lacking the link-traversal from root that was a hallmark of Fielding's original work. Pedants (which let's face it, most of us are) will often describe what is done in practice as "RESTful" rather than "REST" just to acknowledge that they are not implementing Fielding's definition of REST.

bborud 2y ago
I tend to prefer RESTish rather than RESTful since RESTful almost suggests attempting to implement Fielding's ideas but not quite getting there. I think the subset of approaches that try and fail to implement Fielding's ideas is an order of magnitude (or two) smaller than those who go for something that is superficially similar, but has nothing to do with HATEOAS :-). REST is an interesting idea, but I don't think it is a practical idea. It is too hard to design tools and libraries that helps/encourages/forces the user implement HATEOAS sensibly, easily and consistently.
- mandevil 2y ago
  While it is amazing for initial discovery to have everything presented for the developer's inspection, in production it ends up requiring too many network round-trips to actually traverse from root to /resource_name/resource_id/sub_resource_name/sub_resource_id, or an already verbose transaction (everything is serialized and deserialized into strings!) becomes gigantic if you if don't make it hierarchical and just drop every URL into the root response. This is why everyone just builds magic URL endpoints, and hopefully also includes a OpenAPI/Swagger documentation for them so the developer can figure it out. And then keeps the documentation up-to-date as they add new sub_resource endpoints!

TheGoodBarn 2y ago

Just chiming in to say we use oapi-codegen everyday and it’s phenomenal. Migrated away from Swaggo -> oapi during a large migration to be interface first for separating out large vertical slices and it’s been a godsend.

happyweasel 2y ago

Buggy/incomplete Openapi codegen for rust was a huge disappointment for me. At least with grpc some languages are first class citizens. Of course generated code has some uglyness. Kinda sad http2 traffic can be flaky due to bugs in network hardware.

arccy 2y ago

These days there's gprc reflection for discovery https://grpc.io/docs/guides/reflection/ https://grpc.io/docs/guides/reflection/

cpursley 2y ago

I'm piggybacking on the OpenAPI spec as well to generate a SQL-like query syntax along with generated types which makes working with any 3rd party API feel the same. What if you could query any ole' API like this?: Apipe.new(GitHun) |> from("search/repositories") |> eq(:language, "elixir") |> order_by(:updated) |> limit(1) |> execute() This way, you don't have to know about all the available gRPC functions or the 3rd party API's RESTful quirks while retaining built-in documenting and having access to types. https://github.com/cpursley/apipe https://github.com/cpursley/apipe I'm considering building a TS adapter layer so that you can just drop this into your JS/TS project like you would with Supabase: const { data, error } = await apipe.from('search/repositories').eq('language', 'elixir').order_by('updated').limit(1) Where this would run through the Elixir proxy which would do the heavy lifting like async, handle rate limits, etc.

Pooge 2y ago

> https://github.com/oapi-codegen/oapi-codegen https://github.com/oapi-codegen/oapi-codegen I'm using it for a small personal project! Works very well. Thank you!

deleted 2y ago

[deleted]

cyberax 2y ago

> To me, the problem with gRPC is proto files. Every client must be built against .proto files compatible with the server; it's not a discoverable protocol. That's not quite true. You can build an OpenAPI description based on JSON serialization of Protobufs and serve it via Swagger. The gRPC itself also offers built-in reflection (and a nice grpcurl utility that uses it!).