4 ms·

there is something nice about being able to just write some json and curl an endpoint. getting to this point with grpc is effort.

by dastbe 4y ago

jiggawatts 4y ago
This to me always feels like someone saying it's nice to be able to just pour crude into a couple of buckets and transport them around in the back of a truck when the goal is to build an oil refinery. This kind of "make the trivial part easy", "get started quick", "need no tooling" approach feels fast... at first, but then very quickly runs into nigh insurmountable problems. A great analogy I heard was: "If your plan is to go to the moon, you won't get there by making a fast bus."
- harrisonjackson 4y ago
  Most crud apps are not the equivalent of an oil refinery.
  - taeric 4y ago
    At the same time, most crud apps need auth and a whole lot of other considerations. Is it neat they I can copy as curl a request to replay? Yeah. But, that isn't too hard to do with most any framework, is it?
    - dastbe 4y ago
      do it with grpc and regular curl.
- nsteel 4y ago
  I've only limited experience with it but it does the job OK. Can you expand on "nigh insurmountable problems"?
  - jiggawatts 4y ago
    An RPC interface is the same as any other interface in programming, but with more complexity to deal with. Hence, just as we have the "proper tooling" to deal with internal interfaces, there's a stack of tooling for dealing with RPC interfaces. Advice that I provide to dev teams is that every time you introduce a network hop, you're more-or-less going to triple the complexity of that interface boundary. Before, where you might have had a simple function call such as "foo(p1,p2);", you now have to: 1) Define a message type that encapsulates the p1 & p2 arguments into a single thing. 2) Do this on both server and client, consistently, and allowing for versioning to handle rolling upgrades. 3) Encode and decode your actual programming types into this blob. (Across dissimilar languages this is especially fun.) 4) Write the "RPC server" code. 5) Write the "RPC client" code. That last one is the bit that a lot of developers skip, or make it "other people's problem". Imagine you're writing a server for an RPC endpoint. You (internally) define some return message type, call "toJson()" or whatever on it, return it from some HTTP REST route mapping... and you're done. You're done. You. Queue the potentially dozens, hundreds, or possibly hundreds of thousands of developers that have to bang out the client side of this. By hand. Typing it in, based on your documentation... of which there is likely none. Or it's in HTML and not machine readable. They're going to make mistakes. They will assume something is nullable when it isn't, or not-nullable when it is. They'll have no idea what error messages can be expected, or in what format, and the only way they'll find out is... in production. Rare errors they'll likely never handle properly. Meanwhile, with "proper tooling" the server-side developer uses a formal Interface Definition Language (IDL) such as the one used by CORBA, DCOM+, gRPC, Cap'n Proto, or whatever. This IDL is ingested by tools(!) on both the server and client, generating reams of "stub" code, ready for business logic. Better tools will automatically insert the matching doc-comments, so that when typing in a proper IDE (which you use, right?), then hovering over a function definition will show its help summary text live, in context. In enterprise settings, it's common to see code that's basically just a dozen APIs glued together to do something useful. If those APIs are hand-rolled REST, then this takes months of development effort, and then endless maintenance as things just break in weird and unexpected ways... in production. If those APIs are generated with tooling, then hundreds of kilobytes of that boilerplate RPC client code can be spat out in just minutes of effort. More importantly, nobody then ever needs curl to diagnose random errors in production, because the code won't even compile if something doesn't match the schema. If something returns an unexpected value, nobody needs to inspect the wire traffic, because the deserialized value is in memory, visible to a debugger or log trace. Getting "into the weeds" of banging out RPC requests by hand is for people that don't realise that they're not supposed to care what the wire format is! It really is night & day. For example, back in 2006 with Windows Communication Foundation, you could just paste a single URL into Visual Studio, and then you'd be off to the races. It would generate everything for you, and you could just start programming your business logic immediately.
- dastbe 4y ago
  this seems hyperbolic? the overwhelming majority of aws apis are json over the wire (with some variation in how the operation is defined. there is no perceivable downside to this practice, and they’ve built quite the “oil refinery”. being able to easily introspect requests and define your own over the wire was immensely helpful, and it’s painful that i have to reach for a special tool like grpcurl.
jayd16 4y ago
Use gRPCurl.
- dastbe 4y ago
  so another thing i have to install. and i also need to set up the introspection endpoint on my service to make sure it works well.
  - pjmlp 4y ago
    Postman does both.
boulos 4y ago
Disclosure: I used to work on Google Cloud. If you want, there are plenty of JSON proxies including Cloud Endpoints [1]. I also do most things with manual curl'ing, and relied on the Google APIs to have these kinds of "Fine, we'll also take JSON" setups in place. Once you've figured it out, you build the proto in your automation directly. But I basically never use grpcurl either. [1] https://cloud.google.com/endpoints/docs/grpc/transcoding https://cloud.google.com/endpoints/docs/grpc/transcoding

there is something nice about being able to just write some json and curl an endpoint. getting to this point with grpc is effort.

by dastbe 4y ago

there is something nice about being able to just write some json and curl an endpoint. getting to this point with grpc is effort.

jiggawatts 4y ago

This to me always feels like someone saying it's nice to be able to just pour crude into a couple of buckets and transport them around in the back of a truck when the goal is to build an oil refinery. This kind of "make the trivial part easy", "get started quick", "need no tooling" approach feels fast... at first, but then very quickly runs into nigh insurmountable problems. A great analogy I heard was: "If your plan is to go to the moon, you won't get there by making a fast bus."

harrisonjackson 4y ago
Most crud apps are not the equivalent of an oil refinery.
- taeric 4y ago
  At the same time, most crud apps need auth and a whole lot of other considerations. Is it neat they I can copy as curl a request to replay? Yeah. But, that isn't too hard to do with most any framework, is it?
  - dastbe 4y ago
    do it with grpc and regular curl.
nsteel 4y ago
I've only limited experience with it but it does the job OK. Can you expand on "nigh insurmountable problems"?
- jiggawatts 4y ago
  An RPC interface is the same as any other interface in programming, but with more complexity to deal with. Hence, just as we have the "proper tooling" to deal with internal interfaces, there's a stack of tooling for dealing with RPC interfaces. Advice that I provide to dev teams is that every time you introduce a network hop, you're more-or-less going to triple the complexity of that interface boundary. Before, where you might have had a simple function call such as "foo(p1,p2);", you now have to: 1) Define a message type that encapsulates the p1 & p2 arguments into a single thing. 2) Do this on both server and client, consistently, and allowing for versioning to handle rolling upgrades. 3) Encode and decode your actual programming types into this blob. (Across dissimilar languages this is especially fun.) 4) Write the "RPC server" code. 5) Write the "RPC client" code. That last one is the bit that a lot of developers skip, or make it "other people's problem". Imagine you're writing a server for an RPC endpoint. You (internally) define some return message type, call "toJson()" or whatever on it, return it from some HTTP REST route mapping... and you're done. You're done. You. Queue the potentially dozens, hundreds, or possibly hundreds of thousands of developers that have to bang out the client side of this. By hand. Typing it in, based on your documentation... of which there is likely none. Or it's in HTML and not machine readable. They're going to make mistakes. They will assume something is nullable when it isn't, or not-nullable when it is. They'll have no idea what error messages can be expected, or in what format, and the only way they'll find out is... in production. Rare errors they'll likely never handle properly. Meanwhile, with "proper tooling" the server-side developer uses a formal Interface Definition Language (IDL) such as the one used by CORBA, DCOM+, gRPC, Cap'n Proto, or whatever. This IDL is ingested by tools(!) on both the server and client, generating reams of "stub" code, ready for business logic. Better tools will automatically insert the matching doc-comments, so that when typing in a proper IDE (which you use, right?), then hovering over a function definition will show its help summary text live, in context. In enterprise settings, it's common to see code that's basically just a dozen APIs glued together to do something useful. If those APIs are hand-rolled REST, then this takes months of development effort, and then endless maintenance as things just break in weird and unexpected ways... in production. If those APIs are generated with tooling, then hundreds of kilobytes of that boilerplate RPC client code can be spat out in just minutes of effort. More importantly, nobody then ever needs curl to diagnose random errors in production, because the code won't even compile if something doesn't match the schema. If something returns an unexpected value, nobody needs to inspect the wire traffic, because the deserialized value is in memory, visible to a debugger or log trace. Getting "into the weeds" of banging out RPC requests by hand is for people that don't realise that they're not supposed to care what the wire format is! It really is night & day. For example, back in 2006 with Windows Communication Foundation, you could just paste a single URL into Visual Studio, and then you'd be off to the races. It would generate everything for you, and you could just start programming your business logic immediately.
dastbe 4y ago
this seems hyperbolic? the overwhelming majority of aws apis are json over the wire (with some variation in how the operation is defined. there is no perceivable downside to this practice, and they’ve built quite the “oil refinery”. being able to easily introspect requests and define your own over the wire was immensely helpful, and it’s painful that i have to reach for a special tool like grpcurl.

jayd16 4y ago

Use gRPCurl.

dastbe 4y ago
so another thing i have to install. and i also need to set up the introspection endpoint on my service to make sure it works well.
- pjmlp 4y ago
  Postman does both.

boulos 4y ago

Disclosure: I used to work on Google Cloud. If you want, there are plenty of JSON proxies including Cloud Endpoints [1]. I also do most things with manual curl'ing, and relied on the Google APIs to have these kinds of "Fine, we'll also take JSON" setups in place. Once you've figured it out, you build the proto in your automation directly. But I basically never use grpcurl either. [1] https://cloud.google.com/endpoints/docs/grpc/transcoding https://cloud.google.com/endpoints/docs/grpc/transcoding