3 ms·

I have experience working on some JSON RPC Load Balancer, and I can tell that it's one of the worst choices for an API: - Method Name is a part of the body its

by splix 4y ago

I have experience working on some JSON RPC Load Balancer, and I can tell that it's one of the worst choices for an API: - Method Name is a part of the body itself. So you have to parse it to make a decision on how to dispatch it. That's an extra cost. - Error Code is a part of the response. So you have to parse it each time to figure out if it's a success. Also, some clients just give an HTTP Error Code instead. So you have to handle both. - It can also be batched. Which introduces a lot of unspecified use cases. - Like how are you supposed to dispatch those batched? Should they go to the same upstream? Or can it be parallelized? Should the order be preserved? - With batches, the slowest request always blocks the in-flight response. You have to wait for all calls to be finished before producing a final response. That's a huge performance bottleneck. Also, what you do if some of them never finished? - A lot of uncertainties about the ID. Especially because it can be a different type (int or string), which sometimes introduces two different handlers in the code. And there are no guarantees IDs are not repeating in the same batch. To make it worse, some clients rely only on the request/response order ignoring the IDs. - Another problem is that it has no idea of Metadata, which is important for many real systems. For example, to do an Auth. Or do Caching and other optimizations. So as an outcome, everyone just makes two layers. On the HTTP level, you have auth, routing, error handling, multiplexing, etc. And then you realize that JSON RPC only introduces extra complexity.

jillesvangurp 4y ago
Graphql shares many of the same issues. And yet it is popular. And arguably a lot more convoluted. The difference is basically schemas and client generation. The whole point of RPC is that the stuff that happens on the network is just a means to an end and you generate the code that does the server and client stubs that you call. I've seen just about anything on this front. People used to do SOAP which is all of the downsides of graphql and json rpc combined without most of the benefits. Stupidly complicated to do just about anything with it. It's a good reason people don't suggest that anymore for new APIs. REST APIs completely killed SOAP. The only places you see it these days are legacy systems from 15-20 years ago. The thing with just about any kind of parsing overhead on servers is that it is completely dwarfed by network IO getting to the server and then more IO for getting to a database, redis, etc. Add ORM to the mix and you typically get this unoptimized mix of multiple calls to a database (because most orms suck at optimizing their joins). We're talking fractions of a percent of the time spent here. Web servers don't commonly use a lot of CPU time until they are handling dozens/hundreds of requests per second. With a asynchronous IO, you can server thousands of requests. The bottleneck usually becomes your database. Not the webservers. Those tend to run on cheap vms. Add more to scale at the cost of 10-20$ per month. Back in the day when parsing overhead still mattered, we'd speed up our servers with a few simple tricks, which mainly boiled down to not buffering requests and responses. Use streaming parsers on the request payload, iteratively process whatever comes in, and stream the output as results become available. You'd be responding with the first bytes long before the last request byte had arrived. Great for processing large ndjson/csv files and responding with similar output. You can process mang GB of data with a single request this way. Doing this is hard but not impossible with modern asynchronous web framweworks. They all default to buffering everything and then processing and then responding. Reason: it's simpler and the performance difference simply does not matter. But it's still a useful trick if time to the first bytes of your response matters to you and you have a really fast, tightly tuned backend. This is how you get to sub ms responses.
- linza 4y ago
  Not sure if you are more agreeing or disagreeing with GP, but cost is not just computing cost, but also complexity and the higher risk to reliability that comes with it. JSON-RPC makes sense if one only considers the abstract case with a single client and a single server box and minimal networking. Technically it's a viable solution but just not practical. HTTP on the other hand is nice as it separates the business logic stuff like the RPC parameters from stuff that's interesting to the transport-network-thingies that sit between you and the data on a server, like load balancers, DoS protection, authz, rate limiters, ... etc.
  - jillesvangurp 4y ago
    It's a trade-off. Mostly pure REST is nicer for external APIs where it's just more important to follow the principle of the least amount of surprise for the ed user. For better or worse, people kind of know how to deal with REST clients. Graphql, SOAP, etc. all force the user into doing complicated stuff with libraries, figuring out parsing for their techstack, etc. This is fine for internal microservices where you might not care too much about this and can standardize things a bit. But for external users it's more challenging. With simple Rpc stacks (like JsonRpc), basically the up side is you get to rely on code generation to deal with networking. Cutting down on the amount of ceremony for adding new stuff is definitely nice in a fast moving team. But I would not use it for external APIs. It's kind of a legacy way of doing thing. An alternative to something called XmlRpc, which used to be a light weight alternative to SOAP when people were still doing XML and when MS came up with the genius idea to stuff something called an XMLHttpRequest in Internet Explorer. This was what enabled Ajax websites early on. The X in ajax of course stands for XML. That was just before json got really popular and people started getting really pedantic about their REST verbs. Anyway, short history lesson. I'd not over think this and indeed simply use REST for public/external APIs and use whatever works for you internally for e.g. client server traffic between your browser/mobile apps and your servers or intra microservice communication. If it's internal, you control the techstack and you can sacrifice some things for making it faster/easier to develop and use without too much regard for what the other side is going to be using. Because you control both sides. REST APIs can be a bit of a burden to maintain. Lots of boiler plate client and server side. Not always the best choice internally.

by splix 4y ago

jillesvangurp 4y ago

Graphql shares many of the same issues. And yet it is popular. And arguably a lot more convoluted. The difference is basically schemas and client generation. The whole point of RPC is that the stuff that happens on the network is just a means to an end and you generate the code that does the server and client stubs that you call. I've seen just about anything on this front. People used to do SOAP which is all of the downsides of graphql and json rpc combined without most of the benefits. Stupidly complicated to do just about anything with it. It's a good reason people don't suggest that anymore for new APIs. REST APIs completely killed SOAP. The only places you see it these days are legacy systems from 15-20 years ago. The thing with just about any kind of parsing overhead on servers is that it is completely dwarfed by network IO getting to the server and then more IO for getting to a database, redis, etc. Add ORM to the mix and you typically get this unoptimized mix of multiple calls to a database (because most orms suck at optimizing their joins). We're talking fractions of a percent of the time spent here. Web servers don't commonly use a lot of CPU time until they are handling dozens/hundreds of requests per second. With a asynchronous IO, you can server thousands of requests. The bottleneck usually becomes your database. Not the webservers. Those tend to run on cheap vms. Add more to scale at the cost of 10-20$ per month. Back in the day when parsing overhead still mattered, we'd speed up our servers with a few simple tricks, which mainly boiled down to not buffering requests and responses. Use streaming parsers on the request payload, iteratively process whatever comes in, and stream the output as results become available. You'd be responding with the first bytes long before the last request byte had arrived. Great for processing large ndjson/csv files and responding with similar output. You can process mang GB of data with a single request this way. Doing this is hard but not impossible with modern asynchronous web framweworks. They all default to buffering everything and then processing and then responding. Reason: it's simpler and the performance difference simply does not matter. But it's still a useful trick if time to the first bytes of your response matters to you and you have a really fast, tightly tuned backend. This is how you get to sub ms responses.

linza 4y ago
Not sure if you are more agreeing or disagreeing with GP, but cost is not just computing cost, but also complexity and the higher risk to reliability that comes with it. JSON-RPC makes sense if one only considers the abstract case with a single client and a single server box and minimal networking. Technically it's a viable solution but just not practical. HTTP on the other hand is nice as it separates the business logic stuff like the RPC parameters from stuff that's interesting to the transport-network-thingies that sit between you and the data on a server, like load balancers, DoS protection, authz, rate limiters, ... etc.
- jillesvangurp 4y ago
  It's a trade-off. Mostly pure REST is nicer for external APIs where it's just more important to follow the principle of the least amount of surprise for the ed user. For better or worse, people kind of know how to deal with REST clients. Graphql, SOAP, etc. all force the user into doing complicated stuff with libraries, figuring out parsing for their techstack, etc. This is fine for internal microservices where you might not care too much about this and can standardize things a bit. But for external users it's more challenging. With simple Rpc stacks (like JsonRpc), basically the up side is you get to rely on code generation to deal with networking. Cutting down on the amount of ceremony for adding new stuff is definitely nice in a fast moving team. But I would not use it for external APIs. It's kind of a legacy way of doing thing. An alternative to something called XmlRpc, which used to be a light weight alternative to SOAP when people were still doing XML and when MS came up with the genius idea to stuff something called an XMLHttpRequest in Internet Explorer. This was what enabled Ajax websites early on. The X in ajax of course stands for XML. That was just before json got really popular and people started getting really pedantic about their REST verbs. Anyway, short history lesson. I'd not over think this and indeed simply use REST for public/external APIs and use whatever works for you internally for e.g. client server traffic between your browser/mobile apps and your servers or intra microservice communication. If it's internal, you control the techstack and you can sacrifice some things for making it faster/easier to develop and use without too much regard for what the other side is going to be using. Because you control both sides. REST APIs can be a bit of a burden to maintain. Lots of boiler plate client and server side. Not always the best choice internally.