4 ms·
> and the regular old C implementation of Redis Server? To be clear, this article is talking about implementing your own synchronous-RPC-request server, i.e. a
by derefr 5y ago
> and the regular old C implementation of Redis Server?
To be clear, this article is talking about implementing your own synchronous-RPC-request server, i.e. a network service that other services talk to through an API over some network wire protocol, to make requests over a socket and then wait for responses to those requests over that same socket. This article assumes that you already know that that's what you need. This article then offers an additional alternative to the traditional wire protocols one might expose to clients in a synchronous-RPC-request server (RESTful HTTP, gRPC, JSON-RPC over HTTP, JSON-RPC over TCP, etc.); namely, mimicking the wire protocol Redis uses, but exposing your own custom Redis commands. This choice allows you to use existing Redis client libraries as your RPC clients, just as writing a RESTful HTTP server allows you to use existing HTTP client libraries as your RPC clients.
The alternative to doing so, if you want to call it that, would be to write these custom commands as a Redis module in C. But then you have to structure your code to live inside a Redis server, when that might not be at-all what you want, especially if your code already lives inside some other kind of framework, or is written in a managed-runtime language unsuited to plugging into a C server.
Or think of it like this: this article is about taking an existing daemon, written in some arbitrary language (in this case Elixir), where that daemon already speaks some other, slower RPC protocol (e.g. REST over HTTP); and adding an additional Redis-protocol RPC listener to that daemon, so that you can use a Redis client as a drop-in replacement for an HTTP client for doing RPC against the daemon, thus (presumably) lowering per-request protocol overhead for clients that need to pump through a lot of RPC requests.
I do realize that you're suggesting that you could use Redis as an event-bus between two processes that each connect to it via the Redis protocol; and then use a "fire an async request as an event over the bus, and then await a response event containing the request's ref to show up on the bus" RPC strategy, ala Erlang's own gen_server:call messaging strategy. All I can say is that, due to there being three processes and two separate RPC sessions involved, with their own wire-protocol encoding/decoding phases, that's likely higher-overhead than even a direct RESTful-HTTP RPC session between the client and the relevant daemon; let alone a direct Redis RPC session between the client and the daemon.
- dljsjr 5y agoThat’s fair. Although I think the characterization of doing RPC over Redis’s built-in pubsub is a little uncharitable. You’d just fire off a PUBLISH command w/ the channel and payload, and you’d receive the response to the RPC request on a subscriber that you can immediately drop (a pseudo-one-shot channel). It doesn’t have to be an event bus (even though that’s close to how Redis does pubsub internally).
- derefr 5y agoWhen I say "event bus", I mean "an async RPC architecture using a reliable at-least-once message-queuing model, where clients connect to a message broker [e.g. a Redis stream], and publish RPC workloads there; backends connect to the same message broker, and subscribe as consumers of a shared consumer-group for RPC workloads, greedily take messages from the queue, and do work on them; backends that complete RPC workloads publish the workload-results back to the broker on channels specific to the original clients, while ACKing the original workloads on the workloads channel; and clients subscribe to their own RPC workload-results channel, ACKing messages as they receive them." Event bus is the name for this network architecture. And if you're trying to replicate what synchronous client-server RPC does in a distributed M:N system, it's what you'd have to use. You can't use at-most-once/unreliable PUBSUB to replicate how synchronous client-server RPC works, as a client might sit around forever waiting for a response that got silently dropped due to the broker or a backend crashing, without knowing it. All the queues and ACKs are there to replicate what clients get for free from having a direct TCP connection to the server. (Yes, Erlang uses timeouts on gen_server:call to build up distributed-systems abstractions on top of an unreliable message carrier. But everything else in an Erlang system has to be explicitly engineered around having timeouts on one end and idempotent handling of potentially-spurious "leftover" requests on the other. Clients that were originally doing synchronous RPC, where you don't know exactly how they were relying on that synchronous RPC, can switch to a Redis-streams event-bus based messaging protocol as a drop-in replacement for their synchronous client-server RPC, because reliable at-least-once async delivery can embed the semantics of synchronous RPC; but they can't switch to unreliable async pubsub as a drop-in replacement for their synchronous client-server RPC. Doing the latter would require investigation and potentially re-engineering, on both sides. If you don't control one end — e.g. if the clients are third-party mobile apps — then that re-engineering might even be impossible.)