7 ms·
Using gRPC for (local) inter-process communication (2021)
- kats 2y agoCongrats and welcome to the 80s!
- jeffbee 2y agoInteresting that it is taken on faith that unix sockets are faster than inet sockets.
- dangoodmanUT 2y agoAre there resources suggesting otherwise?
- aoeusnth1 2y agoTell me more, I know nothing about IPC
- eqvinox 2y agoThat's because it's logical that implementing network capable segmentation and flow control is more costly than just moving data with internal, native structures. And looking up random benchmarks yields anything from equal performance to 10x faster for Unix domain.
- bluGill 2y agoIt wouldn't surprise me if inet sockets were more optimized though and so unix sockets ended up slower anyway just because nobody has bothered to make them good (which is probably why some of your benchmarks show equal performance). Benchmarks are important.
- eqvinox 2y agoI agree, but practically speaking they're used en masse all across the field and people did bother to make them good [enough]. I suspect the benchmarks where they come up equal are cases where things are limited by other factors (e.g. syscall overhead), though I don't want to make unfounded accusations :)
- sgtnoodle 2y agoI've spent several years optimizing a specialized IPC mechanism for a work project. I've spent time reviewing the Linux Kernel's unix socket source code to understand obscure edge cases. There isn't really much to optimize - it's just copying bytes between buffers. Most of the complexity of the code has to do with permissions and implementing the ability to send file descriptors. All my benchmarks have unambiguously showed unix sockets to be more performant than loopback TCP for my particular use case.
- pjmlp 2y agoAs often in computing, profiling is a foreign word.
- yetanotherdood 2y agoUnix Domain Sockets are the standard mechanism for app->sidecar communication at Google (ex: Talking to the TI envelope for logging etc.)
- jeffbee 2y agoSearch around on Google Docs for my 2018 treatise/rant about how the TI Envelope was the least-efficient program anyone had ever deployed at Google.
- eqvinox 2y agoOk, now it sounds like you're blaming unix sockets for someone's shitty code... No idea what "TI Envelope" is, and a Google search doesn't come up with usable results (oh the irony...) - if it's a logging/metric thing, those are hard to get to perform well regardless of socket type. We ended up using batching with mmap'd buffers for crash analysis. (I.e. the mmap part only comes in if the process terminates abnormally, so we can recover batched unwritten bits.)
- jeffbee 2y ago> Ok, now it sounds like you're blaming unix sockets for someone's shitty code... No, I am just saying that the unix socket is not Brawndo (or maybe it is?), it does not necessarily have what IPCs crave. Sprinkling it into your architecture may or may not be relevant to the efficiency and performance of the result.
- HumblyTossed 2y ago> Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. However, if your project anyway exposes RPCs for public APIs or would benefit from a schema-based serialisation layer it makes sense to use only one tool that combines these—also for IPC. It might make sense. Usually, if you're using IPC, you need it to be as fast as possible and there are several solutions that are much faster.
- dmoy 2y agoI tend to agree. Usually you want as fast as possible. Sometimes you don't though. E.g. Kythe (kythe.io) was designed so that its individual language indexers run with a main driver binary written in Go, and then a subprocess binary written in.... whatever. There's a requirement to talk between the two, but it's not really a lot of traffic (relative to e.g. the CPU cost of the subprocess doing compilation). So what happens in practice is that we used Stubby (like gRPC, except not public), because it was low overhead* to write the handler code for it on both ends, and got us some free other bits as well. * Except when it wasn't lol. It worked great for the first N languages written in langs with good stubby support. But then weird shit (for Google) crawled out of the weeds that didn't have stubby support, so there's some handwaving going on for the long tail.
- monocasa 2y agoI'm not even sure that I'd say usually. Most of the time you're just saying "hey daemon, do this thing that you're already preconfigured for".
- ks2048 2y agoWhat are the other solutions that are much faster? (besides rolling your own mini format).
- HumblyTossed 2y ago> What are the other solutions that are much faster? Why ask the question, then > (besides rolling your own mini format) box it in? Do you think I'm saying using an RPC is bad? I'm not. I simply took issue with the way the article was worded. The thing about engineering is you don't want to do what some blogger says is best. You want to analyze YOUR particular needs and design your system from that. So, with every properly engineered solution there are trade-offs. You want easy? It may run slower. You want faster? You may have to roll your own.
- pengaru 2y agoThere's a mountain of grpc-centric python code at $dayjob and it's been miserable to live with. Maybe it's less awful in c/c++, or at least confers some decent performance there. In python it's hot garbage.
- andy_ppp 2y agoStrongly agree, it’s has loads of problems, my least favourite being the schema is not checked in the way you might think, there’s not even a checksum to say this message and this version of the schema match. So when there’s old services/clients around and people haven’t versioned their schema’s safely (there was no mechanism for this apart from manually checking in PRs) you can get gibberish back for fields that should contain data. It’s basically just a binary blob with whatever schema the client has overlaid so debugging is an absolute pain. Unless you are Google scale use a text based format like JSON and save yourself a lot of hassle.
- discreteevent 2y ago- JSON doesn't have any schema checking either. - You can encode the protocol buffers as JSON if you want a text based format.
- jayd16 2y agoYou can trivially make breaking changes in a JSON blob too. GRPC has well documented ways to make non-breaking changes. If you're working somewhere where breaking schema changes go in with little fanfare and much debugging then I'm not sure JSON will save you. The only way to know is to dig through CLs? Write a test. There's also automated tooling to compare protobuff schemas for breaking changes.
- andy_ppp 2y agoJSON contains a description of the structure of the data that is readable by both machines and humans. JSON can certainly go wrong but it’s much simpler to see when it has because of this. GRPC is usually a binary black box that adds loads of developer time to upskill, debug, figure out error cases and introduces whole new classes of potential bugs. If you are building something that needs binary performance that GRPC provides, go for it, but pretending there is no extra cost over doing the obvious thing is not true.
- DashAnimal 2y agoWhat I loved about Fuchsia was its IPC interface, using FIDL which is like a more optimized version of protobufs. https://fuchsia.dev/fuchsia-src/get-started/learn/fidl/fidl https://fuchsia.dev/fuchsia-src/get-started/learn/fidl/fidl
- DabbyDabberson 2y agoloved - in the past tense?
- pjmlp 2y ago> Using a full-featured RPC framework for IPC seems like overkill when the processes run on the same machine. That is exactly what COM/WinRT, XPC, Android Binder, D-BUS are. Naturally they have several optimisations for local execution.
- jeffbee 2y agoBinder seriously underappreciated, IMHO. But I think it makes sense to use gRPC or something like it if there is any possibility that in the future an "IPC" will become an "RPC" to a foreign host. You don't want to be stuck trying to change an IPC into an RPC if it was foreseeable that it would eventually become remote due to scale.
- pjmlp 2y agoKind of, as anyone with CORBA, DCOM, RMI, .NET Remoting experience has plenty of war stories regarding distributed computing with the expectations of local calls.
- dietr1ch 2y agoIn my mind the abstraction should allow for RPCs and being on the same machine should allow to optimise things a bit, this way you simply build for the general case and lose little to no performance. Think of the loopback, my programs don't know (or at least shouldn't know) that IPs like 127.0.0.5 are special, but then the kernel knows that messages there are not going to go on any wire and handles that differently.
- bunderbunder 2y agoThis could possibly even be dead simple to accomplish if application-level semantics aren't being communicated by co-opting parts of the communication channel's spec. I think that this factor might be the ultimate source of my discomfort with standards like REST. Things like using HTTP verbs and status codes, and encoding parameters into the request's URL, mean that there's almost not even an option to choose a communication channel that's lighter-weight than HTTP.
- jauntywundrkind 2y agoI super dug the talk Building SpiceDB: A gRPC-First Database - Jimmy Zelinskie, authzed which is about a high-performance auth system, which talks to this. https://youtu.be/1PiknT36218 https://youtu.be/1PiknT36218 It's a 4-tier arhcitecture (clients - front end service - query service - database) auth system, and all communication is over grpc (except to the database). Jimmy talks about the advantages of having a very clear contract between systems. There's a ton of really great nitty gritty detail about being super fast with gRPC. https://github.com/planetscale/vtprotobuf https://github.com/planetscale/vtprotobuf for statical-size allocating protobuf rather than slow reflection-based dynamic size. Upcoming memory pooling work to avoid allocations at all. Tons of advantages for observability right out of the box. It's subtle but I also get the impression most gRPC stubs are miserably bad, that Authzed had to go long and far to get away from a lot of gRPC tarpits. This is one of my favorite talks from 2024, and strongly sold me.on how viable gRPC is for internal services. Even if I were doing local multi-process stuff, I would definitely consider gRPC after this talk. The structure & clarity & observability are huge wins, and the performance can be really good if you need it. https://youtu.be/1PiknT36218#t=12m https://youtu.be/1PiknT36218#t=12m 12min is the internal cluster details.
- jzelinskie 2y agoThank you! That really means a lot! >It's subtle but I also get the impression most gRPC stubs are miserably bad, that Authzed had to go long and far to get away from a lot of gRPC tarpits. They aren't terrible, but they also aren't a user experience you want to deliver directly to your customers.
- zackangelo 2y agoHad to reach for a new IPC mechanism recently to implement a multi-GPU LLM inference server. My original implementation just pinned one GPU to its own thread then used message passing between them in the same process but Nvidia's NCCL library hates this for reasons I haven't fully figured out yet. I considered gRPC for IPC since I was already using it for the server's API but dismissed it because it was an order of magnitude slower and I didn't want to drag async into the child PIDs. Serializing the tensors between processes and using the Servo team's ipc-channel crate[0] has worked surprisingly well. If you're using Rust and need a drop-in (ish) replacement for the standard library's channels, give it a shot. [0] https://github.com/servo/ipc-channel https://github.com/servo/ipc-channel
- anilakar 2y agoIn addition to cloud connectivity, we've been using MQTT for IPC in our Linux IIoT gateways and touchscreen terminals and honestly it's been one of the better architectural decisions we've made. Implementing new components for specific customer use cases could not be easier and the component can be easily placed on the hardware or on cloud servers wherever it fits best. I don't see how gRPC could be any worse than that. (The previous iteration before MQTT used HTTP polling and callbacks worked on top of an SSH reverse tunnel abomination. Using MQTT for IPC was kind of an afterthought. The SSH Cthulhu is still in use for everyday remote management because you cannot do Ansible over MQTT, but we're slowly replacing it with Wireguard. I gotta admit that out of all VPN technologies we've experimented with, SSH transport has been the most reliable one in various hostile firewalled environments.)
- goalieca 2y agoMQTT also lends itself to async very well. The event based approach is a real winner.
- DanielHB 2y agoTrying to understand an existing IoT codebase is scarier than reading Lovecraft. Out of curiosity, why were you using SSH reverse tunnel for IPC? Were you using virtualization inside your iot device and for some reason need a tunnel between the guests?
- anilakar 2y agoReverse tunnel SSH was used for remote management. Once we replaced it with MQTT, we realized we could use it for IPC, too. I was not too clear about this in the post above. Before MQTT we used whatever we had at hand; the touchscreen frontend (that ran Qt, not an embedded browser) talked to the local backend with HTTP. SMS messages were first sent to another local HTTP gateway and then via UNIX shm to a master process that templated the message to the smstools3 spool directory. A customer-facing remote management GUI ran on the device's built-in HTTP server but I no longer remember how it interfaced with the rest of the system. The touchscreen GUI might actually have had its own HTTP server for that, because it also handled Modbus communication to external I/O. This was back in the day when everyone was building their own IoT platforms and obviously we were required to dogfood our own product, trying to make it do things it was never meant for. The gateway was originally designed to be an easy solution for AV integrators to remotely manage whole campuses from a single GUI. I'm pretty sure no customer ever wrote a line of code on the platform.
- justinsaccount 2y ago> In our scenario of local IPC, some obvious tuning options exist: data is exchanged via a Unix domain socket (unix:// address) instead of a TCP socket AFAIK at least on linux there is no difference between using a UDS and a tcp socket connected to localhost.
- sgtnoodle 2y agoThere's definitely differences, whether or not it matters for most usages. I've worked on several IPC mechanisms that specifically benefited from one vs. the other.
- deleted 2y ago[deleted]
- palata 2y agoI have been in a similar situation, and gRPC feels heavy. It comes with quite a few dependencies (nothing compared to npm or cargo systems routinely bringing hundreds of course, but enough to be annoying when you have to cross-compile them). Also at first it sounds like you will benefit from all the languages that protobuf supports, but in practice it's not that perfect: some python package may rely on the C++ implementation, and therefore you need to compile it for your specific platform. Some language implementations are just maintain by one person in their free time (a great person, but still), etc. On the other hand, I really like the design of Cap'n Proto, and the library is more lightweight (and hence easier) to compile. But there, it is not clear on which language implementation you can rely other than C++. Also it feels like there are maintainers paid by Google for gRPC, and for Cap'n Proto it's not so clear: it feels like it's essentially Cloudflare employees improving Cap'n Proto for Cloudflare. So if it works perfectly for your use-case, that's great, but I wouldn't expect much support. All that to say: my preferred choice for that would technically be Cap'n Proto, but I wouldn't dare making my company depend on it. Whereas nobody can fire me for depending on Google, I suppose.
- kentonv 2y ago> it feels like it's essentially Cloudflare employees improving Cap'n Proto for Cloudflare. That's correct. At present, it is not anyone's objective to make Cap'n Proto appeal to a mass market. Instead, we maintain it for our specific use cases in Cloudflare. Hopefully it's useful to others too, but if you choose to use it, you should expect that if any changes are needed for your use case, you will have to make those changes yourself. I certainly understand why most people would shy away from that. With that said, gRPC is arguably weird in its own way. I think most people assume that gRPC is what Google is built on, therefore it must be good. But it actually isn't -- internally, Google uses Stubby. gRPC is inspired by Stubby, but very different in implementation. So, who exactly is gRPC's target audience? What makes Google feel it's worthwhile to have 40ish(?) people working on an open source project that they don't actually use much themselves? Honest questions -- I don't know the answer, but I'd like to. (FWIW, the story is a bit different with Protobuf. The Protobuf code is the same code Google uses internally.) (I am the author of Cap'n Proto and also was the one who open sourced Protobuf originally at Google.)
- lyu07282 2y agoCool that they mention buf, it's such a massive improvement over Google's own half abandoned crappy protobuf implementation https://github.com/bufbuild/buf https://github.com/bufbuild/buf
- DanielHB 2y agoAt a project I worked at we were considering using protobuf for IPC between our desktop app and our network framework code which used different languages. The performance was part of the reason (compared to serializing using JSON) but the main reason was just tooling support for automatic type checking. gRPC can generate types from a schema for all popular languages out there. We ended up taking another route but I feel it is important to consider the existing tools ahead of any performance concerns for most cases
- T3RMINATED 2y ago[dead]
- up2isomorphism 2y agoThis the typical situation where : “you can do it” doesn’t mean you should do it, and there is very little sense to advertise it, if at all.