22 ms·

How many RPC/messaging protocols are there to come out of google? Protobuf/GRPC, Flatbuffers, FIDL, more I can't think of?

dmitrygr 5y ago
A company that never encourages or rewards fixing things, but does reward making new things is bound to produce many versions of the same thing. How many chat systems does google have? And how many dozens more have they already killed?
- Barrin92 5y ago
  “No man ever steps in the same river twice. For it's not the same river and he's not the same man.” There can be a lot of value in rebuilding the same things from the ground up. There'll always be differences and novel things that people earlier hadn't seen. Of course it's costly and disruptive because you kill a lot of stuff, but especially in software building new things is relatively fast. People will always complain about companies that kill a lot of products or software but at the end of the day they get more chances to build things from the ground up.
  - doctor_eval 5y ago
    But it's not fast for the people who follow. That's the problem. If you create a product then there is an external cost borne by the users who need to use it. And then if you replace that product with some other product, those users need to retrain. It's really easy to create a new product, but it is a commitment for someone else to adopt it. And just because the cost is externalised, doesn't mean it doesn't exist.
  - rvba 5y ago
    > There can be a lot of value in rebuilding the same things from the ground up. Maybe there is value for managers (who got promotions) and programmers (who got promotions and learned something), but for Google there is no value at all - it losts its users. I remember everyone using gmail chat.. that they "remade" (killed) like two times. Everyone just moved on. Also I dont understand why investors arent unhappy with that. Google can still earn hundreds of millions on chat. Those wont be billions, but it is still money. And "hudred million here, hundred million there" start to add up.
- DaiPlusPlus 5y ago
  > How many chat systems does google have? * Google+ (rip) * Google Allo * Google Chat * Google Groups (separate from Google's Usenet service) * Google Hangouts * Google Messenger * Google Duo * SMS via Google Project Fi * Google Voice > And how many dozens more have they already killed? Too many: https://killedbygoogle.com/ https://killedbygoogle.com/
  - cco 5y ago
    You're missing the chat features of Google Maps, Google Pay, and Google Photos...maybe a few others I'm missing?
    - slightknack 5y ago
      Don't forget the built-in chat in google docs that nobody uses! (Yes, the one other than the commenting system).
      - cco 5y ago
        Oh my goodness, TIL. I know what I'll be doing tomorrow at work...
    - mastax 5y ago
      YouTube had a chat feature at one point recently.
  - google234123 5y ago
    You listed a number of services that are closed or have been merged together to be fair. I'm also not sure what the point of listing sms from a phone service is.
  - sniperjzp 5y ago
    Listing the messaging App here is like comparing apples to oranges, gRPC and FIDL are for different purposes from technical perspective, people must have evaluated leveraging existing tech vs. creating a new one before the investment.
- joshuamorton 5y ago
  Protobuf and FIDL have very different design goals though. One is meant for backwards compatible over the wire communication between loosely coupled RPC systems, and one is meant to be a highly efficient IPC. I expect, for example, that it is vastly more efficient to send a known datastructure over the wire in FIDL, but it is vastly more efficient to parse, modify/extend a repeated subfield, and then reserialize a protobuf. You can't "fix" a system to achieve two goals that are in conflict, sometimes it is in fact better to have N opinionated systems, than one that is only okay at everything.
  - surajrmal 5y ago
    FIDL actually does make backwards compatibility a core concern. Being able to modify an existing type and allowing client and servers to continue to work when out of sync is achieved through a similar mechanism by which protobuf achieves it - a data type in which all members are optional. That said, there are certainly many design decisions that FIDL does do differently because the constraints of the problems it tries to solve are different as you mention.
    - nly 5y ago
      Skimming the docs, it looks like the heavyweight use use of "tables", much like Flatbuffers, seems to be the only mechanism for forward evolution of a protocol. Looks like "structs" you can only remove boxed fields by setting their references to null, but not add fields.
      - surajrmal 5y ago
        Other types such as unions, enums, and bits also have properties that allow for evolution: https://fuchsia.dev/fuchsia-src/development/languages/fidl/guides/compatibility?hl=en#safe_changes_to_members https://fuchsia.dev/fuchsia-src/development/languages/fidl/g...
  - moltenguardian 5y ago
    Are old applications going to run on new kernels? If so, backwards compatibility is a primary goal of an IPC dialog.
- deleted 5y ago
  [deleted]
surajrmal 5y ago
Each serves a different purpose and makes different trade-offs as a result. Flatbuffers is a serialization format, not an rpc solution. grpc is optimized for rpc, whereas FIDL is meant for ipc. Also if you are trying to be comprehensive, don't forget about AIDL/Binder which is used by Android, and mojo, which is used by Chrome. Both of these are IPC, however they are not general enough to be used outside of their respective platforms.
- vletal 5y ago
  It's there really some reason not to generalize IPC to RPC "between processes"? To be more specific, is there really a justifiable specific of either of these applications which prevent us from having a single tool which would cover both?
  - surajrmal 5y ago
    Sure. In IPC, you can have shared resources such as memory. In RPC, you need to assume all parties may be on separate systems. In IPC, you may be able to make guarantees about message ordering that are more challenging with RPC. For instance, packets may be dropped over a network link and packets may arrive out of order. In IPC, you can express backpressure without the need for packet dropping, and therefore can avoid overhead of things like TCP. With IPC, links are trusted, so you can avoid encrypting messages, but that is not something you can safely assume with RPC. Because of these sorts of things, the cost of sending a message is relatively low (measured in us) compared to RPC (measured in hundreds of us), and so the cost of serialization may be of greater concern as it may start to dominate cost. As a result, you may optimize your serialization format differently. Beyond that, an IPC system may allow for advanced usage of the shared kernel which is not possible across different systems each with a separate kernel.
    - spc476 5y ago
      QNX makes no distinction between IPC and RPC. In fact, a program won't be able to tell if the other end of a message port was on the same CPU, same computer, or network (well, maybe by timing results). You didn't even need to recompile code to get that either. It just works right out of the box.
pjmlp 5y ago
You forgot AIDL, used for Android IPC between apps, and for its pseudo microkernel like functionality introduced in Project Treble.
cmrdporcupine 5y ago
You missed Mojo, the IPC system in chromium/ChromeOS. Except there's also an older legacy system still used in spots, though it doesn't have an IDL it's more defined in C++ macros.

dmitrygr 5y ago

A company that never encourages or rewards fixing things, but does reward making new things is bound to produce many versions of the same thing. How many chat systems does google have? And how many dozens more have they already killed?

Barrin92 5y ago
“No man ever steps in the same river twice. For it's not the same river and he's not the same man.” There can be a lot of value in rebuilding the same things from the ground up. There'll always be differences and novel things that people earlier hadn't seen. Of course it's costly and disruptive because you kill a lot of stuff, but especially in software building new things is relatively fast. People will always complain about companies that kill a lot of products or software but at the end of the day they get more chances to build things from the ground up.
- doctor_eval 5y ago
  But it's not fast for the people who follow. That's the problem. If you create a product then there is an external cost borne by the users who need to use it. And then if you replace that product with some other product, those users need to retrain. It's really easy to create a new product, but it is a commitment for someone else to adopt it. And just because the cost is externalised, doesn't mean it doesn't exist.
- rvba 5y ago
  > There can be a lot of value in rebuilding the same things from the ground up. Maybe there is value for managers (who got promotions) and programmers (who got promotions and learned something), but for Google there is no value at all - it losts its users. I remember everyone using gmail chat.. that they "remade" (killed) like two times. Everyone just moved on. Also I dont understand why investors arent unhappy with that. Google can still earn hundreds of millions on chat. Those wont be billions, but it is still money. And "hudred million here, hundred million there" start to add up.
DaiPlusPlus 5y ago
> How many chat systems does google have? * Google+ (rip) * Google Allo * Google Chat * Google Groups (separate from Google's Usenet service) * Google Hangouts * Google Messenger * Google Duo * SMS via Google Project Fi * Google Voice > And how many dozens more have they already killed? Too many: https://killedbygoogle.com/ https://killedbygoogle.com/
- cco 5y ago
  You're missing the chat features of Google Maps, Google Pay, and Google Photos...maybe a few others I'm missing?
  - slightknack 5y ago
    Don't forget the built-in chat in google docs that nobody uses! (Yes, the one other than the commenting system).
    - cco 5y ago
      Oh my goodness, TIL. I know what I'll be doing tomorrow at work...
  - mastax 5y ago
    YouTube had a chat feature at one point recently.
- google234123 5y ago
  You listed a number of services that are closed or have been merged together to be fair. I'm also not sure what the point of listing sms from a phone service is.
- sniperjzp 5y ago
  Listing the messaging App here is like comparing apples to oranges, gRPC and FIDL are for different purposes from technical perspective, people must have evaluated leveraging existing tech vs. creating a new one before the investment.
joshuamorton 5y ago
Protobuf and FIDL have very different design goals though. One is meant for backwards compatible over the wire communication between loosely coupled RPC systems, and one is meant to be a highly efficient IPC. I expect, for example, that it is vastly more efficient to send a known datastructure over the wire in FIDL, but it is vastly more efficient to parse, modify/extend a repeated subfield, and then reserialize a protobuf. You can't "fix" a system to achieve two goals that are in conflict, sometimes it is in fact better to have N opinionated systems, than one that is only okay at everything.
- surajrmal 5y ago
  FIDL actually does make backwards compatibility a core concern. Being able to modify an existing type and allowing client and servers to continue to work when out of sync is achieved through a similar mechanism by which protobuf achieves it - a data type in which all members are optional. That said, there are certainly many design decisions that FIDL does do differently because the constraints of the problems it tries to solve are different as you mention.
  - nly 5y ago
    Skimming the docs, it looks like the heavyweight use use of "tables", much like Flatbuffers, seems to be the only mechanism for forward evolution of a protocol. Looks like "structs" you can only remove boxed fields by setting their references to null, but not add fields.
    - surajrmal 5y ago
      Other types such as unions, enums, and bits also have properties that allow for evolution: https://fuchsia.dev/fuchsia-src/development/languages/fidl/guides/compatibility?hl=en#safe_changes_to_members https://fuchsia.dev/fuchsia-src/development/languages/fidl/g...
- moltenguardian 5y ago
  Are old applications going to run on new kernels? If so, backwards compatibility is a primary goal of an IPC dialog.
deleted 5y ago
[deleted]

surajrmal 5y ago

Each serves a different purpose and makes different trade-offs as a result. Flatbuffers is a serialization format, not an rpc solution. grpc is optimized for rpc, whereas FIDL is meant for ipc. Also if you are trying to be comprehensive, don't forget about AIDL/Binder which is used by Android, and mojo, which is used by Chrome. Both of these are IPC, however they are not general enough to be used outside of their respective platforms.

vletal 5y ago
It's there really some reason not to generalize IPC to RPC "between processes"? To be more specific, is there really a justifiable specific of either of these applications which prevent us from having a single tool which would cover both?
- surajrmal 5y ago
  Sure. In IPC, you can have shared resources such as memory. In RPC, you need to assume all parties may be on separate systems. In IPC, you may be able to make guarantees about message ordering that are more challenging with RPC. For instance, packets may be dropped over a network link and packets may arrive out of order. In IPC, you can express backpressure without the need for packet dropping, and therefore can avoid overhead of things like TCP. With IPC, links are trusted, so you can avoid encrypting messages, but that is not something you can safely assume with RPC. Because of these sorts of things, the cost of sending a message is relatively low (measured in us) compared to RPC (measured in hundreds of us), and so the cost of serialization may be of greater concern as it may start to dominate cost. As a result, you may optimize your serialization format differently. Beyond that, an IPC system may allow for advanced usage of the shared kernel which is not possible across different systems each with a separate kernel.
  - spc476 5y ago
    QNX makes no distinction between IPC and RPC. In fact, a program won't be able to tell if the other end of a message port was on the same CPU, same computer, or network (well, maybe by timing results). You didn't even need to recompile code to get that either. It just works right out of the box.

pjmlp 5y ago

You forgot AIDL, used for Android IPC between apps, and for its pseudo microkernel like functionality introduced in Project Treble.

cmrdporcupine 5y ago

You missed Mojo, the IPC system in chromium/ChromeOS. Except there's also an older legacy system still used in spots, though it doesn't have an IDL it's more defined in C++ macros.