3 ms·

The benchmarks are in the repo + provided as screenshots in the README, but to clarify this is speaking to performance improvements relative to v2 of Bebop.

Kelteseth 3y ago
Ah, thanks! Would be interesting to see benchmarks against cCap'n Proto
- andrew_rfc 3y ago
  We don’t benchmark against FlatBuffers or Cap'n Proto for a few reasons: - They are impossible to benchmark against each other without making an assumption about how often you want to access the data, and which parts of it you want to access. But this means Bebop, Cap'n Proto, FlatBuffers can exist side-by-side / solve different problems: what FlatBuffers and Cap'n Proto do makes sense if you want to access only parts of your objects in limited specific ways, what we do is better if you're always interested in the whole packet. - We don’t benchmark against Capt’n Proto because it does not have a stable web-based implementation, at least not one that has the features that make it so fast natively, so there is nothing to compare. Basically, Bebop is great for message oriented applications or where you need the entirety of your packet deserialized in a single step. So we can only benchmark against similar formats (JSON, Protobufs, MsgPack, etc.)
- neonsunset 3y ago
  cap'n'proto is ancient, unmaintained, and its C# library is one of the most poorly written binary serialization implementations I have seen in my life. You will be better served by gRPC and, if you need absolute maximum performance, MemoryPack (it is C#-first but can generate TypeScript clients). If you need MQ, NATS is a good choice (special kudos to them adopting Yoshifumi Kawai's AlterNATS into NATS v2 C# client implementation).
  - andrew_rfc 3y ago
    If we’re talking specifically about C# binary serialization I’d recommend checking out this independent benchmark another developer did comparing Bebop to everything else https://youtu.be/XMoNYQPi2k8?feature=shared https://youtu.be/XMoNYQPi2k8?feature=shared
    - neonsunset 3y ago
      The comparison uses old and/or deprecated/EOL serializers which no one in their sane mind should use today. Consider referring to my previous comment, thanks! (and by deprecated I mean deprecated 5+ years ago like in the case of Utf8Json) Sadly a lot of videos discussing performance do only limited research and do not approach the matter comprehensively (likely not worth it for YT algorithm favoring quick and easy content). Nothing wrong with offering your own first-party use cases that attempt to simulate realistic workloads on top of suggested patterns for each respective contestant, but it is important to include more performance-oriented frameworks/libraries (which is why I mentioned MemoryPack) that are used e.g. in gamedev sphere which cares a lot about realtime communication. With all that said, it's actually awesome to see a framework that offers first-class C# support, so I probably need to shut up and appreciate the fact :)
  - kentonv 3y ago
    > cap'n'proto is ancient, I mean... not compared to protobuf. > unmaintained, The C++ implementation is well-maintained and heavily used by its own maintainers (e.g. me). Other implementations are maintained to varying degrees, since each implementation has a different maintainer. Agreed the C# implementation isn't maintained. > You will be better served by gRPC Well, this really depends, considering that Cap'n Proto's RPC system is quite different and far more expressive than gRPC.

Kelteseth 3y ago

Ah, thanks! Would be interesting to see benchmarks against cCap'n Proto

andrew_rfc 3y ago
We don’t benchmark against FlatBuffers or Cap'n Proto for a few reasons: - They are impossible to benchmark against each other without making an assumption about how often you want to access the data, and which parts of it you want to access. But this means Bebop, Cap'n Proto, FlatBuffers can exist side-by-side / solve different problems: what FlatBuffers and Cap'n Proto do makes sense if you want to access only parts of your objects in limited specific ways, what we do is better if you're always interested in the whole packet. - We don’t benchmark against Capt’n Proto because it does not have a stable web-based implementation, at least not one that has the features that make it so fast natively, so there is nothing to compare. Basically, Bebop is great for message oriented applications or where you need the entirety of your packet deserialized in a single step. So we can only benchmark against similar formats (JSON, Protobufs, MsgPack, etc.)
neonsunset 3y ago
cap'n'proto is ancient, unmaintained, and its C# library is one of the most poorly written binary serialization implementations I have seen in my life. You will be better served by gRPC and, if you need absolute maximum performance, MemoryPack (it is C#-first but can generate TypeScript clients). If you need MQ, NATS is a good choice (special kudos to them adopting Yoshifumi Kawai's AlterNATS into NATS v2 C# client implementation).
- andrew_rfc 3y ago
  If we’re talking specifically about C# binary serialization I’d recommend checking out this independent benchmark another developer did comparing Bebop to everything else https://youtu.be/XMoNYQPi2k8?feature=shared https://youtu.be/XMoNYQPi2k8?feature=shared
  - neonsunset 3y ago
    The comparison uses old and/or deprecated/EOL serializers which no one in their sane mind should use today. Consider referring to my previous comment, thanks! (and by deprecated I mean deprecated 5+ years ago like in the case of Utf8Json) Sadly a lot of videos discussing performance do only limited research and do not approach the matter comprehensively (likely not worth it for YT algorithm favoring quick and easy content). Nothing wrong with offering your own first-party use cases that attempt to simulate realistic workloads on top of suggested patterns for each respective contestant, but it is important to include more performance-oriented frameworks/libraries (which is why I mentioned MemoryPack) that are used e.g. in gamedev sphere which cares a lot about realtime communication. With all that said, it's actually awesome to see a framework that offers first-class C# support, so I probably need to shut up and appreciate the fact :)
- kentonv 3y ago
  > cap'n'proto is ancient, I mean... not compared to protobuf. > unmaintained, The C++ implementation is well-maintained and heavily used by its own maintainers (e.g. me). Other implementations are maintained to varying degrees, since each implementation has a different maintainer. Agreed the C# implementation isn't maintained. > You will be better served by gRPC Well, this really depends, considering that Cap'n Proto's RPC system is quite different and far more expressive than gRPC.