5 ms·

Faster Protocol Buffers (2019)

59 points by ngaut 4y ago

serbrech 4y ago
The partial decoding is also interesting for another case, I think: polymorphism. The OneOf implementation in go protobuf is ugly, but this trick could make it bareable. I'll have to explore further :)
- tignaj 4y ago
  Here is a OneOf Go implementation I wrote that hopefully is less ugly and is significantly faster: https://github.com/splunk/exp-lazyproto#oneof-fields https://github.com/splunk/exp-lazyproto#oneof-fields
jeffbee 4y ago
In this case it is also possible to encode the message up to and including fields 1 and 2, then repeatedly encoding field 3 and concatenating the resulting buffers. Protobuf is designed to do this.
- arccy 4y ago
  that's less useful if you're trying to do it over multiple rpc calls, which is how this is used
- secondcoming 4y ago
  Oooh, I never even thought about trying that!
vvanders 4y ago
My go-to for needing to deserialize structured data in a fast way these days is flatbuffers[1]. It compacts nicely and more importantly is zero copy/allocation(within the constraints of your language where possible) in deserialize. Which lets you do neat things like mmap it from disk. We used to store 20-30mb of animation data with it and we'd just mmap the whole file and let the kernel handle paging it in/out, worked great. I don't know how up to date their benchmarks[2] are but my experience has been that it beats almost every other off-the-shelf solution(other than maybe capn-proto which has some similar properties). [1] https://google.github.io/flatbuffers/ https://google.github.io/flatbuffers/ [2] https://google.github.io/flatbuffers/flatbuffers_benchmarks.html https://google.github.io/flatbuffers/flatbuffers_benchmarks....
- jeffbee 4y ago
  Those old benchmarks (that have since been deleted) are highly misleading. The protobuf column is essentially what you would get if you just ignored all the performance best practices of protobuf. There's no reason why in C++ you'd need to allocate and deallocate a message struct for every parse. You just reuse it. And you can have allocation-free parses if you want, and you can alias network or file buffers if you want. But the main reason people use protobuf is it is compact, which this table does accurately show. Flatbuffers are always larger.
  - vvanders 4y ago
    There is a tradeoff in space / performance, I've just never found for my use cases it was large enough vs the significant(~10x) perf boost I got. From what I recall in the message format[1] protocol buffers require you to walk quite a few parts of the data structures where in flatbuffers you are just reading offset and traversal is significantly faster. Flatbuffers also let's you control data layout into the packed buffer so you can guarantee cache locality which makes a huge difference when it comes to real-world traversal performance. Data oriented design and FlatBuffers go together really well(which isn't a surprise given some of the background of where FlatBuffers comes from). [1] https://developers.google.com/protocol-buffers/docs/encoding#order https://developers.google.com/protocol-buffers/docs/encoding...
    - jeffbee 4y ago
      OK, but I just want readers to be aware that the whole idea that it could take five minutes to parse a million protobufs is completely preposterous. I reimplemented their benchmark just now and it runs at roughly 8 million protos per second, orders of magnitude faster than they state, and I didn't even do anything to optimize it. https://github.com/jwbee/protobuf-flatbuffer-benchmark https://github.com/jwbee/protobuf-flatbuffer-benchmark
alexchamberlain 4y ago
Did the OP consider sending less data? For example, registering the metric and using an int, rather than sending all of the static data every time. I guess the major downside there is it would make the metric server stateful and I think OpenTelemetry uses UDP.
- lokar 4y ago
  Otel uses grpc, tcp. The issue is the agent/collector is stateless and agnostic about where the data eventually goes. It's not the final destination, just an API router / transformer.
  - tignaj 4y ago
    This. The statelessness of the OTLP is by design. I did consider stateful designs with e.g. shared state dictionary compression but eventually chose not to, so that the intermediaries can remain stateless. An extension to OTLP that uses shared state (and columnar encoding) to achieve more compact representation and is suitable for the last network leg in the data delivery path has been proposed and may become a reality in the future: https://github.com/open-telemetry/oteps/pull/171 https://github.com/open-telemetry/oteps/pull/171
    - jiggawatts 4y ago
      Windows has something like 15,000 performance counters and error metrics that can be collected. There isn’t a system on earth that can even approach this. At scale, I have to pick and choose maybe 20-100 counters for fear of overloading a cluster(!) of servers collecting the data… once a minute. That’s because the protocol overheads cause “write multiplication” of a hundred-to-one or worse. Every byte of metric ends up nearly a kilobyte on the wire. Meanwhile I did some experiments that showed that even with a tiny bit of crude data-oriented design and delta compression a single box could collect 10K metrics across 10K endpoints every second without breaking a sweat. The modern REST / RPC approach is fine for business apps but is an unmitigated disaster for collecting tiny metrics. Set your goals higher than collecting a selected subset of 1% of the available metrics 60x less frequently than admins would like…
quietbritishjim 4y ago
One option not mentioned in this article is use of unknown fields [1] (e.g. C++ documentation is at [2]). Rather than changing the types of the field from a message to bytes, simply remove them altogether: message Metric { // MetricDescriptor metric_descriptor = 1; // Resource resource = 2; repeated Int64TimeSeries int64_timeseries = 3; } When then message is deserialised, those will be placed in a separate binary buffer, which is used directly when reserialising. It even works for other types of field. Originally this functionality was lost in the transition from proto2 to proto3 but was added back a long time ago now. This feels like a cleaner solution to me as it's what this feature is really intended for. [1] https://developers.google.com/protocol-buffers/docs/proto3#unknowns https://developers.google.com/protocol-buffers/docs/proto3#u... [2] https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.unknown_field_set https://developers.google.com/protocol-buffers/docs/referenc...
tignaj 4y ago
Article author here, good to see it on HN, someone else has submitted it (thanks :-)). If you are interested in the topic you may be also interested in a research library I wrote recently: https://github.com/splunk/exp-lazyproto https://github.com/splunk/exp-lazyproto, which among other things exploits the partial (de)serialization technique. This is just a prototype for now, one day I may actually do a production quality implementation.
- ploppyploppy 4y ago
  How relevant is this (excellent) post in 2022? Has the tech changed at all on this front?

59 points by ngaut 4y ago

The partial decoding is also interesting for another case, I think: polymorphism. The OneOf implementation in go protobuf is ugly, but this trick could make it bareable. I'll have to explore further :)

tignaj 4y ago
Here is a OneOf Go implementation I wrote that hopefully is less ugly and is significantly faster: https://github.com/splunk/exp-lazyproto#oneof-fields https://github.com/splunk/exp-lazyproto#oneof-fields

jeffbee 4y ago

In this case it is also possible to encode the message up to and including fields 1 and 2, then repeatedly encoding field 3 and concatenating the resulting buffers. Protobuf is designed to do this.

arccy 4y ago
that's less useful if you're trying to do it over multiple rpc calls, which is how this is used
secondcoming 4y ago
Oooh, I never even thought about trying that!

vvanders 4y ago

My go-to for needing to deserialize structured data in a fast way these days is flatbuffers[1]. It compacts nicely and more importantly is zero copy/allocation(within the constraints of your language where possible) in deserialize. Which lets you do neat things like mmap it from disk. We used to store 20-30mb of animation data with it and we'd just mmap the whole file and let the kernel handle paging it in/out, worked great. I don't know how up to date their benchmarks[2] are but my experience has been that it beats almost every other off-the-shelf solution(other than maybe capn-proto which has some similar properties). [1] https://google.github.io/flatbuffers/ https://google.github.io/flatbuffers/ [2] https://google.github.io/flatbuffers/flatbuffers_benchmarks.html https://google.github.io/flatbuffers/flatbuffers_benchmarks....

jeffbee 4y ago
Those old benchmarks (that have since been deleted) are highly misleading. The protobuf column is essentially what you would get if you just ignored all the performance best practices of protobuf. There's no reason why in C++ you'd need to allocate and deallocate a message struct for every parse. You just reuse it. And you can have allocation-free parses if you want, and you can alias network or file buffers if you want. But the main reason people use protobuf is it is compact, which this table does accurately show. Flatbuffers are always larger.
- vvanders 4y ago
  There is a tradeoff in space / performance, I've just never found for my use cases it was large enough vs the significant(~10x) perf boost I got. From what I recall in the message format[1] protocol buffers require you to walk quite a few parts of the data structures where in flatbuffers you are just reading offset and traversal is significantly faster. Flatbuffers also let's you control data layout into the packed buffer so you can guarantee cache locality which makes a huge difference when it comes to real-world traversal performance. Data oriented design and FlatBuffers go together really well(which isn't a surprise given some of the background of where FlatBuffers comes from). [1] https://developers.google.com/protocol-buffers/docs/encoding#order https://developers.google.com/protocol-buffers/docs/encoding...
  - jeffbee 4y ago
    OK, but I just want readers to be aware that the whole idea that it could take five minutes to parse a million protobufs is completely preposterous. I reimplemented their benchmark just now and it runs at roughly 8 million protos per second, orders of magnitude faster than they state, and I didn't even do anything to optimize it. https://github.com/jwbee/protobuf-flatbuffer-benchmark https://github.com/jwbee/protobuf-flatbuffer-benchmark

alexchamberlain 4y ago

Did the OP consider sending less data? For example, registering the metric and using an int, rather than sending all of the static data every time. I guess the major downside there is it would make the metric server stateful and I think OpenTelemetry uses UDP.

lokar 4y ago
Otel uses grpc, tcp. The issue is the agent/collector is stateless and agnostic about where the data eventually goes. It's not the final destination, just an API router / transformer.
- tignaj 4y ago
  This. The statelessness of the OTLP is by design. I did consider stateful designs with e.g. shared state dictionary compression but eventually chose not to, so that the intermediaries can remain stateless. An extension to OTLP that uses shared state (and columnar encoding) to achieve more compact representation and is suitable for the last network leg in the data delivery path has been proposed and may become a reality in the future: https://github.com/open-telemetry/oteps/pull/171 https://github.com/open-telemetry/oteps/pull/171
  - jiggawatts 4y ago
    Windows has something like 15,000 performance counters and error metrics that can be collected. There isn’t a system on earth that can even approach this. At scale, I have to pick and choose maybe 20-100 counters for fear of overloading a cluster(!) of servers collecting the data… once a minute. That’s because the protocol overheads cause “write multiplication” of a hundred-to-one or worse. Every byte of metric ends up nearly a kilobyte on the wire. Meanwhile I did some experiments that showed that even with a tiny bit of crude data-oriented design and delta compression a single box could collect 10K metrics across 10K endpoints every second without breaking a sweat. The modern REST / RPC approach is fine for business apps but is an unmitigated disaster for collecting tiny metrics. Set your goals higher than collecting a selected subset of 1% of the available metrics 60x less frequently than admins would like…

quietbritishjim 4y ago

One option not mentioned in this article is use of unknown fields [1] (e.g. C++ documentation is at [2]). Rather than changing the types of the field from a message to bytes, simply remove them altogether: message Metric { // MetricDescriptor metric_descriptor = 1; // Resource resource = 2; repeated Int64TimeSeries int64_timeseries = 3; } When then message is deserialised, those will be placed in a separate binary buffer, which is used directly when reserialising. It even works for other types of field. Originally this functionality was lost in the transition from proto2 to proto3 but was added back a long time ago now. This feels like a cleaner solution to me as it's what this feature is really intended for. [1] https://developers.google.com/protocol-buffers/docs/proto3#unknowns https://developers.google.com/protocol-buffers/docs/proto3#u... [2] https://developers.google.com/protocol-buffers/docs/reference/cpp/google.protobuf.unknown_field_set https://developers.google.com/protocol-buffers/docs/referenc...

tignaj 4y ago

Article author here, good to see it on HN, someone else has submitted it (thanks :-)). If you are interested in the topic you may be also interested in a research library I wrote recently: https://github.com/splunk/exp-lazyproto https://github.com/splunk/exp-lazyproto, which among other things exploits the partial (de)serialization technique. This is just a prototype for now, one day I may actually do a production quality implementation.

ploppyploppy 4y ago
How relevant is this (excellent) post in 2022? Has the tech changed at all on this front?