4 ms·
I work on OpenTelemetry Java, including a Java Agent that is originally a fork from Datadog's excellent base which they contributed to the project. Protobuf de
by anuraaga 6y ago
I work on OpenTelemetry Java, including a Java Agent that is originally a fork from Datadog's excellent base which they contributed to the project.
Protobuf definitely has a sizable footprint, but I guess it's still ok compared to other popular libraries like Guava and Jackson. I suspect Richard is saving an article for gRPC+Netty which is extremely large :-) Lots of classes, and every shaded Netty on a classpath has its own arena of pooled buffers.
Wire format I like well enough though. I think protobuf does provide good expressiveness to allow nesting if that fits the bill, or more flat structures, where it is a joy to output repeated fields completely independently and even interleaved in a stream.
It's unclear to me if protobuf itself is a bad wire format for telemetry or heavy use of nesting is. The OpenTelemetry Protocol is highly nested to be completely denormalized. This reduces data on the wire (or not so much compared to gzip, but can be considered a hand-coded gzip), but it means all data has to be traversed before any can be sent to compute the length prefix. I don't know which wins in practice - I've just never seen any comparison of the exact same data with different formats. I'm hoping the CPU cache alleviates the double traversal though. But a less nested format in protobuf would easily be possible.