9 ms·

Parsing Protobuf like never before

248 points by ibobev 1y ago

See also https://buf.build/blog/hyperpb https://buf.build/blog/hyperpb (via https://news.ycombinator.com/item?id=44661785 https://news.ycombinator.com/item?id=44661785)

UncleEntity 1y ago
> In other words, a UPB parser is actually configuration for an interpreter VM, which executes Protobuf messages as its bytecode. This is kind of confusing, the VM is runtime crafted to parse a single protobuf message type and only this message type? The Second Futamura Projection, I suppose... Or the VM is designed specifically around generic protobuf messages and it can parse any random message but only if it's a protobuf message? I've been working on the design of a similar system but for general binary parsing (think bison/yacc for binary data) and hadn't even considered doing data over specialized VM vs. bytecode+data over general VM. Honestly, since it's designed around 'maximum laziness' (it just parses/verifies and creates metadata over the input so you only pay for decoding bytes you actually use) and I/O overhead is way greater than the VM dispatching trying this out is probably one of those "premature optimization is the root of all evil" cases but intriguing none the less.
- tonyarkles 1y ago
  Based on what I know about the structure of protobufs internally and without having looked deep into what UPB is doing... I'd guess it could probably be a stack machine that treats (byte)+ as opcodes. Most of the time I'd think of it as parser -> AST -> bytecode, but I think the "grammar" of protobufs would allow your parser to essentially emit terminals as they're parsed straight to the VM as instructions to execute.
  - UncleEntity 1y ago
    In the couple days since I posted my confusion (threads merged or something) I consulted the daffy robots and figured out how it all works. Also had them come up with a design document for "a specialized compiler and virtual machine architecture for parsing Protocol Buffer messages that achieves significant performance improvements through a novel compilation pipeline combining protobuf-specific AST optimization, continuation-passing style transformations, and tail call interpreter execution." Interesting times we live in...
- haberman 1y ago
  I think I can shed some light on this, as the creator and lead of upb. Calling a Protobuf Parser an "interpreter VM" is a little bit of rhetorical flourish. It comes from the observation that there are some deep structural similarities between the two, which I first observed in an article a few years back: https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html https://blog.reverberate.org/2021/04/21/musttail-efficient-i... > It may seem odd to compare interpreter loops to protobuf parsers, but the nature of the protobuf wire format makes them more similar than you might expect. The protobuf wire format is a series of tag/value pairs, where the tag contains a field number and wire type. This tag acts similarly to an interpreter opcode: it tells us what operation we need to perform to parse this field’s data. Like interpreter opcodes, protobuf field numbers can come in any order, so we have to be prepared to dispatch to any part of the code at any time. This means that the overall structure of a protobuf parser is conceptually a while() loop surrounding a switch() statement, just like a VM interpreter. The tricky part is that the set of "case" labels for a Protobuf parser is message-specific and defined by the fields in the schema. How do we accommodate that? The traditional answer was to generate a function per message and use the schema's field numbers as the case labels. You can see an example of that here (in C++): https://github.com/protocolbuffers/protobuf/blob/f763a2a86084371fd0da95f3eeb879c2ff26b06d/src/google/protobuf/descriptor.pb.cc#L2175-L2227 https://github.com/protocolbuffers/protobuf/blob/f763a2a8608... More recently, we've moved towards making Protobuf parsing more data-driven, where each field's schema is compiled into data that is passed as an argument to a generic Protobuf parser function. We call this "table-driven parsing", and from my read of the blog article, I believe this is what Miguel is doing with hyperpb. The trick then becomes how to make this table-driven dispatch as fast as possible, to simulate what the switch() statement would have done. That question is what I cover at length in the article mentioned above.
  - anonymoushn 1y ago
    Really great. I wonder, for the "types encoded as code" approach, is there any benefit to fast paths for data with fields in ascending order? For some json parsers with types encoded as code I have observed some speedup from either hard-coding a known key order or assuming keys in some order and providing a fallback in case an unexpected key is encountered. For users who are stuck with protobuf forever because of various services using it and various data being encoded this way, the historical data could plausibly be canonicalized and written back in large chunks when it is accessed, so that one need not pay the entire cost of canonicalizing it all at once. But of course the icache concerns are still just as bad.
mdhb 1y ago
I’d really love to see more work bringing the best parts of protobuf to a standardised serialization format like CBOR. I’d make the same argument for gRPC-web to something like WHATWG streams and or WebTransport. There is a lot of really cool and important learnings in both but it’s also so tied up in weird tooling and assumptions. Let’s rebase on IETF and W3C standards
- youngtaff 1y ago
  Would be good to see support for encoding / decoding CBOR exposed as a broswer API - they currently use CBOR internally for WebAuthn so I’d hope it’s bnot too hard
- cyberax 1y ago
  You can easily do this. Protobuf supports pluggable writers, and iterating over a schema is pretty easy. We do it for the JSONB. I'm not sure the purpose, though. Protobuf is great for its inflexible schema, and CBOR is great for its flexible data representation. A separate CBOR schema would be a better fit, there's CDDL but it has no traction.
irq-1 1y ago
https://github.com/bufbuild/hyperpb-go https://github.com/bufbuild/hyperpb-go
skybrian 1y ago
This is excellent: an in-depth description showing how the Go internals make writing fast interpreters difficult, by someone who is far more determined than I ever was to make it fast anyway. I’ve assumed that writing fast interpreters wasn’t a use case the Go team cared much about, but if it makes protobuf parsing faster, maybe it will get some attention, and some of these low-level tricks will no longer be necessary?
- RGBCube 1y ago
  > and some of these low-level tricks will no longer be necessary? Don't count on it. This language is Golang.
  - anonymoushn 1y ago
    Hi to my favorite turkish teenager. You really don't get to generate fast codecs from normal-looking code in any language :(
    - actionfromafar 1y ago
      In as-far-as wuffs looks normal, wuffs?
      - anonymoushn 1y ago
        wuffs is extremely cool, but you do find the expected blobs of intrinsics for specific simd extensions in this code.
alexozer 1y ago
So am I identifying the bottlenecks that motivate this design correctly? 1. Go FFI is slow 2. Per-proto generated code specialization is slow, because of icache pressure I know there's more to the optimization story here, but I guess these are the primary motivations for the VM over just better code generation or implementing a parser in non-Go?
- johnisgood 1y ago
  I keep hearing that Go's C FFI is slow, why is that? How much slower is it in comparison to other languages?
  - malkia 1y ago
    I've asked ChatGPT to summarize (granted my prompt might not be ideal), but some points to note, here just first in details others in the link at the bottom: Calling C from Go (or vice versa) often requires switching from Go's lightweight goroutine model to a full OS thread model because: - Go's scheduler manages goroutines on M:N threads, but C doesn't cooperate with Go's scheduler. - If C code blocks (e.g., on I/O or mutex), Go must assume the worst and parks the thread, spawning another to keep Go alive. * Cost: This means entering/exiting cgo is significantly more expensive than a normal Go call. There’s a syscall-like overhead. ... This was only the first issue, but then it follows with "Go runtime can't see inside C to know is it allocating, blocking, spinning, etc.", then "Stack switching", "Thread Affinity and TLS", "Debug/Profiling support overhead", "Memory Ownership and GC barriers" All here - https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d08d https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d0...
    - johnisgood 1y ago
      Just to roll with your way: https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e1674b https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e167... Please do not take the numbers below at face value. I still expect an actual reply to my initial comment. Per-call overhead: C (baseline) - ~30 ns Rust (unsafe) - ~30 ns C# (P/Invoke) - ~30-50 ns LuaJIT - ~30-50 ns Go (cgo) - ~40-60 ns Java (22, FFM) - ~40-70 ns Java (JNI) - ~300-1000 ns Perl (XS) - ~500-1000 ns Python (ctypes) - ~10,000-30,000 ns Common Lisp (SBCL) - ~500-1500 ns Seems like Go is still fast enough as opposed to other programming languages with GC, so I am not sure it is fair to Go.
jeffrallen 1y ago
> hyperpb is a brand new library, written in the most cursed Go imaginable This made me LOL.
dumah 1y ago
Fantastic post. Please do one on your analysis and optimization workflow and tooling!
dang 1y ago
Related ongoing thread: Hyperpb: Faster dynamic Protobuf parsing - https://news.ycombinator.com/item?id=44661785 https://news.ycombinator.com/item?id=44661785
ryukoposting 1y ago
> Every type contributes to a cost on the instruction cache, meaning that if your program parses a lot of different types, it will essentially flush your instruction cache any time you enter a parser. Worse still, if a parse involves enough types, the parser itself will hit instruction decoding throughput issues. Interesting. This makes me wonder how nanopb would benchmark against the parsers shown in the graphs. nanopb's whole schtick is that it's pure C99 and it doesn't generate separate parsing functions for each message. nanopb is what a lot of embedded code uses, due to the small footprint.
tschellenbach 1y ago
last time i benchmarked msgpack and protobuf against each other it was near flat for my use case. JSON was 2-3x slower, but msgpack and protobuf were near equal. might be different now after this release, exciting :)
- Analemma_ 1y ago
  Keep in mind that some of the performance bottlenecks the author is talking about and optimizing for show up in large-scale uses and possibly not in benchmarks. In particular, the "parsing too many different types blows away your instruction cache" issue will only show up if you are actually parsing lots of types, otherwise UPB is not necessary.
ptspts 1y ago
Where are the benchmarks comparing hyperpb to other proto parsers?
cyberax 1y ago
I would love to benchmark it against the old gogo protobuf. One thing that really slows down PB is its insistence on using pointers everywhere. Gogo protobuf used value types where possible, significantly reducing the GC pressure. It feels like arenas are just an attempt to bring that back.
dgan 1y ago
maybe if performance is really an issue, protobufs shouldn't be used ? there is also flatbuffers, capnp and purely C++ serializers: zpp, yas... but protobufs are definitely convenient !
- philwelch 1y ago
  If performance isn’t an issue, why not JSON?
  - foobiekr 1y ago
    performance is always an issue when the alternative is 2+ orders of magnitude slower.
  - 9rx 1y ago
    Protobuf tooling isn't great, but JSON tooling is far worse. Developer experience matters too. (Technically protobuf supports JSON encoding, but its additional work to enable, again diminishing the developer experience)
- 9rx 1y ago
  If execution performance is really that much of an issue you're not going to choose Go anyway, rendering this project moot. What is a real issue is marketing performance. As it turns out, everyone loves a story about something going fast. So it pays to create these kinds of optimizations for the sake of the story, used to sell the product to the developer who makes choices based on what they think feels good.
nemo1618 1y ago
There are two ways to look at this. First is that, if the parsing library for your codec includes a compiler, VM, and PGO, your codec must be extremely cursed and you should take a step back and think about your life. Second is that, if the parsing library for your codec includes a compiler, VM, and PGO, your codec must be wildly popular and adds enormous value.
- VikingCoder 1y ago
  If you want to do something several hundreds of billions of times per day, you probably want to do it very efficiently.
  - benlivengood 1y ago
    Or, you know, several hundreds of billions of times per second...
- pantalaimon 1y ago
  You can also prase Protobuf on the very low end, if you don't need super high throughput https://jpa.kapsi.fi/nanopb/ https://jpa.kapsi.fi/nanopb/
- cryptonector 1y ago
  I'm not sure which part you're objecting to. If it's compilation at run-time, then I agree: it should be done at build time. But in this case it's really not a big deal. If you're objecting to needing a compiler, then... you're not even wrong.
AceJohnny2 1y ago
Offtopic, but is anyone using CapnProto, the ProtoBuf former maintainer's (kentonv around here) subsequent project? https://capnproto.org https://capnproto.org If so, how does it compare in practice? (what does Cloudflare Workers use?)
- necubi 1y ago
  Cloudflare is almost entirely run on Cap'n Proto, including the entire workers platform
  - kentonv 1y ago
    The Workers platform uses Cap'n Proto extensively, as one might expect (with me being the author of Cap'n Proto and the lead dev on Workers). Some other parts of Cloudflare use it (including the logging pipeline, which used it before I even joined), but there are also many services using gRPC or just JSON. Each team makes their own decisions. I have it on my TODO list to write a blog post about Workers' use of Cap'n Proto. By far the biggest wins for us are from the RPC system -- the serialization honestly doesn't matter so much. That said, the ecosystem around Cap'n Proto is obviously lacking compared to protobuf. For the Cloudflare Workers Runtime team specifically, the fact that we own it and can make any changes we need to balances this out. But I'm hesitant to recommend it to people at other companies, unless you are eager to jump into the source code whenever there's a problem.
    - anonymoushn 1y ago
      A while ago I talked to some team that was planning a migration to GraphQL, long after this was generally thought to be a bad idea. The lead seemed really attached to the "composable RPCs" aspect of the thing, and at the time it seemed like nothing else offered this. It would be quite cool if capnproto became a more credible option for this sort of situation. At the time users could read about the rpc composition/promise passing/"negative latency" stuff, but it was not quite implemented.
    - motorest 1y ago
      Thank you for posting here. Always insightful, always a treat.
spenczar5 1y ago
JITting bytecode VMs really seem fantastic at this. A similar trick works in Avro in Python (https://journal.spencerwnelson.com/entries/avro.html https://journal.spencerwnelson.com/entries/avro.html). I suppose the insight is that schema documents are programs, just in a weird language, and compiled programs are fast.
nikolayasdf123 1y ago
yeah, wish Go binary encoding/decoding was faster and leaner, maybe even with zero memory copy
crabbone 1y ago
Not enough programmers talk to QA, and so are oblivious to testing. The benchmarking here is... pointless. Here's my story (of writing a Protobuf parser) to illustrate why. I already wrote about it before, but it keeps happening, so, maybe repeating it is not such a bad thing. So... the official (coming from Google) Protobuf parser for Python is ridiculously bad. It generates Python code for the parser. Not only is this unnecessary, it's also executed poorly. So, I decided to roll my own. Also, I decided to make it multi-threaded (I wrote it in C, with Python bindings). Then I tried to benchmark it against the official implementation. I don't trust the quality of Python language implementation, so, that came under the spotlight first. For example, I chose Python's enumerators to represent Protobuf enumerators. Turned out that was a horrible decision because instantiation of Python's enumerators is extremely resource-intensive. I had to make a few similar adjustments, searching for the less resource-hungry Python built-in data-structures. And yet I was still far behind on my benchmarks. So, I tried to understand what the Google's C++ code was doing. And then I realized that Google's parser in the "first pass" only extracted the top-level messages, not even trying to parse the hierarchical structure. This structure was only extracted on-demand. So, essentially, I was competing against memcpy()... But, here's another catch: most real-world applications will not want the "intermediate" Protobuf structure made up of general-purpose data-structures such as dictionaries or enumerators. They will want them immediately parsed into domain-specific data-structures. So, a parser that offers this kind of functionality might be slower to parse into generic data-structures, but will win in real-world benchmarks. Of course, the percentage of useful payload is also important. Typically, applications underutilize the messages they exchange. But, to what degree is different and completely dependent on application design. Similarly, mapping into domain-specific objects will depend on the sort of information the application wants to exchange. Protobuf is bad for column / tabular data, but somewhat better for object graph kind of data. Protobuf is bad for large messages with duplicated content, but is somewhat better for short messages with little duplication. All of this, and much more, allows for a lot of variability in benchmark design, and would allow me, or anyone writing a parser to skew the numbers in whatever direction I want... to amuse the general public.
karel-3d 1y ago
OK reading this, I have imposter syndrome again.
cryptonector 1y ago
Does hyperb compile PB types into something like a byte-compiled or AST description of them that it then interprets at run-time? EDIT: Yes https://github.com/bufbuild/hyperpb-go/blob/main/internal/tdp/vm/vm.go#L17 https://github.com/bufbuild/hyperpb-go/blob/main/internal/td... This is very cool. The ASN.1 compiler I [sort of] maintain has an option to generate C code or a "template", and the "template" is just an AST that gets interpreted at run-time. I did not come up with that idea, but the people who did did it because it's faster to do the AST interpretation thing than to generate lots of object code -- the reason is that the templates and the template interpreter are smaller than the object code for the alternative, and cache effects add up.
adsharma 1y ago
I see some discussion here about protobufs being widely used and preventing innovation because it's good enough and comes from Google. I see a second effect that's probably more regressive: using protobufs (designed for RPC) as a database schema definition language or a disk data structure description language. Much prefer something like typespec to describe types.And then derive the rpc and disk schema languages from that programatically. My first attempt was to use the flatbuffer schema language as a starting point. But being attached to a serialization format can be a net negative. Did not find traction. Motivation: can not stand field numbers in a database schema when other migration mechanisms exist.

248 points by ibobev 1y ago

See also https://buf.build/blog/hyperpb https://buf.build/blog/hyperpb (via https://news.ycombinator.com/item?id=44661785 https://news.ycombinator.com/item?id=44661785)

UncleEntity 1y ago

> In other words, a UPB parser is actually configuration for an interpreter VM, which executes Protobuf messages as its bytecode. This is kind of confusing, the VM is runtime crafted to parse a single protobuf message type and only this message type? The Second Futamura Projection, I suppose... Or the VM is designed specifically around generic protobuf messages and it can parse any random message but only if it's a protobuf message? I've been working on the design of a similar system but for general binary parsing (think bison/yacc for binary data) and hadn't even considered doing data over specialized VM vs. bytecode+data over general VM. Honestly, since it's designed around 'maximum laziness' (it just parses/verifies and creates metadata over the input so you only pay for decoding bytes you actually use) and I/O overhead is way greater than the VM dispatching trying this out is probably one of those "premature optimization is the root of all evil" cases but intriguing none the less.

tonyarkles 1y ago
Based on what I know about the structure of protobufs internally and without having looked deep into what UPB is doing... I'd guess it could probably be a stack machine that treats (byte)+ as opcodes. Most of the time I'd think of it as parser -> AST -> bytecode, but I think the "grammar" of protobufs would allow your parser to essentially emit terminals as they're parsed straight to the VM as instructions to execute.
- UncleEntity 1y ago
  In the couple days since I posted my confusion (threads merged or something) I consulted the daffy robots and figured out how it all works. Also had them come up with a design document for "a specialized compiler and virtual machine architecture for parsing Protocol Buffer messages that achieves significant performance improvements through a novel compilation pipeline combining protobuf-specific AST optimization, continuation-passing style transformations, and tail call interpreter execution." Interesting times we live in...
haberman 1y ago
I think I can shed some light on this, as the creator and lead of upb. Calling a Protobuf Parser an "interpreter VM" is a little bit of rhetorical flourish. It comes from the observation that there are some deep structural similarities between the two, which I first observed in an article a few years back: https://blog.reverberate.org/2021/04/21/musttail-efficient-interpreters.html https://blog.reverberate.org/2021/04/21/musttail-efficient-i... > It may seem odd to compare interpreter loops to protobuf parsers, but the nature of the protobuf wire format makes them more similar than you might expect. The protobuf wire format is a series of tag/value pairs, where the tag contains a field number and wire type. This tag acts similarly to an interpreter opcode: it tells us what operation we need to perform to parse this field’s data. Like interpreter opcodes, protobuf field numbers can come in any order, so we have to be prepared to dispatch to any part of the code at any time. This means that the overall structure of a protobuf parser is conceptually a while() loop surrounding a switch() statement, just like a VM interpreter. The tricky part is that the set of "case" labels for a Protobuf parser is message-specific and defined by the fields in the schema. How do we accommodate that? The traditional answer was to generate a function per message and use the schema's field numbers as the case labels. You can see an example of that here (in C++): https://github.com/protocolbuffers/protobuf/blob/f763a2a86084371fd0da95f3eeb879c2ff26b06d/src/google/protobuf/descriptor.pb.cc#L2175-L2227 https://github.com/protocolbuffers/protobuf/blob/f763a2a8608... More recently, we've moved towards making Protobuf parsing more data-driven, where each field's schema is compiled into data that is passed as an argument to a generic Protobuf parser function. We call this "table-driven parsing", and from my read of the blog article, I believe this is what Miguel is doing with hyperpb. The trick then becomes how to make this table-driven dispatch as fast as possible, to simulate what the switch() statement would have done. That question is what I cover at length in the article mentioned above.
- anonymoushn 1y ago
  Really great. I wonder, for the "types encoded as code" approach, is there any benefit to fast paths for data with fields in ascending order? For some json parsers with types encoded as code I have observed some speedup from either hard-coding a known key order or assuming keys in some order and providing a fallback in case an unexpected key is encountered. For users who are stuck with protobuf forever because of various services using it and various data being encoded this way, the historical data could plausibly be canonicalized and written back in large chunks when it is accessed, so that one need not pay the entire cost of canonicalizing it all at once. But of course the icache concerns are still just as bad.

mdhb 1y ago

I’d really love to see more work bringing the best parts of protobuf to a standardised serialization format like CBOR. I’d make the same argument for gRPC-web to something like WHATWG streams and or WebTransport. There is a lot of really cool and important learnings in both but it’s also so tied up in weird tooling and assumptions. Let’s rebase on IETF and W3C standards

youngtaff 1y ago
Would be good to see support for encoding / decoding CBOR exposed as a broswer API - they currently use CBOR internally for WebAuthn so I’d hope it’s bnot too hard
cyberax 1y ago
You can easily do this. Protobuf supports pluggable writers, and iterating over a schema is pretty easy. We do it for the JSONB. I'm not sure the purpose, though. Protobuf is great for its inflexible schema, and CBOR is great for its flexible data representation. A separate CBOR schema would be a better fit, there's CDDL but it has no traction.

irq-1 1y ago

https://github.com/bufbuild/hyperpb-go https://github.com/bufbuild/hyperpb-go

skybrian 1y ago

This is excellent: an in-depth description showing how the Go internals make writing fast interpreters difficult, by someone who is far more determined than I ever was to make it fast anyway. I’ve assumed that writing fast interpreters wasn’t a use case the Go team cared much about, but if it makes protobuf parsing faster, maybe it will get some attention, and some of these low-level tricks will no longer be necessary?

RGBCube 1y ago
> and some of these low-level tricks will no longer be necessary? Don't count on it. This language is Golang.
- anonymoushn 1y ago
  Hi to my favorite turkish teenager. You really don't get to generate fast codecs from normal-looking code in any language :(
  - actionfromafar 1y ago
    In as-far-as wuffs looks normal, wuffs?
    - anonymoushn 1y ago
      wuffs is extremely cool, but you do find the expected blobs of intrinsics for specific simd extensions in this code.

alexozer 1y ago

So am I identifying the bottlenecks that motivate this design correctly? 1. Go FFI is slow 2. Per-proto generated code specialization is slow, because of icache pressure I know there's more to the optimization story here, but I guess these are the primary motivations for the VM over just better code generation or implementing a parser in non-Go?

johnisgood 1y ago
I keep hearing that Go's C FFI is slow, why is that? How much slower is it in comparison to other languages?
- malkia 1y ago
  I've asked ChatGPT to summarize (granted my prompt might not be ideal), but some points to note, here just first in details others in the link at the bottom: Calling C from Go (or vice versa) often requires switching from Go's lightweight goroutine model to a full OS thread model because: - Go's scheduler manages goroutines on M:N threads, but C doesn't cooperate with Go's scheduler. - If C code blocks (e.g., on I/O or mutex), Go must assume the worst and parks the thread, spawning another to keep Go alive. * Cost: This means entering/exiting cgo is significantly more expensive than a normal Go call. There’s a syscall-like overhead. ... This was only the first issue, but then it follows with "Go runtime can't see inside C to know is it allocating, blocking, spinning, etc.", then "Stack switching", "Thread Affinity and TLS", "Debug/Profiling support overhead", "Memory Ownership and GC barriers" All here - https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d08d https://chatgpt.com/share/688172c3-9fa4-800a-9b8f-e1252b57d0...
  - johnisgood 1y ago
    Just to roll with your way: https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e1674b https://chatgpt.com/share/688177c9-ebc0-8011-88cc-9514d8e167... Please do not take the numbers below at face value. I still expect an actual reply to my initial comment. Per-call overhead: C (baseline) - ~30 ns Rust (unsafe) - ~30 ns C# (P/Invoke) - ~30-50 ns LuaJIT - ~30-50 ns Go (cgo) - ~40-60 ns Java (22, FFM) - ~40-70 ns Java (JNI) - ~300-1000 ns Perl (XS) - ~500-1000 ns Python (ctypes) - ~10,000-30,000 ns Common Lisp (SBCL) - ~500-1500 ns Seems like Go is still fast enough as opposed to other programming languages with GC, so I am not sure it is fair to Go.

jeffrallen 1y ago

> hyperpb is a brand new library, written in the most cursed Go imaginable This made me LOL.

dumah 1y ago

Fantastic post. Please do one on your analysis and optimization workflow and tooling!

dang 1y ago

Related ongoing thread: Hyperpb: Faster dynamic Protobuf parsing - https://news.ycombinator.com/item?id=44661785 https://news.ycombinator.com/item?id=44661785

ryukoposting 1y ago

> Every type contributes to a cost on the instruction cache, meaning that if your program parses a lot of different types, it will essentially flush your instruction cache any time you enter a parser. Worse still, if a parse involves enough types, the parser itself will hit instruction decoding throughput issues. Interesting. This makes me wonder how nanopb would benchmark against the parsers shown in the graphs. nanopb's whole schtick is that it's pure C99 and it doesn't generate separate parsing functions for each message. nanopb is what a lot of embedded code uses, due to the small footprint.

tschellenbach 1y ago

last time i benchmarked msgpack and protobuf against each other it was near flat for my use case. JSON was 2-3x slower, but msgpack and protobuf were near equal. might be different now after this release, exciting :)

Analemma_ 1y ago
Keep in mind that some of the performance bottlenecks the author is talking about and optimizing for show up in large-scale uses and possibly not in benchmarks. In particular, the "parsing too many different types blows away your instruction cache" issue will only show up if you are actually parsing lots of types, otherwise UPB is not necessary.

ptspts 1y ago

Where are the benchmarks comparing hyperpb to other proto parsers?

cyberax 1y ago

I would love to benchmark it against the old gogo protobuf. One thing that really slows down PB is its insistence on using pointers everywhere. Gogo protobuf used value types where possible, significantly reducing the GC pressure. It feels like arenas are just an attempt to bring that back.

dgan 1y ago

maybe if performance is really an issue, protobufs shouldn't be used ? there is also flatbuffers, capnp and purely C++ serializers: zpp, yas... but protobufs are definitely convenient !

philwelch 1y ago
If performance isn’t an issue, why not JSON?
- foobiekr 1y ago
  performance is always an issue when the alternative is 2+ orders of magnitude slower.
- 9rx 1y ago
  Protobuf tooling isn't great, but JSON tooling is far worse. Developer experience matters too. (Technically protobuf supports JSON encoding, but its additional work to enable, again diminishing the developer experience)
9rx 1y ago
If execution performance is really that much of an issue you're not going to choose Go anyway, rendering this project moot. What is a real issue is marketing performance. As it turns out, everyone loves a story about something going fast. So it pays to create these kinds of optimizations for the sake of the story, used to sell the product to the developer who makes choices based on what they think feels good.

nemo1618 1y ago

There are two ways to look at this. First is that, if the parsing library for your codec includes a compiler, VM, and PGO, your codec must be extremely cursed and you should take a step back and think about your life. Second is that, if the parsing library for your codec includes a compiler, VM, and PGO, your codec must be wildly popular and adds enormous value.

VikingCoder 1y ago
If you want to do something several hundreds of billions of times per day, you probably want to do it very efficiently.
- benlivengood 1y ago
  Or, you know, several hundreds of billions of times per second...
pantalaimon 1y ago
You can also prase Protobuf on the very low end, if you don't need super high throughput https://jpa.kapsi.fi/nanopb/ https://jpa.kapsi.fi/nanopb/
cryptonector 1y ago
I'm not sure which part you're objecting to. If it's compilation at run-time, then I agree: it should be done at build time. But in this case it's really not a big deal. If you're objecting to needing a compiler, then... you're not even wrong.

AceJohnny2 1y ago

Offtopic, but is anyone using CapnProto, the ProtoBuf former maintainer's (kentonv around here) subsequent project? https://capnproto.org https://capnproto.org If so, how does it compare in practice? (what does Cloudflare Workers use?)

necubi 1y ago
Cloudflare is almost entirely run on Cap'n Proto, including the entire workers platform
- kentonv 1y ago
  The Workers platform uses Cap'n Proto extensively, as one might expect (with me being the author of Cap'n Proto and the lead dev on Workers). Some other parts of Cloudflare use it (including the logging pipeline, which used it before I even joined), but there are also many services using gRPC or just JSON. Each team makes their own decisions. I have it on my TODO list to write a blog post about Workers' use of Cap'n Proto. By far the biggest wins for us are from the RPC system -- the serialization honestly doesn't matter so much. That said, the ecosystem around Cap'n Proto is obviously lacking compared to protobuf. For the Cloudflare Workers Runtime team specifically, the fact that we own it and can make any changes we need to balances this out. But I'm hesitant to recommend it to people at other companies, unless you are eager to jump into the source code whenever there's a problem.
  - anonymoushn 1y ago
    A while ago I talked to some team that was planning a migration to GraphQL, long after this was generally thought to be a bad idea. The lead seemed really attached to the "composable RPCs" aspect of the thing, and at the time it seemed like nothing else offered this. It would be quite cool if capnproto became a more credible option for this sort of situation. At the time users could read about the rpc composition/promise passing/"negative latency" stuff, but it was not quite implemented.
  - motorest 1y ago
    Thank you for posting here. Always insightful, always a treat.

spenczar5 1y ago

JITting bytecode VMs really seem fantastic at this. A similar trick works in Avro in Python (https://journal.spencerwnelson.com/entries/avro.html https://journal.spencerwnelson.com/entries/avro.html). I suppose the insight is that schema documents are programs, just in a weird language, and compiled programs are fast.

nikolayasdf123 1y ago

yeah, wish Go binary encoding/decoding was faster and leaner, maybe even with zero memory copy

crabbone 1y ago

Not enough programmers talk to QA, and so are oblivious to testing. The benchmarking here is... pointless. Here's my story (of writing a Protobuf parser) to illustrate why. I already wrote about it before, but it keeps happening, so, maybe repeating it is not such a bad thing. So... the official (coming from Google) Protobuf parser for Python is ridiculously bad. It generates Python code for the parser. Not only is this unnecessary, it's also executed poorly. So, I decided to roll my own. Also, I decided to make it multi-threaded (I wrote it in C, with Python bindings). Then I tried to benchmark it against the official implementation. I don't trust the quality of Python language implementation, so, that came under the spotlight first. For example, I chose Python's enumerators to represent Protobuf enumerators. Turned out that was a horrible decision because instantiation of Python's enumerators is extremely resource-intensive. I had to make a few similar adjustments, searching for the less resource-hungry Python built-in data-structures. And yet I was still far behind on my benchmarks. So, I tried to understand what the Google's C++ code was doing. And then I realized that Google's parser in the "first pass" only extracted the top-level messages, not even trying to parse the hierarchical structure. This structure was only extracted on-demand. So, essentially, I was competing against memcpy()... But, here's another catch: most real-world applications will not want the "intermediate" Protobuf structure made up of general-purpose data-structures such as dictionaries or enumerators. They will want them immediately parsed into domain-specific data-structures. So, a parser that offers this kind of functionality might be slower to parse into generic data-structures, but will win in real-world benchmarks. Of course, the percentage of useful payload is also important. Typically, applications underutilize the messages they exchange. But, to what degree is different and completely dependent on application design. Similarly, mapping into domain-specific objects will depend on the sort of information the application wants to exchange. Protobuf is bad for column / tabular data, but somewhat better for object graph kind of data. Protobuf is bad for large messages with duplicated content, but is somewhat better for short messages with little duplication. All of this, and much more, allows for a lot of variability in benchmark design, and would allow me, or anyone writing a parser to skew the numbers in whatever direction I want... to amuse the general public.

karel-3d 1y ago

OK reading this, I have imposter syndrome again.

cryptonector 1y ago

Does hyperb compile PB types into something like a byte-compiled or AST description of them that it then interprets at run-time? EDIT: Yes https://github.com/bufbuild/hyperpb-go/blob/main/internal/tdp/vm/vm.go#L17 https://github.com/bufbuild/hyperpb-go/blob/main/internal/td... This is very cool. The ASN.1 compiler I [sort of] maintain has an option to generate C code or a "template", and the "template" is just an AST that gets interpreted at run-time. I did not come up with that idea, but the people who did did it because it's faster to do the AST interpretation thing than to generate lots of object code -- the reason is that the templates and the template interpreter are smaller than the object code for the alternative, and cache effects add up.

adsharma 1y ago

I see some discussion here about protobufs being widely used and preventing innovation because it's good enough and comes from Google. I see a second effect that's probably more regressive: using protobufs (designed for RPC) as a database schema definition language or a disk data structure description language. Much prefer something like typespec to describe types.And then derive the rpc and disk schema languages from that programatically. My first attempt was to use the flatbuffer schema language as a starting point. But being attached to a serialization format can be a net negative. Did not find traction. Motivation: can not stand field numbers in a database schema when other migration mechanisms exist.