3 ms·
Yeah, that's some weird editorializing on behalf of the author. The "flaw" is mentioned only in the headline, the text just explains how the format works. It's
by newaccount74 3y ago
Yeah, that's some weird editorializing on behalf of the author. The "flaw" is mentioned only in the headline, the text just explains how the format works. It's not a flaw, just working as designed.
There's also this bit:
> Google makes it computationally expensive to decode, alter, and re-encode without the C++ source proto files
Yeah, it's computationally expensive if you use unoptimized Python code to do it. If you write your code in C (or another compiled language) then scanning 1.8MB of protobuf code should be trivial, with or without the proto source files.
I'm pretty sure that making Protobuf files hard to decode without sources is not a design goal. If it was, they did a pretty lousy job.
- kccqzy 3y agoExactly. It's not exactly computationally expensive if you can do it at 2GB/s! https://news.ycombinator.com/item?id=26931581 https://news.ycombinator.com/item?id=26931581
- fsckboy 3y agoit's computationally expensive if it's more computation than all the other things you'd like your hardware to do, i think is a better definition.
- newaccount74 3y agoNo, that's not the issue. The author seemed to have parsed the protobuf data by converting it all to a huge hierarchy of thousands (millions?) of Python objects. Of course that is slow and takes a few seconds. That's so many allocations! But if you are just looking for a specific tag, there's no need for that, I'm pretty sure you can parse protobuf in place without allocations, which would be a million times faster. You could maybe even do it in Python with reasonable speed.
- MayeulC 3y agoHeck, you may be able to do it with eBPF or some other low-level hardware-accelerated packet filter.