16 ms·
A practical guide to protocol buffers in Go
- pattyshack101 12y agoThe current golang protobuf api / implementation is unsuitable for large scale system. See https://groups.google.com/forum/#!topic/Golang-nuts/vxJNXdiMhrc https://groups.google.com/forum/#!topic/Golang-nuts/vxJNXdiM... for discussion (you have to scroll down the thread a bit).
- crawshaw 12y agoThat thread points out a lot of problems with both the current implementation of protos in Go and the inherent difficulties mapping of a code generator that was designed for C++ onto Go. I have had to deal with those issues and more using protos in Go. However, it does not make them unsuitable. We have many large scale systems at Google written in Go using this proto library. Some are high QPS, some use large protos, some have complex definitions. Each requires care and thought, but they work.
- minaandrawos 12y agoWhenever you get a chance, would you care to share some of the main pain points to look out for from your experience as well as the workarounds? I haven't encountered major issues (small issues yes, major issues not yet) so far implementing protos , however my projects scales are typically not as massive as Google's. So for the sake of my quest to expand my horizons, any feedback will be greatly appreciated
- crawshaw 12y agoThe most important piece of advice is not proto specific: use pprof. It may sound obvious, but even though I have been using it for years, I find myself still not running it enough. Proto decoding specific: if you're parsing a message with lots of fields, you can define a new message with just the fields you care about. I.e. if you're parsing message M { optional int32 F1 = 1; // ... optional int32 F100 = 100; } but you only care about F1 and F2 in this message, define message SmallM { optional int32 F1 = 1; optional int32 F2 = 2; } If you use SmallM to decode your messages, it will skip a lot of work filling out fields F3-F100, saving you a lot of CPU.
- oldmanjay 12y agoLots of double exclamation points in there. I hate that I have a hard time taking it seriously because of such a trivial detail, but I do.
- zabcik 12y agoYeah, it made me view the whole article as amateur and stopped reading.
- minaandrawos 12y agoInteresting feedback, I didn't expect the exclamation marks to have such an affect. It is more of my personal writing style, however since there were two remarks on the exclamation marks, I edited the article and cleaned out some of the exclamation marks. Hope you enjoy the article.
- oldmanjay 12y agoI do find it an interesting article, I don't want you to think I'm entirely negative. I just kept stumbling at the punctuation. It's more about me than you.
- joshbaptiste 12y agoIt's a blog, not a peer reviewed ACM article. I read and learn from others no matter how they decide to express themselves as long as the content is relevant and helpful at solving current or future issues.
- minaandrawos 12y agoThank you, I appreciate the comment
- jalfresi 12y agoThoroughly interesting with just the right amount of detail to get off and experimenting on my own projects. Nice one!