3 ms·
It's possible to encode a protobuf as JSON and we do it all the time at Google. In browsers, native JSON parsing is very fast and the data is compressed, so goi
by skybrian 11y ago
It's possible to encode a protobuf as JSON and we do it all the time at Google. In browsers, native JSON parsing is very fast and the data is compressed, so going to a binary format doesn't seem worthwhile. The .proto file is used basically as an IDL from which we generate code.
- teh 11y agoCan I ask which library you are using? I found a few [1] but none seem super robust. Also, how do you deal with the bytes type? [1] https://code.google.com/p/protobuf-json/ https://code.google.com/p/protobuf-json/ https://github.com/benhodgson/protobuf-to-dict https://github.com/benhodgson/protobuf-to-dict
- philsnow 11y agoJust a guess here: I would have an agreed-upon key name suffix like "__b64_enc" or something. Serializers take a field "foo" of type bytes and serialize it as "foo__b64_enc": b64(value), and deserializers strip and base64-decode. edit: it's exactly what you would do if you wanted to pass any binary data as json over the wire, regardless of whether you're using protobufs. you'd just get it "for free" (meaning you wouldn't have to write the boilerplate, not that you don't have to en/decode).
- skybrian 11y agoThere's no standard for this and it's mostly not open source as far as I know. The overall approach is called "JSPB" but there are various flavors. A typical use case is for a web app that has its own private RPC to its own servers, so interop isn't an issue. Also, web apps generally don't need or want to work with binary data, so better not to send it. I recently became the maintainer of the Dart protobuf library which supports both JSON and binary format [1], [2]. However, the JSON format isn't necessarily compatible with other protobuf libraries you've seen. [1] https://github.com/dart-lang/dart-protobuf https://github.com/dart-lang/dart-protobuf [2] https://github.com/dart-lang/dart-protoc-plugin https://github.com/dart-lang/dart-protoc-plugin
- cletus 11y agoPersonally I've found JSON encoded protobufs to be almost universally awful. The most common method is to use an array indexed by the field number. I've seen protobufs with hundreds of fields so that's hundreds of nulls as the string "null". The alternative is to have JSON objects with attributes named after the protobufs field name. This isn't without warts either and seems to be less prevelant in my experience. Another problem is JavaScript doesn't support all the data types you can get in protobufs, most notably int64s. Protobufs are relatively space efficient (eg variable width int types). JSON encoded protobufs much less so. Perhaps the rise of browser support for raw binary data will make this less awful. Many consider it a virtue to use the same code on the client and server. It explains things like this and GWT. Personally opi think this is horribly misguided and a fools errand. You want to decouple your client and server as much as possible (IMHO). Disclaimer: I work for Google
- haberman 11y ago> The most common method is to use an array indexed by the field number. I've seen protobufs with hundreds of fields so that's hundreds of nulls as the string "null". What you are describing here is known as the "JSPB" wire format. This is a serialization that is only ever used for JavaScript, and only used there because, historically, parsing binary protobufs in JavaScript was too slow. With TypedArray and other JavaScript enhancements, this is changing. Ideally, JSPB wire format would be phased out completely. > The alternative is to have JSON objects with attributes named after the protobufs field name. This isn't without warts either and seems to be less prevelant in my experience. It's about to become a lot more prevalent with proto3, which features first-class JSON support. See: https://developers.google.com/protocol-buffers/docs/proto3#json https://developers.google.com/protocol-buffers/docs/proto3#j... Disclaimer: I work on the protobuf team.
- skybrian 11y agoproto3's JSON is an improvement on ascii protobufs, but since it uses field names, it doesn't have the same backward compatibility guarantees as a format that uses tag numbers. It would be nice if we had a standardized JSPB wire format that used tag numbers, rather than the various unofficial implementations we have now.