5 ms·

Sure. You would use gRPC when: - You want to have inter-service RPC. - You want not only unary calls, but also bidirectional streaming. - You want

by q3k 6y ago

Sure. You would use gRPC when: - You want to have inter-service RPC. - You want not only unary calls, but also bidirectional streaming. - You want a well-defined schema for your RPC data and methods. - You want a cross-language solution that guarantees interoperability (no more JSON parsing differences! [1]) [1] - http://seriot.ch/parsing_json.php http://seriot.ch/parsing_json.php

wnevets 6y ago
>Parsing JSON is a Minefield and there are people who complain it's too simple, i.e no comments
- q3k 6y ago
  It's one of those 'simple at first glace' standards. If you want to confuse someone who advocates for JSON's 'simplicity' as a feature, ask them what happens when their favorite JSON deserializer receives repeated dictionary keys (yes, that's valid JSON per RFC 7159).
  - deckard1 6y ago
    there is no standard so simple that humans won't make a mess of it. You think "CSV, comma-separated-values, it's right in the name!" And then you write a parser for it and realize it won't read CSV files Excel generates. Years ago I was porting an r5rs Scheme app from Guile to other Scheme systems. The entire standard is 50 pages of fairly readable basic English. You would not believe the amount of differences different interpreter/compilers have on their agreement as to what a symbol can be. Or under-specified cases such as integer->char (one system goes out of its way to be a dick about it and purposely not use ASCII, trading pragmatics for pedantry)
  - jeffbee 6y ago
    % echo '{"a": 5, "a": 42}' | jq { "a": 42 } Pretty much what I expected and also what you would get if you saw two instances of the same optional non-repeated field in a protocol buffer message. What were you expecting or wanting?
    - Groxx 6y ago
      Their point was likely that implementations vary on the interpretation. Which is a bit of a problem for rpc systems.
    - q3k 6y ago
      Here's the kicker: while this might sensible to you, some JSON implementations out there will reject duplicate fields (fail a parse completely), and the RFC does not even specify what is the correct behavior (override previous, ignore duplicates, fail entire parse, return non-unique keyed dictionary, something else entirely?). So while to you and I this behavior might be expected (although I'm still not sure that overriding previous fields is more obvious than ignoring repeated fields) - some library implements thought differently, and there isn't even an agreed on standard. Arguing about this isn't purely academic, either - there have been security vulnerabilities resulting from these differences [1]. [1] - https://justi.cz/security/2017/11/14/couchdb-rce-npm.html https://justi.cz/security/2017/11/14/couchdb-rce-npm.html
      - jeffbee 6y ago
        Interesting. Protobuf specifies this last-instance-wins behavior, and it can be pretty useful. It allows you to override a field by simply appending a few bytes, without having to re-encode a whole message. JSON I guess doesn't have as much concern for efficiency as protobuf has.
jwalton 6y ago
In other words, when you want to use ASN.1[0], but you don't know what ASN.1 is so you use something the overly complicated version Google made instead. ;) [0](https://en.wikipedia.org/wiki/Asn.1 https://en.wikipedia.org/wiki/Asn.1)
- sagarm 6y ago
  This seems like a very high level spec, not a library or toolkit. There's also zero mention of practical concerns like protocol evolution, a canonical wire encoding etc.
  - jwalton 6y ago
    ASN.1 is indeed a spec. You'd need to find a library to use it, and there are more in C than in whatever fancy modern language you're probably using. But, there are lots to choose from. There's multiple canonical wire encodings. XER is the "XML encoding rules" if you want something human readable. "BER" is the binary format, although it has some ambiguities so there's "DER", the Distinguished Encoding Rules, which resolves a lot of that by using essentially a subset of BER and specifying how you should behave in various corner cases. In practice, you want to write your messages as DER, but accept messages from other parties using the full BER. It's an older standard, but it's used heavily in telecom. To pick an example, if you make a cell phone call, especially out in the sticks somewhere, there's probably going to be a media gateway controller that figures out how to route your call without decompressing and recompressing it a bunch of times, and it talks to the various devices routing your call over H.248, which is specified entirely in ASN.1.
- wahern 6y ago
  To be fair, in practice people don't choose gRPC as a protocol and serialization standard so much as they choose preexisting gRPC libraries and code generators. Open source ASN.1 tooling sucks while Google maintains gRPC tooling for a great many languages. This is why gRPC, Thrift, etc have so much more mindshare than ASN.1 in the open source community. The only good open source ASN.1 (DER, PER, etc) code generator for C is asn1c. I believe Java has a couple of good libraries. But other than that ASN.1 tooling is a horrible train wreck of broken or abandoned projects that don't provide adequate ASN.1 tooling. Many people rely on OpenSSL's DER library, but its far too low-level. AFAICT, the same is true for Python and other high-level languages--no open source projects where can you pass an ASN.1 specification to generate (at compile-time or run-time) a full serializer and deserializer. There are plenty of commercial, proprietary ASN.1 tooling solutions out there. Presumably it's why ASN.1 has persisted as long as it has in industry. Even Fabrice Bellard sells a commercial solution: https://bellard.org/ffasn1/ https://bellard.org/ffasn1/ When you have access to good tooling ASN.1 is arguably superior to the open source alternatives as there aren't as many broken corner cases that can cause interoperability problems. We only have ourselves to blame. If I ever have time I want to write an ASN.1 spec parser using LPeg that can generate LPeg-based DER parsers. I already have a fairly comprehensive LPeg-based DER parser for PKIX messages, but generating that from an ASN.1 spec is a significant step up in conceptual complexity. While I've written more than my fair share of parsers before, I've never written a proper parser generator, so it's a steeper hill to climb for me even though I already have more ASN.1 experience than most.

by q3k 6y ago

wnevets 6y ago

>Parsing JSON is a Minefield and there are people who complain it's too simple, i.e no comments

q3k 6y ago
It's one of those 'simple at first glace' standards. If you want to confuse someone who advocates for JSON's 'simplicity' as a feature, ask them what happens when their favorite JSON deserializer receives repeated dictionary keys (yes, that's valid JSON per RFC 7159).
- deckard1 6y ago
  there is no standard so simple that humans won't make a mess of it. You think "CSV, comma-separated-values, it's right in the name!" And then you write a parser for it and realize it won't read CSV files Excel generates. Years ago I was porting an r5rs Scheme app from Guile to other Scheme systems. The entire standard is 50 pages of fairly readable basic English. You would not believe the amount of differences different interpreter/compilers have on their agreement as to what a symbol can be. Or under-specified cases such as integer->char (one system goes out of its way to be a dick about it and purposely not use ASCII, trading pragmatics for pedantry)
- jeffbee 6y ago
  % echo '{"a": 5, "a": 42}' | jq { "a": 42 } Pretty much what I expected and also what you would get if you saw two instances of the same optional non-repeated field in a protocol buffer message. What were you expecting or wanting?
  - Groxx 6y ago
    Their point was likely that implementations vary on the interpretation. Which is a bit of a problem for rpc systems.
  - q3k 6y ago
    Here's the kicker: while this might sensible to you, some JSON implementations out there will reject duplicate fields (fail a parse completely), and the RFC does not even specify what is the correct behavior (override previous, ignore duplicates, fail entire parse, return non-unique keyed dictionary, something else entirely?). So while to you and I this behavior might be expected (although I'm still not sure that overriding previous fields is more obvious than ignoring repeated fields) - some library implements thought differently, and there isn't even an agreed on standard. Arguing about this isn't purely academic, either - there have been security vulnerabilities resulting from these differences [1]. [1] - https://justi.cz/security/2017/11/14/couchdb-rce-npm.html https://justi.cz/security/2017/11/14/couchdb-rce-npm.html
    - jeffbee 6y ago
      Interesting. Protobuf specifies this last-instance-wins behavior, and it can be pretty useful. It allows you to override a field by simply appending a few bytes, without having to re-encode a whole message. JSON I guess doesn't have as much concern for efficiency as protobuf has.

jwalton 6y ago

In other words, when you want to use ASN.1[0], but you don't know what ASN.1 is so you use something the overly complicated version Google made instead. ;) [0](https://en.wikipedia.org/wiki/Asn.1 https://en.wikipedia.org/wiki/Asn.1)

sagarm 6y ago
This seems like a very high level spec, not a library or toolkit. There's also zero mention of practical concerns like protocol evolution, a canonical wire encoding etc.
- jwalton 6y ago
  ASN.1 is indeed a spec. You'd need to find a library to use it, and there are more in C than in whatever fancy modern language you're probably using. But, there are lots to choose from. There's multiple canonical wire encodings. XER is the "XML encoding rules" if you want something human readable. "BER" is the binary format, although it has some ambiguities so there's "DER", the Distinguished Encoding Rules, which resolves a lot of that by using essentially a subset of BER and specifying how you should behave in various corner cases. In practice, you want to write your messages as DER, but accept messages from other parties using the full BER. It's an older standard, but it's used heavily in telecom. To pick an example, if you make a cell phone call, especially out in the sticks somewhere, there's probably going to be a media gateway controller that figures out how to route your call without decompressing and recompressing it a bunch of times, and it talks to the various devices routing your call over H.248, which is specified entirely in ASN.1.
wahern 6y ago
To be fair, in practice people don't choose gRPC as a protocol and serialization standard so much as they choose preexisting gRPC libraries and code generators. Open source ASN.1 tooling sucks while Google maintains gRPC tooling for a great many languages. This is why gRPC, Thrift, etc have so much more mindshare than ASN.1 in the open source community. The only good open source ASN.1 (DER, PER, etc) code generator for C is asn1c. I believe Java has a couple of good libraries. But other than that ASN.1 tooling is a horrible train wreck of broken or abandoned projects that don't provide adequate ASN.1 tooling. Many people rely on OpenSSL's DER library, but its far too low-level. AFAICT, the same is true for Python and other high-level languages--no open source projects where can you pass an ASN.1 specification to generate (at compile-time or run-time) a full serializer and deserializer. There are plenty of commercial, proprietary ASN.1 tooling solutions out there. Presumably it's why ASN.1 has persisted as long as it has in industry. Even Fabrice Bellard sells a commercial solution: https://bellard.org/ffasn1/ https://bellard.org/ffasn1/ When you have access to good tooling ASN.1 is arguably superior to the open source alternatives as there aren't as many broken corner cases that can cause interoperability problems. We only have ourselves to blame. If I ever have time I want to write an ASN.1 spec parser using LPeg that can generate LPeg-based DER parsers. I already have a fairly comprehensive LPeg-based DER parser for PKIX messages, but generating that from an ASN.1 spec is a significant step up in conceptual complexity. While I've written more than my fair share of parsers before, I've never written a proper parser generator, so it's a steeper hill to climb for me even though I already have more ASN.1 experience than most.