5 ms·

I've had the great misfortune to be writing code against an XML-RPC service recently. It's really a miserable design, even compared to contemporaries like Sun R

I've had the great misfortune to be writing code against an XML-RPC service recently. It's really a miserable design, even compared to contemporaries like Sun RPC. * Incredibly verbose. Even a trivial array value like [1, 2, 3] encodes to over a hundred bytes. I'm convinced the author was not aware of attributes, or for some reason held a grudge against them. * There's only like five primitive types, so implementations that want to transmit wild and unusual values such as 64-bit integers need to define their own ad-hoc extensions. * It's written in XML but there's no namespace, and implementations may or may not use namespaces for their extensions. Is an int64 represented as `i8` or `{http://ws.apache.org/xmlrpc/namespaces/extensions}i8 http://ws.apache.org/xmlrpc/namespaces/extensions}i8`? Depends on which library you're talking to! * There's a native "date/time" type, but the syntax is unspecified other than being ISO8601-ish. Don't even get me started on timezones, which will probably be in the local time of the server, but might be in UTC or BST or who knows what. * The XML-RPC spec claims that <string> elements can be used to transmit any character except '<' and '&', including binary data, including NUL. I want you to imagine what that looks like on the decoding end. Your XML parser is just humming along, decoding UTF-8 and lexing some tags, and all the sudden it comes across a big blob of fucking binary data in the middle of the document. Why was this allowed when the spec also defines a <base64> element?? The XML-RPC spec ought to come with a Surgeon General's warning that reading it might give you brain worms.

masklinn 4y ago
> I'm convinced the author was not aware of attributes, or for some reason held a grudge against them. That is completely unsurprising, attributes suck ass for data, what would XML-RPC even have used them for? Replacing <int> by <integer width=“32”>? Other serialisation formats make the same choice (e.g. plists) because it’s so much simpler. > There's only like five primitive types, so implementations that want to transmit wild and unusual values such as 64-bit integers need to define their own ad-hoc extensions. An issue which, which very much unfortunate, is not exactly shocking. Json would have the same if it had integers in the first place. > including binary data, including NUL Nul is a perfectly valid UTF8 character.
- jmillikin 4y ago
  > That is completely unsurprising, attributes suck ass for data, > what would XML-RPC even have used them for? Instead of this: <struct> <member> <name>faultCode</name> <value><int>-123</int></value> </member> <member> <name>faultString</name> <value><string>some fault</string></value> </member> </struct> It could have been: <struct> <member name="faultCode"><int>-123</int></member> <member name="faultString"><string>some fault</string></member> </struct> or even: <struct> <int name="faultCode">-123</int> <string name="faultString">some fault</string> </struct> -- > An issue which, which very much unfortunate, is not exactly shocking. > Json would have the same if it had integers in the first place. XDR (used in Sun RPC) was defined in the '80s and had 64-bit integers. The DEC Alpha line had been out for 5 years or so when XML-RPC was published. It's not like 64-bit integers were unusual at the time. > Nul is a perfectly valid UTF8 character. NUL is not permitted in XML documents. The XML-RPC spec's explicit directive to allow NUL puts it at odds with standards-conforming XML parsers.
  - ytch 4y ago
    This reminds me that Apple's Plist[1] is similar too: <array> <dict> <key>CFBundleTypeName</key> <string>ShapeEditDocument</string> <key>LSHandlerRank</key> <string>Owner</string> <key>LSItemContentTypes</key> <array> <string>com.example.shape-doc</string> </array> </dict> </array> ( Taken from[2] ) Maybe it's common to design XML schema like this at that time? [1] https://en.wikipedia.org/wiki/Property_list https://en.wikipedia.org/wiki/Property_list [2] https://github.com/robovm/apple-ios-samples/blob/1f6b14ef6e2bda610ef8f1c93b16bf76b8594fbb/ShapeEditBuildingaSimpleiCloudDocumentApp/ShapeEdit/Info.plist https://github.com/robovm/apple-ios-samples/blob/1f6b14ef6e2...
    - i386 4y ago
      You’re absolutely correct. That’s how XML was written in the late 1990s.
dtech 4y ago
> I'm convinced the author was not aware of attributes, or for some reason held a grudge against them. The XML heyday is a little before my time, but I think attributes were considered improper design compared to only using tags by purists.
- kevincox 4y ago
  It is arguably "good design" because tags are infinitely extensible whereas attributes are not. For example in the case the author supplied imagine if you wanted to add a compression type to a member. Your only choice would be an attribute which means that it can only be a string. <struct> <member name="faultCode"><int>-123</int></member> <member name="faultString"><string>some fault</string></member> </struct> If in the original you could add a new element inside the member like <compression> <algorithm>zstd</algorithm> <dictionary>dict7</dictionary> </compression> Not the best example but the point stands. Of course the complaint is valid because while extensibility is valuable you need to weigh it against the cost of the verbosity (both in use and for developers).
  - jmillikin 4y ago
    Note that in the specific case of XML-RPC, extensibility is forbidden by the spec: A <fault> struct may not contain members other than those specified. This is true for all other structures. We believe the specification is flexible enough so that all reasonable data-transfer needs can be accomodated within the specified structures. If you believe strongly that this is not true, please post a message on the discussion group. At one point the author did intend to allow elements to contain user-defined children[0], but it seems like their position changed between that mailing list post and the spec update 7 days later. [0] https://web.archive.org/web/19991010205056/http://discuss.userland.com/msgReader$2135 https://web.archive.org/web/19991010205056/http://discuss.us...
- wiredfool 4y ago
  My recollection of the personalities involved is that the author is absolutely not a purist. He's a pragmatist, and this was simple, understandable, and worked well enough at the time. He was doing this because he needed the functionality, not because it was the best design possible. I also think that this design was also reasonably well aligned with the internal XML capabilities of Frontier.
wiredfool 4y ago
At the time, the point was simple, non-binary RPC calls. I'm pretty sure that the spec came from a hacked up client and server, then frozen, with all warts intact. I'd be surprised if the initial (and final) spec had more than a handful of weeks of actual work in it. At the time, it did mostly work, and it did enable an explosion of connectivity in the early weblog adjacent world. SOAP was basically XML-RPC++, with namespaces, 4 part harmony, and 8x10 color glossy photographs. It's the codification of XML-RPC adding all of those XML features, data types, and interface definitions. It's also a miracle if there's any interop between different stacks.
stefan_ 4y ago
My favorite is DCE RPC, which specifies in every header if the data is little endian or big endian and most importantly, how floating point values are encoded (choose one: IEEE, VAX, CRAY, IBM) [1]. 1: https://pubs.opengroup.org/onlinepubs/9629399/chap14.htm https://pubs.opengroup.org/onlinepubs/9629399/chap14.htm
jillesvangurp 4y ago
Compared to SOAP, this was pretty minimalistic and a pragmatic way to talk to a server from a browser. And of course people actually used SOAP as a really convoluted way to do RPC. This was the lightweight alternative. Json rpc also briefly was a thing. I had the "pleasure" of dealing with a few SOAP web services a few times. That stuff was just completely horrible and unusable. HTML5 became a thing when W3C began insisting that XHTML should be a thing, complete with namespaces and other nonsense. That was peak XML basically. Developers rejected it. Browser vendors and content creators basically stepped in and fixed HTML and CSS properly because w3c could not be bothered to step over the semantics and actually define how that stuff was supposed to behave in the real world (i.e. specifiying the semantics of their semantic HTML). Come to think about it, there was a whole bunch of things that came out of w3c that essentially was about using namespaced XML: web service architecture, RDF, semantic web, XHTML, etc. All that stuff seems to have faded away into obscurity. XML namespaces sound like a good idea until you realize that it just means endless verbosity that doesn't really serve much purpose and that really complicates things like parsing. Thankfully, people simply refused to apply that to json. Most json continues to be free of that nonsense. Same with yaml and similar formats. Works great without a lot of namespace urls.
- oofbey 4y ago
  It wasn’t “people” who saved JSON from this nonsense. It was Douglas Crockford, alone. I have been frequently annoyed at him for not allowing comments in JSON, but the explicit reason was that people were using json comments to encode all the metadata and crap from XML like namespaces to make fully reversible JSON/XML converters. And Crockford very much did not want JSON to be compatible with XML - he wanted it to be a simpler alternative. So good on him.
  - kstrauser 4y ago
    I hadn’t heard that explanation for not having comments, but if true, I want to buy him a drink.
  - kortex 4y ago
    > the explicit reason was that people were using json comments to encode all the metadata and crap from XML like namespaces to make fully reversible JSON/XML converters Oh is that why. I had heard that comments were taken out because people were using it for "parser directives" or something vague. This makes a lot more sense (although I'm sure there were all manners of hacks in the comments). Definitely a bullet dodged there.
icedchai 4y ago
This brings back bad memories. I once worked on a mobile app that used XML-RPC for communications, in the late 2000's. I remember looking at packet captures of this stuff. Ugh.

masklinn 4y ago

> I'm convinced the author was not aware of attributes, or for some reason held a grudge against them. That is completely unsurprising, attributes suck ass for data, what would XML-RPC even have used them for? Replacing <int> by <integer width=“32”>? Other serialisation formats make the same choice (e.g. plists) because it’s so much simpler. > There's only like five primitive types, so implementations that want to transmit wild and unusual values such as 64-bit integers need to define their own ad-hoc extensions. An issue which, which very much unfortunate, is not exactly shocking. Json would have the same if it had integers in the first place. > including binary data, including NUL Nul is a perfectly valid UTF8 character.

jmillikin 4y ago
> That is completely unsurprising, attributes suck ass for data, > what would XML-RPC even have used them for? Instead of this: <struct> <member> <name>faultCode</name> <value><int>-123</int></value> </member> <member> <name>faultString</name> <value><string>some fault</string></value> </member> </struct> It could have been: <struct> <member name="faultCode"><int>-123</int></member> <member name="faultString"><string>some fault</string></member> </struct> or even: <struct> <int name="faultCode">-123</int> <string name="faultString">some fault</string> </struct> -- > An issue which, which very much unfortunate, is not exactly shocking. > Json would have the same if it had integers in the first place. XDR (used in Sun RPC) was defined in the '80s and had 64-bit integers. The DEC Alpha line had been out for 5 years or so when XML-RPC was published. It's not like 64-bit integers were unusual at the time. > Nul is a perfectly valid UTF8 character. NUL is not permitted in XML documents. The XML-RPC spec's explicit directive to allow NUL puts it at odds with standards-conforming XML parsers.
- ytch 4y ago
  This reminds me that Apple's Plist[1] is similar too: <array> <dict> <key>CFBundleTypeName</key> <string>ShapeEditDocument</string> <key>LSHandlerRank</key> <string>Owner</string> <key>LSItemContentTypes</key> <array> <string>com.example.shape-doc</string> </array> </dict> </array> ( Taken from[2] ) Maybe it's common to design XML schema like this at that time? [1] https://en.wikipedia.org/wiki/Property_list https://en.wikipedia.org/wiki/Property_list [2] https://github.com/robovm/apple-ios-samples/blob/1f6b14ef6e2bda610ef8f1c93b16bf76b8594fbb/ShapeEditBuildingaSimpleiCloudDocumentApp/ShapeEdit/Info.plist https://github.com/robovm/apple-ios-samples/blob/1f6b14ef6e2...
  - i386 4y ago
    You’re absolutely correct. That’s how XML was written in the late 1990s.

dtech 4y ago

> I'm convinced the author was not aware of attributes, or for some reason held a grudge against them. The XML heyday is a little before my time, but I think attributes were considered improper design compared to only using tags by purists.

kevincox 4y ago
It is arguably "good design" because tags are infinitely extensible whereas attributes are not. For example in the case the author supplied imagine if you wanted to add a compression type to a member. Your only choice would be an attribute which means that it can only be a string. <struct> <member name="faultCode"><int>-123</int></member> <member name="faultString"><string>some fault</string></member> </struct> If in the original you could add a new element inside the member like <compression> <algorithm>zstd</algorithm> <dictionary>dict7</dictionary> </compression> Not the best example but the point stands. Of course the complaint is valid because while extensibility is valuable you need to weigh it against the cost of the verbosity (both in use and for developers).
- jmillikin 4y ago
  Note that in the specific case of XML-RPC, extensibility is forbidden by the spec: A <fault> struct may not contain members other than those specified. This is true for all other structures. We believe the specification is flexible enough so that all reasonable data-transfer needs can be accomodated within the specified structures. If you believe strongly that this is not true, please post a message on the discussion group. At one point the author did intend to allow elements to contain user-defined children[0], but it seems like their position changed between that mailing list post and the spec update 7 days later. [0] https://web.archive.org/web/19991010205056/http://discuss.userland.com/msgReader$2135 https://web.archive.org/web/19991010205056/http://discuss.us...
wiredfool 4y ago
My recollection of the personalities involved is that the author is absolutely not a purist. He's a pragmatist, and this was simple, understandable, and worked well enough at the time. He was doing this because he needed the functionality, not because it was the best design possible. I also think that this design was also reasonably well aligned with the internal XML capabilities of Frontier.

wiredfool 4y ago

At the time, the point was simple, non-binary RPC calls. I'm pretty sure that the spec came from a hacked up client and server, then frozen, with all warts intact. I'd be surprised if the initial (and final) spec had more than a handful of weeks of actual work in it. At the time, it did mostly work, and it did enable an explosion of connectivity in the early weblog adjacent world. SOAP was basically XML-RPC++, with namespaces, 4 part harmony, and 8x10 color glossy photographs. It's the codification of XML-RPC adding all of those XML features, data types, and interface definitions. It's also a miracle if there's any interop between different stacks.

stefan_ 4y ago

My favorite is DCE RPC, which specifies in every header if the data is little endian or big endian and most importantly, how floating point values are encoded (choose one: IEEE, VAX, CRAY, IBM) [1]. 1: https://pubs.opengroup.org/onlinepubs/9629399/chap14.htm https://pubs.opengroup.org/onlinepubs/9629399/chap14.htm

jillesvangurp 4y ago

Compared to SOAP, this was pretty minimalistic and a pragmatic way to talk to a server from a browser. And of course people actually used SOAP as a really convoluted way to do RPC. This was the lightweight alternative. Json rpc also briefly was a thing. I had the "pleasure" of dealing with a few SOAP web services a few times. That stuff was just completely horrible and unusable. HTML5 became a thing when W3C began insisting that XHTML should be a thing, complete with namespaces and other nonsense. That was peak XML basically. Developers rejected it. Browser vendors and content creators basically stepped in and fixed HTML and CSS properly because w3c could not be bothered to step over the semantics and actually define how that stuff was supposed to behave in the real world (i.e. specifiying the semantics of their semantic HTML). Come to think about it, there was a whole bunch of things that came out of w3c that essentially was about using namespaced XML: web service architecture, RDF, semantic web, XHTML, etc. All that stuff seems to have faded away into obscurity. XML namespaces sound like a good idea until you realize that it just means endless verbosity that doesn't really serve much purpose and that really complicates things like parsing. Thankfully, people simply refused to apply that to json. Most json continues to be free of that nonsense. Same with yaml and similar formats. Works great without a lot of namespace urls.

oofbey 4y ago
It wasn’t “people” who saved JSON from this nonsense. It was Douglas Crockford, alone. I have been frequently annoyed at him for not allowing comments in JSON, but the explicit reason was that people were using json comments to encode all the metadata and crap from XML like namespaces to make fully reversible JSON/XML converters. And Crockford very much did not want JSON to be compatible with XML - he wanted it to be a simpler alternative. So good on him.
- kstrauser 4y ago
  I hadn’t heard that explanation for not having comments, but if true, I want to buy him a drink.
- kortex 4y ago
  > the explicit reason was that people were using json comments to encode all the metadata and crap from XML like namespaces to make fully reversible JSON/XML converters Oh is that why. I had heard that comments were taken out because people were using it for "parser directives" or something vague. This makes a lot more sense (although I'm sure there were all manners of hacks in the comments). Definitely a bullet dodged there.

icedchai 4y ago

This brings back bad memories. I once worked on a mobile app that used XML-RPC for communications, in the late 2000's. I remember looking at packet captures of this stuff. Ugh.