7 ms·

This is funny, but it's also a real world example of the kind of encoding nightmare that made SOAP RPC encoding really awkward. Various SOAP toolkits used to se

This is funny, but it's also a real world example of the kind of encoding nightmare that made SOAP RPC encoding really awkward. Various SOAP toolkits used to serialize a missing value as the empty string, or a literal value like "null" or 0, or all sorts of awfulness. I think the correct thing for the spec is to set xsi:nil="true" as an attribute on the XML tag in question, but IIRC about half the toolkits didn't understand that. (I speak in the past tense of SOAP because I am an optimist.)

narrator 15y ago
They must have got the absolutely brilliant idea of conflating null with the empty string from Oracle.
joelhaasnoot 15y ago
Ran into this with a REST XML API recently where someone was trying to do some reflection-type serialization of XML. The API had longitude and latitude of all train stations, and some genious decided to call the tags 'lat' and 'long'. 'long' conflicted with the datatype Long and it wasn't fun. Version 2 of the API has fixed this issue luckily.
- nicholaides 15y ago
  "Lat" and "Long" seem like great tag names for this purpose. It sounds like the problem wasn't this guy, but the "reflection-type serialization of XML".
  - joelhaasnoot 15y ago
    Sure, it's partly both, but 'lat' and 'lon' are used almost universally across Geo-related APIs. See Google Maps for instance.
kodablah 15y ago
I think the absence of the element/attribute is the best way to define null assuming your XSD is set up properly. Many XML marshalling libraries work well with this approach. (note, I too have long since abandoned SOAP)
- scott_w 15y ago
  I've had the joy of working with a SOAP endpoint that doesn't recognise <element /> syntax, which left me having to create attributes assigned to '' in my Python code, so SUDS would generate the <element></element> syntax for me. They also massively over-engineered the endpoint, constantly wrapping elements within elements, for no real reason.
  - acdha 15y ago
    That's always annoyed me: sure, XML can be heavyweight but if we're going to use it we should at least get the benefits. Naturally that line of reasoning didn't get very far with the “maintainers” of an internal purported-SSO system with a SOAP endpoint which crashed on non-ASCII data or SQL special characters in the submitted username / password values.
danudey 15y ago
I worked at a company where we were replacing the user-facing component of our giant, ugly PHP storefront with a Rails version; in doing so, our developers implemented a JSON bridge between the two, allowing the frontend and backend to operate separately, using separate databases (and actually, they were in separate data centres). As we were testing, we found that some products in our database would cause a JSON decoding error on the Rails side. After a few minutes, we realized the problem. We had a string field for something (product IDs, manufacturer SKU, etc). On the PHP side, the JSON encoder was using PHP's is_numeric() for each field to see if the field was a number (to determine how to encode it). Some of the SKUs, however, happened to be composed entirely of digits, and for those, PHP encoded them into the JSON as integer values. This, of course, broke on the Rails end, because Rails was expecting a string value and got an integer value. In the end, we had to write a surprising amount of code to work around the brain damage involved, since regardless of what we tried to do PHP wanted, by default, to send things as integers whenever possible. I believe the final fix was to actually patch the JSON encoder library and special-case that field.
- Achshar 15y ago
  could you have added some text at the end of the id before sending it and then removed the extra bits from the end on recieving side? something like a parity value.
  - davidcuddeback 15y ago
    Or just call String#to_s on the Rails side.
    - DougBTX 15y ago
      That would probably work, but it means that the problem + workaround is spread out across two systems rather than being contained in just one. Also, working around it on the consumer side means that any new consumers (or any new string fields!) will need to use the workaround too, further spreading out the problem. Better to keep it encapsulated in one system if possible.
scott_s 15y ago
This is really a fundamental problem: how do you indicate operation failure? This relies on two things: the range (the valid output values) of the operation itself, and the range of the datatype you're mapping the operation's result to. If the operation and the datatype's range are not equal, then you can indicate failure inside the return value by applying special meaning to invalid values. But if the operation and the datatype's range are equal, then you need another distinct value to indicate failure. The difficulty is in recognizing which situation you're in, and as you point out, this is one where, effectively, the operation and the datatype have the same range.
wpietri 15y ago
Wow. I love that the S stands for Simple. "You keep using that word. I do not think it means what you think it means."
- stcredzero 15y ago
  It started out that way, but then they made all of CORBA's mistakes.
  - smacktoward 15y ago
    Yeah -- before the enterprise types got a hold of it, SOAP was actually fairly pleasant to work with. Sigh. Oh well. You can kind of get a flavor of what pre-enterprise-jackassery SOAP was like to work with by looking at Dave Winer's XML-RPC (spec: http://xmlrpc.scripting.com/spec.html http://xmlrpc.scripting.com/spec.html), which was one of the precursors of SOAP.
  - arethuza 15y ago
    That's a good point - didn't CORBA also start fairly straightforward (I can't believe I said that) but then grew extra layers of mind numbing complexity for transactions, security etc. - pretty much like the various weird WS-* specifications that most people seem to ignore?
- threedaymonk 15y ago
  A lot of protocols and standards containing the word 'simple; aren't. Most of them, in fact! I have a suspicion that this is because these designs start as antitheses to existing complex designs. 'Aha!' say the designers. 'We won't repeat those mistakes!' But because they proceed from the same basic assumptions as the complex designs they try to replace, they always produce something complex in the end, because they never really understood simplicity.
  - yread 15y ago
    SMTP is pretty simple I would say

narrator 15y ago

They must have got the absolutely brilliant idea of conflating null with the empty string from Oracle.

joelhaasnoot 15y ago

Ran into this with a REST XML API recently where someone was trying to do some reflection-type serialization of XML. The API had longitude and latitude of all train stations, and some genious decided to call the tags 'lat' and 'long'. 'long' conflicted with the datatype Long and it wasn't fun. Version 2 of the API has fixed this issue luckily.

nicholaides 15y ago
"Lat" and "Long" seem like great tag names for this purpose. It sounds like the problem wasn't this guy, but the "reflection-type serialization of XML".
- joelhaasnoot 15y ago
  Sure, it's partly both, but 'lat' and 'lon' are used almost universally across Geo-related APIs. See Google Maps for instance.

kodablah 15y ago

I think the absence of the element/attribute is the best way to define null assuming your XSD is set up properly. Many XML marshalling libraries work well with this approach. (note, I too have long since abandoned SOAP)

scott_w 15y ago
I've had the joy of working with a SOAP endpoint that doesn't recognise <element /> syntax, which left me having to create attributes assigned to '' in my Python code, so SUDS would generate the <element></element> syntax for me. They also massively over-engineered the endpoint, constantly wrapping elements within elements, for no real reason.
- acdha 15y ago
  That's always annoyed me: sure, XML can be heavyweight but if we're going to use it we should at least get the benefits. Naturally that line of reasoning didn't get very far with the “maintainers” of an internal purported-SSO system with a SOAP endpoint which crashed on non-ASCII data or SQL special characters in the submitted username / password values.

danudey 15y ago

I worked at a company where we were replacing the user-facing component of our giant, ugly PHP storefront with a Rails version; in doing so, our developers implemented a JSON bridge between the two, allowing the frontend and backend to operate separately, using separate databases (and actually, they were in separate data centres). As we were testing, we found that some products in our database would cause a JSON decoding error on the Rails side. After a few minutes, we realized the problem. We had a string field for something (product IDs, manufacturer SKU, etc). On the PHP side, the JSON encoder was using PHP's is_numeric() for each field to see if the field was a number (to determine how to encode it). Some of the SKUs, however, happened to be composed entirely of digits, and for those, PHP encoded them into the JSON as integer values. This, of course, broke on the Rails end, because Rails was expecting a string value and got an integer value. In the end, we had to write a surprising amount of code to work around the brain damage involved, since regardless of what we tried to do PHP wanted, by default, to send things as integers whenever possible. I believe the final fix was to actually patch the JSON encoder library and special-case that field.

Achshar 15y ago
could you have added some text at the end of the id before sending it and then removed the extra bits from the end on recieving side? something like a parity value.
- davidcuddeback 15y ago
  Or just call String#to_s on the Rails side.
  - DougBTX 15y ago
    That would probably work, but it means that the problem + workaround is spread out across two systems rather than being contained in just one. Also, working around it on the consumer side means that any new consumers (or any new string fields!) will need to use the workaround too, further spreading out the problem. Better to keep it encapsulated in one system if possible.

scott_s 15y ago

This is really a fundamental problem: how do you indicate operation failure? This relies on two things: the range (the valid output values) of the operation itself, and the range of the datatype you're mapping the operation's result to. If the operation and the datatype's range are not equal, then you can indicate failure inside the return value by applying special meaning to invalid values. But if the operation and the datatype's range are equal, then you need another distinct value to indicate failure. The difficulty is in recognizing which situation you're in, and as you point out, this is one where, effectively, the operation and the datatype have the same range.

wpietri 15y ago

Wow. I love that the S stands for Simple. "You keep using that word. I do not think it means what you think it means."

stcredzero 15y ago
It started out that way, but then they made all of CORBA's mistakes.
- smacktoward 15y ago
  Yeah -- before the enterprise types got a hold of it, SOAP was actually fairly pleasant to work with. Sigh. Oh well. You can kind of get a flavor of what pre-enterprise-jackassery SOAP was like to work with by looking at Dave Winer's XML-RPC (spec: http://xmlrpc.scripting.com/spec.html http://xmlrpc.scripting.com/spec.html), which was one of the precursors of SOAP.
- arethuza 15y ago
  That's a good point - didn't CORBA also start fairly straightforward (I can't believe I said that) but then grew extra layers of mind numbing complexity for transactions, security etc. - pretty much like the various weird WS-* specifications that most people seem to ignore?
threedaymonk 15y ago
A lot of protocols and standards containing the word 'simple; aren't. Most of them, in fact! I have a suspicion that this is because these designs start as antitheses to existing complex designs. 'Aha!' say the designers. 'We won't repeat those mistakes!' But because they proceed from the same basic assumptions as the complex designs they try to replace, they always produce something complex in the end, because they never really understood simplicity.
- yread 15y ago
  SMTP is pretty simple I would say