5 ms·

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JS

(I've used both DataPower and COBOL - I've got a healthy respect for robust, long-lived legacy systems). I must admit I was scratching my head on this. The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it. Maybe unchecking this causes the legacy backend to get hammered? There are valid non-spec concerns with order. Replies here seem to assume stupidity here. It's a valid reason, but it's not the only one. Equally, the author doesn't ask "why" - why would it take so long, why was that option enabled?

blattimwind 7y ago
> JWT needs it for example (and I'd assume many signature models) You almost never need canonical representations for signing things. I would even say that if you need a canonical representation to sign your things, then that is a design smell of your cryptographic protocol.
- moby_click 7y ago
  You don't _need_ canonical representations for signing, but then you can't let go of the representation used for computing the signature. What is the argument against a requirement to only sign canonical data?
- atemerev 7y ago
  Could you please care to explain more? What is the "design smell", and what is the alternative solution to canonicalization?
- jwilliams 7y ago
  I didn't mention canonicalization. My point is that serialized JSON is ordered - which I think is exactly the same property you're referring to.
  - coldtea 7y ago
    That's an implementation detail. Serialized JSON can also be printed, but your systems shouldn't depend that JSON is always ink on paper.
moby_click 7y ago
If you require serialized JSON in a specific order, you are not requiring JSON, but some dialect of JSON and that should be clearly communicated. If, for some reason, a client needed a special serialization, i'd ask them for a spec, because the JSON spec does not apply anymore, and be done with it.
- jwilliams 7y ago
  I think my argument is a bit different -- sure, this is an interface (maybe to a COBOL copybook in this case?). And yes, it's not "JSON". Agree it's a leaky abstraction. Equally agree it should be documented. Often legacy systems have weirdness after seeing decades of edge-cases. Weirdness that makes them robust in all sorts of unlikely ways. Equally makes them poorly documented, leaky, opaque, and frustrating. But I certainly have a lot of pause with the sentiment of "some idiot checked the JSON order checkbox on DataPower". My first thought is instead "I wonder why someone thought this was necessary."
  - moby_click 7y ago
    Yes, that sentiment is highly unprofessional. 9x6 months translates to "that box is checked for reason" and "it's not going to change". Even if things are going to change, it's probably going to affect one function on your part and you are producing valid JSON anyway. Surely you can bill a fortune 500 company for a function to put things in order.
  - cat199 7y ago
    Money down 3 months later: 1) the gateway or more likely something behind it starts crashing for memory errors when someone is serializing some stupidly large array into it 2) using ordered vs non-ordered keys on input changes the behavior and fixes it 3) enabling this checkbox to prevent it fixes the issue and guards against it happening again 4) everyone has forgotten about this web weenies inquiry into the issue 5) due to #4, it takes like 3 weeks of system crashes and hairy debugging to find the cause 6) this guy is long gone, having ridden off into the sunset of smugness Also: 1) fixing the serialization-dependent memory allocation 4 layers deep on the back-end to allow things to operate safely without this checkbox requires a complex change to several other components and associated system-wide validation testing which would take: drumroll "9 people 6 months to complete" 2) What is the point of this javadoc reference and how exactly does it relate to the issue at hand? Here's a random code doc reference too!: https://pymotw.com/3/collections/ordereddict.html https://pymotw.com/3/collections/ordereddict.html and here also a discussion about impacts: https://mail.python.org/pipermail/python-dev/2016-September/thread.html#146327 https://mail.python.org/pipermail/python-dev/2016-September/...
cntlzw 7y ago
Justgoogled DataPower, out of curiosity, and I found XML Accelerator XA35. I did not know there is HARDWARE for XML PROCESSING! Blows my mind.
- diskotek 7y ago
  Yep, back in the SOAP days we used to have a lot of XML flying around that would need to be converted into one format or another via XSLT. You can certainly run XSLT on a piece of XML in your own code, but having a dedicated piece of hardware do it for you with a slick GUI was always an easy sell for IBM to management.
  - cntlzw 7y ago
    Could you elaborate? I am really curious how this hardware worked. Was it a HTML Gui with file upload and everything? Or just some server, port and proprietary protocol? Maybe these things are on Ebay for cheap. Oh, that looks like fun.
- lioeters 7y ago
  Impressed to learn about this hardware for XML processing. I came across an old article that mentioned the price started at $55K!
cntlzw 7y ago
No idea why someone needs ordered key-value-pairs. I had the please of working with a device that relied upon ordered query string parameters. Why? Because they encrypted the values with RC4.
clusmore 7y ago
I can't help but think of the story of Chesterton's Fence [1]. >There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, 'I don't see the use of this; let us clear it away.' To which the more intelligent type of reformer will do well to answer: 'If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.' [1]: https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fen...
inopinatus 7y ago
> JWT needs it I already had at least four solid reasons not to use JWT. Still, adding another just keeps the dumpster fire burning. By which I mean, in the most oblique and backhanded fashion, that there’s not going to be any reason for validating JSON order that doesn’t ultimately reveal or confirm a design gaffe somewhere down the rabbit hole.
coldtea 7y ago
>The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it Whether the file has an order is an implementation detail. There might be no static file at all, for example, it could be an endpoint returning you the JSON, and it could return different orders for the exact same items every time you call it. That would still be perfectly valid. If you "need it", don't put your data into an object, put it in a list of objects. If the underlying layers need "ordered objects" (in other words, ordered hashmaps) they don't really use JSON.
- Spivak 7y ago
  This isn't really an issue of spec because these systems recognize a completely different protocol 'OrderedJSON' which interprets {} as an ordered list of key-value pairs instead of a set. The mistake is assuming that two syntactically compatible protocols were actually the same protocol. uint =! int.
  - coldtea 7y ago
    Only there's no such protocol. It's some adhoc implementation within a single system (or a single IBM ecosystem).

blattimwind 7y ago

> JWT needs it for example (and I'd assume many signature models) You almost never need canonical representations for signing things. I would even say that if you need a canonical representation to sign your things, then that is a design smell of your cryptographic protocol.

moby_click 7y ago
You don't _need_ canonical representations for signing, but then you can't let go of the representation used for computing the signature. What is the argument against a requirement to only sign canonical data?
atemerev 7y ago
Could you please care to explain more? What is the "design smell", and what is the alternative solution to canonicalization?
jwilliams 7y ago
I didn't mention canonicalization. My point is that serialized JSON is ordered - which I think is exactly the same property you're referring to.
- coldtea 7y ago
  That's an implementation detail. Serialized JSON can also be printed, but your systems shouldn't depend that JSON is always ink on paper.

moby_click 7y ago

If you require serialized JSON in a specific order, you are not requiring JSON, but some dialect of JSON and that should be clearly communicated. If, for some reason, a client needed a special serialization, i'd ask them for a spec, because the JSON spec does not apply anymore, and be done with it.

jwilliams 7y ago
I think my argument is a bit different -- sure, this is an interface (maybe to a COBOL copybook in this case?). And yes, it's not "JSON". Agree it's a leaky abstraction. Equally agree it should be documented. Often legacy systems have weirdness after seeing decades of edge-cases. Weirdness that makes them robust in all sorts of unlikely ways. Equally makes them poorly documented, leaky, opaque, and frustrating. But I certainly have a lot of pause with the sentiment of "some idiot checked the JSON order checkbox on DataPower". My first thought is instead "I wonder why someone thought this was necessary."
- moby_click 7y ago
  Yes, that sentiment is highly unprofessional. 9x6 months translates to "that box is checked for reason" and "it's not going to change". Even if things are going to change, it's probably going to affect one function on your part and you are producing valid JSON anyway. Surely you can bill a fortune 500 company for a function to put things in order.
- cat199 7y ago
  Money down 3 months later: 1) the gateway or more likely something behind it starts crashing for memory errors when someone is serializing some stupidly large array into it 2) using ordered vs non-ordered keys on input changes the behavior and fixes it 3) enabling this checkbox to prevent it fixes the issue and guards against it happening again 4) everyone has forgotten about this web weenies inquiry into the issue 5) due to #4, it takes like 3 weeks of system crashes and hairy debugging to find the cause 6) this guy is long gone, having ridden off into the sunset of smugness Also: 1) fixing the serialization-dependent memory allocation 4 layers deep on the back-end to allow things to operate safely without this checkbox requires a complex change to several other components and associated system-wide validation testing which would take: drumroll "9 people 6 months to complete" 2) What is the point of this javadoc reference and how exactly does it relate to the issue at hand? Here's a random code doc reference too!: https://pymotw.com/3/collections/ordereddict.html https://pymotw.com/3/collections/ordereddict.html and here also a discussion about impacts: https://mail.python.org/pipermail/python-dev/2016-September/thread.html#146327 https://mail.python.org/pipermail/python-dev/2016-September/...

cntlzw 7y ago

Justgoogled DataPower, out of curiosity, and I found XML Accelerator XA35. I did not know there is HARDWARE for XML PROCESSING! Blows my mind.

diskotek 7y ago
Yep, back in the SOAP days we used to have a lot of XML flying around that would need to be converted into one format or another via XSLT. You can certainly run XSLT on a piece of XML in your own code, but having a dedicated piece of hardware do it for you with a slick GUI was always an easy sell for IBM to management.
- cntlzw 7y ago
  Could you elaborate? I am really curious how this hardware worked. Was it a HTML Gui with file upload and everything? Or just some server, port and proprietary protocol? Maybe these things are on Ebay for cheap. Oh, that looks like fun.
lioeters 7y ago
Impressed to learn about this hardware for XML processing. I came across an old article that mentioned the price started at $55K!

cntlzw 7y ago

No idea why someone needs ordered key-value-pairs. I had the please of working with a device that relied upon ordered query string parameters. Why? Because they encrypted the values with RC4.

clusmore 7y ago

I can't help but think of the story of Chesterton's Fence [1]. >There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, 'I don't see the use of this; let us clear it away.' To which the more intelligent type of reformer will do well to answer: 'If you don't see the use of it, I certainly won't let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.' [1]: https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fence https://en.m.wikipedia.org/wiki/Wikipedia:Chesterton%27s_fen...

inopinatus 7y ago

> JWT needs it I already had at least four solid reasons not to use JWT. Still, adding another just keeps the dumpster fire burning. By which I mean, in the most oblique and backhanded fashion, that there’s not going to be any reason for validating JSON order that doesn’t ultimately reveal or confirm a design gaffe somewhere down the rabbit hole.

coldtea 7y ago

>The JSON spec might not specify order, but the serialized JSON is ordered by nature. JWT needs it for example (and I'd assume many signature models). Or you might have a caching layer that needs it Whether the file has an order is an implementation detail. There might be no static file at all, for example, it could be an endpoint returning you the JSON, and it could return different orders for the exact same items every time you call it. That would still be perfectly valid. If you "need it", don't put your data into an object, put it in a list of objects. If the underlying layers need "ordered objects" (in other words, ordered hashmaps) they don't really use JSON.

Spivak 7y ago
This isn't really an issue of spec because these systems recognize a completely different protocol 'OrderedJSON' which interprets {} as an ordered list of key-value pairs instead of a set. The mistake is assuming that two syntactically compatible protocols were actually the same protocol. uint =! int.
- coldtea 7y ago
  Only there's no such protocol. It's some adhoc implementation within a single system (or a single IBM ecosystem).