4 ms·

> what should be used instead to encode structured data, or perhaps more importantly what should have been used to encode graph like things such as map/lists/ob

> what should be used instead to encode structured data, or perhaps more importantly what should have been used to encode graph like things such as map/lists/objects in the 2000’s Lisp?

josteink 7y ago
That this answer is voted down, on HN of all places, is kinda sad. It may not be the popular answer, but it’s definitely a valid answer.
- taberiand 7y ago
  It was because I said Lisp instead of s-expressions - I find imprecise language is always a good way to garner down-votes on HN
soapdog 7y ago
I think you mean s-expressions instead of lisp and if you do, I'm with you. They are really neat and underused. Now, if you want to carry a whole language runtime for the structured data format you'll probably end up with the "which lisp" kind of problem in which each app end up using something different and then the structured data files are no longer exactly portable (until you devise a portable standard yadayadayada)
- taberiand 7y ago
  Yes! I meant s-expressions, the name escaped me - thanks.
- AnimalMuppet 7y ago
  I have to admit, once you've seen S-expressions, XML looks insanely bloated and verbose. The advantage XML would have in that situation is that, because it's so verbose, if you get a malformed XML, you can eyeball-parse it and often figure out how to hand-edit it to make it valid. If it is valid, you can also see exactly how the schema you were sent differs from the schema you expected. An S-expression, having less redundancy, also is potentially more brittle.
  - wruza 7y ago
    If malforming is regular (like someone printf’d or typed in bad xmls), then the same is true for any human-readable format, since you know data and what it should look like. If not, (like in randomly broken packet), then you solve the problem at the wrong level. By-hand error-correcting verbosity is hardly a selling point of the application level protocol.
    - AnimalMuppet 7y ago
      > By-hand error-correcting verbosity is hardly a selling point of the application level protocol. It's a selling point while you're trying to figure out how to get it working. Once you have it working, it's not - but by then, you have it working, so why change it?
alkonaut 7y ago
All you need on .NET/Java in 2007 then is a lisp parser/interpreter. I know they are easy to write, but they aren't as easy to write as something you don't have to write at all. Let's not forget, many one of these configuration files and data formats are one-off hacks that were meant to be replaced by a real format, a real parser, a real DSL etc. The reason the xml config/dsl/format stuck is because it worked. And it was cheap and easy.
- sedachv 7y ago
  S-expressions have been around since the 1960s. McCarthy was already proposing S-expressions for what XML would become, in 1975. Stop whining and finding excuses, and start using s-expressions. http://www-formal.stanford.edu/jmc/cbcl2/cbcl2.html http://www-formal.stanford.edu/jmc/cbcl2/cbcl2.html
  - alkonaut 7y ago
    I'm not trying to pick the best format for the job, I'm usually trying to pick the least bad one that's available IN the platform/standard library I'm using.
thom 7y ago
XML (and JSON) have the advantage of names, which makes it slightly easier when it comes to querying (and indeed building indexes) over lots of data. I'd be amazed if there wasn't tons of work on this for S-expressions, but I can imagine it's slightly clunky.
- rauhl 7y ago
  What do you mean by names here? S-expressions have symbols, which serve the exact same purpose as what I think you might mean: an interned string value which can be cheaply used more than once.
  - thom 7y ago
    Positions within lists aren't named. Sure you can do: ((foo . (bar . baz)) (foo . (bar . baz)) (foo . (bar . baz)) (foo . (bar . baz))) And say "index all the foos", but you're mixing the structure and the content, in a way that JSON and XML explicitly separate.
    - thom 7y ago
      Not saying you can't specify some schema here, but there's nothing native to S-expressions that makes it quite as transparent and simple to specify a path into a data structure.
carapace 7y ago
> Abstract Syntax Notation One (ASN.1) is a standard interface description language for defining data structures that can be serialized and deserialized in a cross-platform way. It is broadly used in telecommunications and computer networking, and especially in cryptography. > ASN.1 is a joint standard of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and ISO/IEC, originally defined in 1984... https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One
- RantyDave 7y ago
  I think ASN.1 was designed for a world with (a) waterfall development and (b) static deployments - ie no over the air updates. Under the circumstances, messing up is simply not an option - hence defining the standard so clearly and for so many use cases. Today, of course, we treat the entirety of our deployed infrastructure as 'merely' a platform to write code. And not only are experimentation and failure OK, they're positively encouraged. Velocity became important.
  - dfox 7y ago
    You can meaningfully decode any DER encoded ASN.1 structure and serialize it back without any knowledge of the schema. Somewhat surprisingly you cannot do that with all instances of XML documents.
- AnimalMuppet 7y ago
  The first thing wrong is that you have to serialize and deserialize it. Operationally, that's inconvenient, and it shows that they're optimizing for network bandwidth. But these days, squeezing the most out of each bit is, in most cases, not a defensible design decision. Then, once you deserialize it, it's still a printable version of ASN.1. Sure, it's unambiguous, rigidly defined, and standardized. It's still gouge-your-eyeballs-out horrible to try to do anything with. Say you get an XML message over the wire with a bit flipped. If you look at it, you have a good chance to be able to figure out what went wrong, edit one character, and you can now process it. If you get an ASN.1 message in the same condition, it's pretty much game over (though there may be special tools that could save you). Say you get an XML, and you don't know the schema. You look at it, and you can see what's going on. You get an ASN.1 where you don't know the schema, and you can be totally sunk. (If I recall correctly, in ASN.1, you can have schemas that are private, that is, not specified in the standard.)

> what should be used instead to encode structured data, or perhaps more importantly what should have been used to encode graph like things such as map/lists/objects in the 2000’s Lisp?

josteink 7y ago

That this answer is voted down, on HN of all places, is kinda sad. It may not be the popular answer, but it’s definitely a valid answer.

taberiand 7y ago
It was because I said Lisp instead of s-expressions - I find imprecise language is always a good way to garner down-votes on HN

soapdog 7y ago

I think you mean s-expressions instead of lisp and if you do, I'm with you. They are really neat and underused. Now, if you want to carry a whole language runtime for the structured data format you'll probably end up with the "which lisp" kind of problem in which each app end up using something different and then the structured data files are no longer exactly portable (until you devise a portable standard yadayadayada)

taberiand 7y ago
Yes! I meant s-expressions, the name escaped me - thanks.
AnimalMuppet 7y ago
I have to admit, once you've seen S-expressions, XML looks insanely bloated and verbose. The advantage XML would have in that situation is that, because it's so verbose, if you get a malformed XML, you can eyeball-parse it and often figure out how to hand-edit it to make it valid. If it is valid, you can also see exactly how the schema you were sent differs from the schema you expected. An S-expression, having less redundancy, also is potentially more brittle.
- wruza 7y ago
  If malforming is regular (like someone printf’d or typed in bad xmls), then the same is true for any human-readable format, since you know data and what it should look like. If not, (like in randomly broken packet), then you solve the problem at the wrong level. By-hand error-correcting verbosity is hardly a selling point of the application level protocol.
  - AnimalMuppet 7y ago
    > By-hand error-correcting verbosity is hardly a selling point of the application level protocol. It's a selling point while you're trying to figure out how to get it working. Once you have it working, it's not - but by then, you have it working, so why change it?

alkonaut 7y ago

All you need on .NET/Java in 2007 then is a lisp parser/interpreter. I know they are easy to write, but they aren't as easy to write as something you don't have to write at all. Let's not forget, many one of these configuration files and data formats are one-off hacks that were meant to be replaced by a real format, a real parser, a real DSL etc. The reason the xml config/dsl/format stuck is because it worked. And it was cheap and easy.

sedachv 7y ago
S-expressions have been around since the 1960s. McCarthy was already proposing S-expressions for what XML would become, in 1975. Stop whining and finding excuses, and start using s-expressions. http://www-formal.stanford.edu/jmc/cbcl2/cbcl2.html http://www-formal.stanford.edu/jmc/cbcl2/cbcl2.html
- alkonaut 7y ago
  I'm not trying to pick the best format for the job, I'm usually trying to pick the least bad one that's available IN the platform/standard library I'm using.

thom 7y ago

XML (and JSON) have the advantage of names, which makes it slightly easier when it comes to querying (and indeed building indexes) over lots of data. I'd be amazed if there wasn't tons of work on this for S-expressions, but I can imagine it's slightly clunky.

rauhl 7y ago
What do you mean by names here? S-expressions have symbols, which serve the exact same purpose as what I think you might mean: an interned string value which can be cheaply used more than once.
- thom 7y ago
  Positions within lists aren't named. Sure you can do: ((foo . (bar . baz)) (foo . (bar . baz)) (foo . (bar . baz)) (foo . (bar . baz))) And say "index all the foos", but you're mixing the structure and the content, in a way that JSON and XML explicitly separate.
  - thom 7y ago
    Not saying you can't specify some schema here, but there's nothing native to S-expressions that makes it quite as transparent and simple to specify a path into a data structure.

carapace 7y ago

> Abstract Syntax Notation One (ASN.1) is a standard interface description language for defining data structures that can be serialized and deserialized in a cross-platform way. It is broadly used in telecommunications and computer networking, and especially in cryptography. > ASN.1 is a joint standard of the International Telecommunication Union Telecommunication Standardization Sector (ITU-T) and ISO/IEC, originally defined in 1984... https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One https://en.wikipedia.org/wiki/Abstract_Syntax_Notation_One

RantyDave 7y ago
I think ASN.1 was designed for a world with (a) waterfall development and (b) static deployments - ie no over the air updates. Under the circumstances, messing up is simply not an option - hence defining the standard so clearly and for so many use cases. Today, of course, we treat the entirety of our deployed infrastructure as 'merely' a platform to write code. And not only are experimentation and failure OK, they're positively encouraged. Velocity became important.
- dfox 7y ago
  You can meaningfully decode any DER encoded ASN.1 structure and serialize it back without any knowledge of the schema. Somewhat surprisingly you cannot do that with all instances of XML documents.
AnimalMuppet 7y ago
The first thing wrong is that you have to serialize and deserialize it. Operationally, that's inconvenient, and it shows that they're optimizing for network bandwidth. But these days, squeezing the most out of each bit is, in most cases, not a defensible design decision. Then, once you deserialize it, it's still a printable version of ASN.1. Sure, it's unambiguous, rigidly defined, and standardized. It's still gouge-your-eyeballs-out horrible to try to do anything with. Say you get an XML message over the wire with a bit flipped. If you look at it, you have a good chance to be able to figure out what went wrong, edit one character, and you can now process it. If you get an ASN.1 message in the same condition, it's pretty much game over (though there may be special tools that could save you). Say you get an XML, and you don't know the schema. You look at it, and you can see what's going on. You get an ASN.1 where you don't know the schema, and you can be totally sunk. (If I recall correctly, in ASN.1, you can have schemas that are private, that is, not specified in the standard.)