3 ms·

There's one case where I like XML. Interoperating with someone who, starting with a clean sheet, would have concocted something worse than XML (EDI is a popular

There's one case where I like XML. Interoperating with someone who, starting with a clean sheet, would have concocted something worse than XML (EDI is a popular example). That said, I really don't see the value of whitespace and comments in our RPC traffic. I wish the industry were mature enough to successfully use ASN.1 for this stuff.

ct4ul4u 17y ago
There must be some campaign of secrecy around ASN.1. When I mention it to otherwise well-informed developers, they have never heard of it. It doesn't help that the really high performance marshalling libraries aren't open source.
- eugenejen 17y ago
  There was a real time blog search engine, pubsub.com. The internal messages was encoded in ASN.1. Bob Wyman, CTO of the pubsub.com was part of the ASN.1 committees in 80's. How can no one hear ASN.1? Andy Tanenbaum's Computer Network mentions it. SSL uses part of it.
  - metajack 17y ago
    I've heard of it, but have no direct experience. Everyone I've met who has direct experience just shudders whenever it is mentioned. My understanding is that it is extremely complex and difficult to implement.
  - bobwyman 17y ago
    Eugene, I wasn't actually a formal member of the standards committee, however, since my products at DEC (ALL-IN-1 mail, etc.) were some of the earliest to use ASN.1, I was very involved in the standardization process. ASN.1 is everywhere whether or not people realize it. Just about all telecommunications systems (cell phones, etc.) use it internally. As you say, SSL uses it -- since public key certificates are encoded in a form of ASN.1 also the IETF SNMP protocol relies on ASN.1. Other examples would include X.400 and X.500 standards and the LDAP which is a simplified version of X.500. You may be interested to know that internally at Google, we use the "Protocol Buffer" format instead of XML for all internal processing. That format is, like ASN.1 a "tag/length/value" binary encoding. For more information see: http://code.google.com/apis/protocolbuffers/ http://code.google.com/apis/protocolbuffers/
    - eugenejen 17y ago
      Bob, I know that. and Facebook uses Thrift which is now part of Apache incubator project. I may need to deal with some large data set with Cassandra database, which communicates with external programs in Thrift format messages. As a side note, I hope Google will finish real time search api soon and open the firehose :-)
- tentonova 17y ago
  The primary issues with ASN.1 seem to be that the availability libraries are poor and the documentation is both hard to find, and difficult to follow.
ajross 17y ago
ASN.1 is no better. You get some space efficiency (and maybe some CPU efficiency depending on how special-cased your parser is) at the cost of a specification that is, frankly, much more complicated than XML. The use of text is the right thing; if you really care about performance at a level that naively compressed text formats can't provide, you frankly need to be defining your own wire protocols anyway.
- tentonova 17y ago
  The use of text is the right thing; Why? It's genuinely inefficient. This is not academic, as easily illustrated by serializing even a megabyte of binary data in a text-encoded document. Base64+GZIP is not a solution. ... if you really care about performance at a level that naively compressed text formats can't provide ... Why are we optimizing a transfer serialization for your own readability and not the devices we're actually targeting? I've wasted days trying to get barely acceptable XML parsing performance on the iPhone, and similar lower-end ARM devices are becoming much more prevalent these days. ... you frankly need to be defining your own wire protocols anyway. Protocols like XMPP dictate the use of XML. There wouldn't be nearly as many performance issues if they'd used something akin to Thrift, Protobuf, or even the woefully misunderstood ASN.1 DER. Since XMPP uses XML, they've had to define complex out-of-band protocols for transfering file data, instead of simply sending the data in-line as simple binary XMPP messages. I could not more strongly disagree with your position. Protocol implementations should use well-defined, efficient binary serialization mechanisms (such as Protobuf or Thrift) wherever possible to maximize performance, and minimize both serialization overhead and implementation complexity.
  - metajack 17y ago
    Even with a binary format, you'd be sending data out of band in XMPP. XMPP is designed for interchanging _small_ bits of data. Large data blocks destroy latency, so they are usually sent out of band. Plenty of large data blocks occur without any binary file transfers, like disco items responses from large services. The solution to this problem isn't a binary encoding of XMPP, it's multiple simultaneous streams. The problem exists regardless of the encoding. Some of the solutions for this set up internal streams (In-band bytestreams and in-band Jingle are two examples). Since well defined protocols already exist for voice/video and other such things, those are used when possible. This isn't a bad thing. No technology needs to do everything itself. Also, the complexity of out-of-band protocols in XMPP has nothing at all to do with XML. It has to do with NATs and firewalls. It is not easy to set up peer-to-peer communication on the Internet that actually works. SIP and RTP and their ilk are quite complex, I agree. Jingle also has many simple out-of-band and in-band set ups you can use as well, but they aren't nearly as complete or flexible.

ct4ul4u 17y ago

There must be some campaign of secrecy around ASN.1. When I mention it to otherwise well-informed developers, they have never heard of it. It doesn't help that the really high performance marshalling libraries aren't open source.

eugenejen 17y ago
There was a real time blog search engine, pubsub.com. The internal messages was encoded in ASN.1. Bob Wyman, CTO of the pubsub.com was part of the ASN.1 committees in 80's. How can no one hear ASN.1? Andy Tanenbaum's Computer Network mentions it. SSL uses part of it.
- metajack 17y ago
  I've heard of it, but have no direct experience. Everyone I've met who has direct experience just shudders whenever it is mentioned. My understanding is that it is extremely complex and difficult to implement.
- bobwyman 17y ago
  Eugene, I wasn't actually a formal member of the standards committee, however, since my products at DEC (ALL-IN-1 mail, etc.) were some of the earliest to use ASN.1, I was very involved in the standardization process. ASN.1 is everywhere whether or not people realize it. Just about all telecommunications systems (cell phones, etc.) use it internally. As you say, SSL uses it -- since public key certificates are encoded in a form of ASN.1 also the IETF SNMP protocol relies on ASN.1. Other examples would include X.400 and X.500 standards and the LDAP which is a simplified version of X.500. You may be interested to know that internally at Google, we use the "Protocol Buffer" format instead of XML for all internal processing. That format is, like ASN.1 a "tag/length/value" binary encoding. For more information see: http://code.google.com/apis/protocolbuffers/ http://code.google.com/apis/protocolbuffers/
  - eugenejen 17y ago
    Bob, I know that. and Facebook uses Thrift which is now part of Apache incubator project. I may need to deal with some large data set with Cassandra database, which communicates with external programs in Thrift format messages. As a side note, I hope Google will finish real time search api soon and open the firehose :-)
tentonova 17y ago
The primary issues with ASN.1 seem to be that the availability libraries are poor and the documentation is both hard to find, and difficult to follow.

ajross 17y ago

ASN.1 is no better. You get some space efficiency (and maybe some CPU efficiency depending on how special-cased your parser is) at the cost of a specification that is, frankly, much more complicated than XML. The use of text is the right thing; if you really care about performance at a level that naively compressed text formats can't provide, you frankly need to be defining your own wire protocols anyway.

tentonova 17y ago
The use of text is the right thing; Why? It's genuinely inefficient. This is not academic, as easily illustrated by serializing even a megabyte of binary data in a text-encoded document. Base64+GZIP is not a solution. ... if you really care about performance at a level that naively compressed text formats can't provide ... Why are we optimizing a transfer serialization for your own readability and not the devices we're actually targeting? I've wasted days trying to get barely acceptable XML parsing performance on the iPhone, and similar lower-end ARM devices are becoming much more prevalent these days. ... you frankly need to be defining your own wire protocols anyway. Protocols like XMPP dictate the use of XML. There wouldn't be nearly as many performance issues if they'd used something akin to Thrift, Protobuf, or even the woefully misunderstood ASN.1 DER. Since XMPP uses XML, they've had to define complex out-of-band protocols for transfering file data, instead of simply sending the data in-line as simple binary XMPP messages. I could not more strongly disagree with your position. Protocol implementations should use well-defined, efficient binary serialization mechanisms (such as Protobuf or Thrift) wherever possible to maximize performance, and minimize both serialization overhead and implementation complexity.
- metajack 17y ago
  Even with a binary format, you'd be sending data out of band in XMPP. XMPP is designed for interchanging _small_ bits of data. Large data blocks destroy latency, so they are usually sent out of band. Plenty of large data blocks occur without any binary file transfers, like disco items responses from large services. The solution to this problem isn't a binary encoding of XMPP, it's multiple simultaneous streams. The problem exists regardless of the encoding. Some of the solutions for this set up internal streams (In-band bytestreams and in-band Jingle are two examples). Since well defined protocols already exist for voice/video and other such things, those are used when possible. This isn't a bad thing. No technology needs to do everything itself. Also, the complexity of out-of-band protocols in XMPP has nothing at all to do with XML. It has to do with NATs and firewalls. It is not easy to set up peer-to-peer communication on the Internet that actually works. SIP and RTP and their ilk are quite complex, I agree. Jingle also has many simple out-of-band and in-band set ups you can use as well, but they aren't nearly as complete or flexible.