5 ms·

I really wish standards bodies would create validation tests like this so that implementers can better ensure that they are compliant. It seems that reference

by aero142 17y ago

I really wish standards bodies would create validation tests like this so that implementers can better ensure that they are compliant. It seems that reference implementations and test suites would improve standards compliance immensely.

cscotta 17y ago
I couldn't agree more. A good validation test and set of examples are worth a hundred specs. I've been working on a project for the past couple months that involves using XMPP for client-to-client communication. While there are some good how-tos and libraries, the process would be infinitely easier if XMPP.org's documentation was in terms of reference examples and sample packets/messages, rather than incredibly dense specs written in convoluted techspec language that takes some time to comprehend, even when reading intently. A refreshing counterexample is the RubySpec project. While Ruby began with only a reference implementation, the process of developing a comprehensive test suite that is able to measure conformance to the reference implementation has created an invaluable metric for other implementers. While JavaScript has a very thorough specification, Sputnik's project of building a test suite that measures the conformance of an interpreter to the spec will be helpful in understanding how far we have to go, and in which direction we're moving.
barrkel 17y ago
I believe that such test suites would rival the complexity of the practical implementations. Reference implementations sound nice, and may work well for small things, but become large engineering efforts where they are really needed. What would a reference implementation of a Linux distribution look like, for example?
- aero142 17y ago
  I think you are right for particular cases, but CSS, HTML, and Javascript are all feasible to create test suites for. Either in the form of unit tests, or in the form of written test cases. From this input the expected output looks like this. And, unlike the spec itself, these example cases can grow over time to be more complete. A partial test suite is still better than no test suite. I am all for pragmatism and if a particular domain being spec'd doesn't lend itself to a particular method of validation, then don't do it, be there are plenty of specs that could benefit that don't have them.
- jcapote 17y ago
  Perhaps something like this? http://www.linuxfoundation.org/collaborate/workgroups/lsb http://www.linuxfoundation.org/collaborate/workgroups/lsb

by aero142 17y ago

cscotta 17y ago

I couldn't agree more. A good validation test and set of examples are worth a hundred specs. I've been working on a project for the past couple months that involves using XMPP for client-to-client communication. While there are some good how-tos and libraries, the process would be infinitely easier if XMPP.org's documentation was in terms of reference examples and sample packets/messages, rather than incredibly dense specs written in convoluted techspec language that takes some time to comprehend, even when reading intently. A refreshing counterexample is the RubySpec project. While Ruby began with only a reference implementation, the process of developing a comprehensive test suite that is able to measure conformance to the reference implementation has created an invaluable metric for other implementers. While JavaScript has a very thorough specification, Sputnik's project of building a test suite that measures the conformance of an interpreter to the spec will be helpful in understanding how far we have to go, and in which direction we're moving.

barrkel 17y ago

I believe that such test suites would rival the complexity of the practical implementations. Reference implementations sound nice, and may work well for small things, but become large engineering efforts where they are really needed. What would a reference implementation of a Linux distribution look like, for example?

aero142 17y ago
I think you are right for particular cases, but CSS, HTML, and Javascript are all feasible to create test suites for. Either in the form of unit tests, or in the form of written test cases. From this input the expected output looks like this. And, unlike the spec itself, these example cases can grow over time to be more complete. A partial test suite is still better than no test suite. I am all for pragmatism and if a particular domain being spec'd doesn't lend itself to a particular method of validation, then don't do it, be there are plenty of specs that could benefit that don't have them.
jcapote 17y ago
Perhaps something like this? http://www.linuxfoundation.org/collaborate/workgroups/lsb http://www.linuxfoundation.org/collaborate/workgroups/lsb