4 ms·
The Acid tests were obviously great at the time, but in retrospect some of the choices were not ideal. The concept of having a single page that only renders cor
by jgraham 5y ago
The Acid tests were obviously great at the time, but in retrospect some of the choices were not ideal. The concept of having a single page that only renders correctly when all the relevant conditions are met makes it hard to write broad tests for the feature; the Acid tests often depended on deep edge cases of one aspect of a feature, but lacked coverage of other aspects of the same feature. There was also an element of function-follows-form; the requirement to render an overall memorable image in the passing case often meant that the tests themselves were not well isolated, and developers working on the bugs would have to spend considerable time producing reduced test cases. For Acid 3 in particular, the need for 100 exactly tests meant that some features were given extremely shallow coverage, and the general hype around being the first implementation to pass an Acid test encouraged vendors to do the minimum necessary to make the test pass, rather than providing a useful implementation of the feature.
In the time since the Acid tests, there has been much more focus on building up the process for sharing tests as a matter of course in browser development; web-platform-tests is the result of that effort. Interop 2022 adds some more public accountability on top of web-platform-tests, which hopefully creates some of the same incentives as Acid tests for everyone to get the same set of features above the bar where they're no longer providing problems for web developers or end users. But it's intentionally set up to use a library of many smaller tests, and be paced as a marathon rather than a sprint, in the hope of avoiding some of the pitfalls of the Acid-test approach.