3 ms·

> you can just implement the spec and have a web browser that works Don't forget the amazing work being done at http://testthewebforward.org/ http://testtheweb

> you can just implement the spec and have a web browser that works Don't forget the amazing work being done at http://testthewebforward.org/ http://testthewebforward.org/ with the Web Platform Tests and your own work on the CSSWG tests! Beyond the specs, having a cross-browser set of tests that have especially good coverage on newer specs has been a huge leg-up for Servo! It not only helps us ensure interop for those features that are tested, but (puts on former dev manager hat) it really helps quantify how many work items are left until a feature is closer to completion. I especially notice it when there are areas like CSS 2.1 where the tests are less comprehensive and it's harder to quantify what our progress/status is.

Manishearth 10y ago
So much this. These tests are super useful. They don't just test for the spec, they also test for known behavior, which makes it possible to find spec bugs. Plus they contain a very good framework which we can easily add our own tests into, instead of rewriting an HTTP testing server and whatnot.
gsnedders 10y ago
To be fair, having tests somewhat relies on having a spec. OK, so in theory we could've had a cross-browser set of tests with some requirement along the lines of "this web page relies on this behaviour", but ultimately you'd still end up with contradictory tests. :) I wonder if it's worthwhile to spend some time working on the 2.1 testsuite to try and improve coverage at some point. Probably? Unlikely to find so many bugs in mature implementations, hence why so much effort has been spent elsewhere.
- jgraham 10y ago
  <vested interest hat on> For too long it was considered OK to have a spec with either no tests, or tests that focused on the wrong thing ("is it possible to implement this spec" vs "do implementations behave in compatible ways"). It is still considered too OK for browser vendors to write their own tests for a feature and not share them with the rest of the community. However, pressure from other platforms is changing attitudes here as it becomes increasingly apparent that without a stronger degree of interoperability, developers will vote with their feet and target platforms with fewer bugs to trip over.
- larsberg 10y ago
  I think it's worthwhile! Lack of a comprehensive test suite means: 1) There are seem to still be regressions in pretty fundamental stuff when new features are added. Usually caught before things reach "release" trains, but I see it in bug trackers all the time. 2) Other layout engine managers I've talked with who want to do a layout system rewrite are terrified of anything that doesn't look like a large-scale refactoring because of this issue. I've had the "how are you testing and measuring progress / how do you know you're done with layout?" conversation with most of our engine peers at this point :-)

Manishearth 10y ago

So much this. These tests are super useful. They don't just test for the spec, they also test for known behavior, which makes it possible to find spec bugs. Plus they contain a very good framework which we can easily add our own tests into, instead of rewriting an HTTP testing server and whatnot.

gsnedders 10y ago

To be fair, having tests somewhat relies on having a spec. OK, so in theory we could've had a cross-browser set of tests with some requirement along the lines of "this web page relies on this behaviour", but ultimately you'd still end up with contradictory tests. :) I wonder if it's worthwhile to spend some time working on the 2.1 testsuite to try and improve coverage at some point. Probably? Unlikely to find so many bugs in mature implementations, hence why so much effort has been spent elsewhere.

jgraham 10y ago
<vested interest hat on> For too long it was considered OK to have a spec with either no tests, or tests that focused on the wrong thing ("is it possible to implement this spec" vs "do implementations behave in compatible ways"). It is still considered too OK for browser vendors to write their own tests for a feature and not share them with the rest of the community. However, pressure from other platforms is changing attitudes here as it becomes increasingly apparent that without a stronger degree of interoperability, developers will vote with their feet and target platforms with fewer bugs to trip over.
larsberg 10y ago
I think it's worthwhile! Lack of a comprehensive test suite means: 1) There are seem to still be regressions in pretty fundamental stuff when new features are added. Usually caught before things reach "release" trains, but I see it in bug trackers all the time. 2) Other layout engine managers I've talked with who want to do a layout system rewrite are terrified of anything that doesn't look like a large-scale refactoring because of this issue. I've had the "how are you testing and measuring progress / how do you know you're done with layout?" conversation with most of our engine peers at this point :-)