4 ms·

> By now, HTML and the standards surrounding it have become so big that starting from a blank slate nowadays is next to impossible. It's a two sided coin: HTML

> By now, HTML and the standards surrounding it have become so big that starting from a blank slate nowadays is next to impossible. It's a two sided coin: HTML now defines a lot of the basic web platform parts in enough detail that you can just implement the spec and have a web browser that works, whereas in the 90s (and most of the 00s) browsers spent huge amounts of resources on reverse-engineering each other. Pretty much the only way to be web compatible then was to start with something already mostly web compatible (consider the fact that Apple's first two or three years of work on WebKit were essentially spent pretty much just fixing web compat bugs!).

Manishearth 10y ago
Well, we still do lots of reverse-engineering, because the specs are often wrong :)
- gsnedders 10y ago
  A lot is relative. :) Honestly, from what I see, the amount you do is relatively minuscule compared with what Opera was doing ten years ago, while Opera had far more marketshare which is normally the best way to avoid web compat issues in the first place.
- nickpsecurity 10y ago
  It's a rule of thumb in most software I found studying high-assurance. Inevitably, some academic or commercial team wants to robustly implement some standard (esp a protocol). They notice it's specified in a combo of English and implementation code. They start doing formal specifications of English spec. Every time, IIRC, they find various inconsistencies and such that already did or could lead to real-world problems. Showed me the value of formal specs but also important lesson: using informal specs will almost always require reverse engineering and extensive testing due to, at very least, English ambiguity or implementation deviations.
  - Manishearth 10y ago
    So most of the spec bugs I've come across don't have to do with English vagaries; they have to do with the reverse-engineering not being perfect. Converting the nebulous concept of expected behavior into procedural text is hard, and there are bugs. So yeah, "implementation deviations" is usually the issue, but not really English ambiguity, at least with web specs. I put my pedant hat on when looking at proposed changes to web specs, and I believe the rest of the community does too :)
    - nickpsecurity 10y ago
      Interesting. Thanks for the perspective.
    - gsnedders 10y ago
      There have certainly been spec bugs that could be found from a formal specification, some of which are security related (most of the TLS bugs found by miTLS have been protocol bugs simply found as a result of encoding much of the semantics formally). That said, while I certainly have a fondness for formal specifications, I also always end up feeling like they have a lot of downsides. They're often very verbose and hard to skim, and they're also another language that anyone implementing the specification has to learn prior to implementing it (or they can just wing it, but then your formal specification is worth nothing!).
      - nickpsecurity 10y ago
        Re specs downsides The wisdom from old days was to use relatively, simple, formal languages that were easy to train people on and use. B, Z, and TLA had good results plus lasted over time. The other thing was to do English spec, formal spec, and code side-by-side with each team looking for inconsistencies. The developers usually picked up the basics of the technique rather quickly but at least one specialist was always necessary to make first use go smoothly. So, that's still my recommendation. A subset of those, especially focusing on states and contracts, done in same fashion against English specs and clear code should take lower work than going all Isabelle, NuPRL, or PVS on a problem. Second application is also always easier. Reuse where possible helps as with code. So, some difficulties and training but probably easier than some frameworks, etc IT people learn today.
larsberg 10y ago
> you can just implement the spec and have a web browser that works Don't forget the amazing work being done at http://testthewebforward.org/ http://testthewebforward.org/ with the Web Platform Tests and your own work on the CSSWG tests! Beyond the specs, having a cross-browser set of tests that have especially good coverage on newer specs has been a huge leg-up for Servo! It not only helps us ensure interop for those features that are tested, but (puts on former dev manager hat) it really helps quantify how many work items are left until a feature is closer to completion. I especially notice it when there are areas like CSS 2.1 where the tests are less comprehensive and it's harder to quantify what our progress/status is.
- Manishearth 10y ago
  So much this. These tests are super useful. They don't just test for the spec, they also test for known behavior, which makes it possible to find spec bugs. Plus they contain a very good framework which we can easily add our own tests into, instead of rewriting an HTTP testing server and whatnot.
- gsnedders 10y ago
  To be fair, having tests somewhat relies on having a spec. OK, so in theory we could've had a cross-browser set of tests with some requirement along the lines of "this web page relies on this behaviour", but ultimately you'd still end up with contradictory tests. :) I wonder if it's worthwhile to spend some time working on the 2.1 testsuite to try and improve coverage at some point. Probably? Unlikely to find so many bugs in mature implementations, hence why so much effort has been spent elsewhere.
  - jgraham 10y ago
    <vested interest hat on> For too long it was considered OK to have a spec with either no tests, or tests that focused on the wrong thing ("is it possible to implement this spec" vs "do implementations behave in compatible ways"). It is still considered too OK for browser vendors to write their own tests for a feature and not share them with the rest of the community. However, pressure from other platforms is changing attitudes here as it becomes increasingly apparent that without a stronger degree of interoperability, developers will vote with their feet and target platforms with fewer bugs to trip over.

Manishearth 10y ago

Well, we still do lots of reverse-engineering, because the specs are often wrong :)

gsnedders 10y ago
A lot is relative. :) Honestly, from what I see, the amount you do is relatively minuscule compared with what Opera was doing ten years ago, while Opera had far more marketshare which is normally the best way to avoid web compat issues in the first place.
nickpsecurity 10y ago
It's a rule of thumb in most software I found studying high-assurance. Inevitably, some academic or commercial team wants to robustly implement some standard (esp a protocol). They notice it's specified in a combo of English and implementation code. They start doing formal specifications of English spec. Every time, IIRC, they find various inconsistencies and such that already did or could lead to real-world problems. Showed me the value of formal specs but also important lesson: using informal specs will almost always require reverse engineering and extensive testing due to, at very least, English ambiguity or implementation deviations.
- Manishearth 10y ago
  So most of the spec bugs I've come across don't have to do with English vagaries; they have to do with the reverse-engineering not being perfect. Converting the nebulous concept of expected behavior into procedural text is hard, and there are bugs. So yeah, "implementation deviations" is usually the issue, but not really English ambiguity, at least with web specs. I put my pedant hat on when looking at proposed changes to web specs, and I believe the rest of the community does too :)
  - nickpsecurity 10y ago
    Interesting. Thanks for the perspective.
  - gsnedders 10y ago
    There have certainly been spec bugs that could be found from a formal specification, some of which are security related (most of the TLS bugs found by miTLS have been protocol bugs simply found as a result of encoding much of the semantics formally). That said, while I certainly have a fondness for formal specifications, I also always end up feeling like they have a lot of downsides. They're often very verbose and hard to skim, and they're also another language that anyone implementing the specification has to learn prior to implementing it (or they can just wing it, but then your formal specification is worth nothing!).
    - nickpsecurity 10y ago
      Re specs downsides The wisdom from old days was to use relatively, simple, formal languages that were easy to train people on and use. B, Z, and TLA had good results plus lasted over time. The other thing was to do English spec, formal spec, and code side-by-side with each team looking for inconsistencies. The developers usually picked up the basics of the technique rather quickly but at least one specialist was always necessary to make first use go smoothly. So, that's still my recommendation. A subset of those, especially focusing on states and contracts, done in same fashion against English specs and clear code should take lower work than going all Isabelle, NuPRL, or PVS on a problem. Second application is also always easier. Reuse where possible helps as with code. So, some difficulties and training but probably easier than some frameworks, etc IT people learn today.

larsberg 10y ago

> you can just implement the spec and have a web browser that works Don't forget the amazing work being done at http://testthewebforward.org/ http://testthewebforward.org/ with the Web Platform Tests and your own work on the CSSWG tests! Beyond the specs, having a cross-browser set of tests that have especially good coverage on newer specs has been a huge leg-up for Servo! It not only helps us ensure interop for those features that are tested, but (puts on former dev manager hat) it really helps quantify how many work items are left until a feature is closer to completion. I especially notice it when there are areas like CSS 2.1 where the tests are less comprehensive and it's harder to quantify what our progress/status is.

Manishearth 10y ago
So much this. These tests are super useful. They don't just test for the spec, they also test for known behavior, which makes it possible to find spec bugs. Plus they contain a very good framework which we can easily add our own tests into, instead of rewriting an HTTP testing server and whatnot.
gsnedders 10y ago
To be fair, having tests somewhat relies on having a spec. OK, so in theory we could've had a cross-browser set of tests with some requirement along the lines of "this web page relies on this behaviour", but ultimately you'd still end up with contradictory tests. :) I wonder if it's worthwhile to spend some time working on the 2.1 testsuite to try and improve coverage at some point. Probably? Unlikely to find so many bugs in mature implementations, hence why so much effort has been spent elsewhere.
- jgraham 10y ago
  <vested interest hat on> For too long it was considered OK to have a spec with either no tests, or tests that focused on the wrong thing ("is it possible to implement this spec" vs "do implementations behave in compatible ways"). It is still considered too OK for browser vendors to write their own tests for a feature and not share them with the rest of the community. However, pressure from other platforms is changing attitudes here as it becomes increasingly apparent that without a stronger degree of interoperability, developers will vote with their feet and target platforms with fewer bugs to trip over.