3 ms·

So I agree that there is a great deal of accidental complexity associated with filing bugs, and browser developers could certainly do better here. I also see th

by jgraham 13y ago

So I agree that there is a great deal of accidental complexity associated with filing bugs, and browser developers could certainly do better here. I also see that there is a certain disconnect between the perceptions of a browser developer/QA, for whom "stable" version means "outdated version" and those of a normal user. On the other hand, if you do web development work professionally (and I appreciate that you personally might not), you should seriously consider running at least the beta versions of browsers on a day-to-day basis, and consider testing in something even newer. If you don't, it's much more likely that a browser change/bug will regress your site for tens or hundreds of millions of users. Having said that, I would much rather have a bug report based on an outdated — sorry, stable — version of the browser than no bug report at all. As for the rapid release cycle, my feeling is that process dramatically decreases the bug count. It forces everyone to keep the code in a roughly shippable condition all the time, and provides early feedback if a problem is introduced. I don't have data at hand to backup this hypothesis, but I wouldn't be surprised if someone at Mozilla/Google did. It is, however, unequivocally false that no one cares about the quality of browsers or that all the focus is on features. Browser vendors have suites of hundreds of thousands of automated tests that every commit must pass. They require tests to accompany code changes. There is a big emphasis on correctness in initial implementation and on maintaining that quality through subsequent releases. However, as you note, this doesn't always go to plan. But submitting tests to the W3C is one way to help make things better. Not only will these automatically get picked up by vendors for their automated test systems, as time goes on we expect it to become more visible who is passing what tests. So there will be a pressure to compete on correctness as well as performance. Of course much like performance benchmarks, if test conformance rates become a big visible thing, there is a temptation to start submitting large numbers of tests for narrow areas where you know that you do well and the competition does badly. Also, not all tests are equally important. So some care is needed in the way that conformance data is presented to prevent people jumping to conclusions from the fact that Firefox 132 scores 89% on the DOM MindControl API but Chrome 140 gets only 87%. Nevertheless, especially for professional web developers, submitting test cases for browser bugs is one of the most effective things that you can do both to get the bug fixed and to advance the state of the web platform.

Silhouette 13y ago
On the other hand, if you do web development work professionally (and I appreciate that you personally might not), you should seriously consider running at least the beta versions of browsers on a day-to-day basis, and consider testing in something even newer. If you don't, it's much more likely that a browser change/bug will regress your site for tens or hundreds of millions of users. Again, I appreciate what you're saying, and it seems a reasonable request. However, I think it's unrealistic for a lot of developers. For example, I work on multiple web-based projects in various capacities. Right now, just to test changes on one of my sites before releasing them, I run manual tests on about half a dozen desktop browsers and several mobile devices. (Automated testing is great, but there is only so far you can get with scripting, particularly when it comes to things like rendering glitches and responsiveness.) If I put a change live on one of my sites, or worse, hand it over to a client or ship a web app embedded in network-enabled hardware, I shouldn't have to be constantly checking for blatant regressions in basic functionality every few weeks. In fact, I didn't have to, until the teams behind various popular browsers and plug-ins decided that the kind of major/minor/point releases the entire industry had been using successfully for decades weren't sufficient and pushing out whatever was ready at an arbitrary six-weekly interval was a better idea. As for the rapid release cycle, my feeling is that process dramatically decreases the bug count. It forces everyone to keep the code in a roughly shippable condition all the time, and provides early feedback if a problem is introduced. That's an admirable intent, but it clearly doesn't always work in practice. For a vast and complicated piece of software like a modern web browser, slipping in an occasional regression in a corner case is all but inevitable, but there have been your-browser-doesn't-work-at-all scale howlers that have been pushed out sometimes too. I suspect what this really exposes is an over-reliance on automated tests and a false sense of security if all of the tests pass. With development at this kind of pace, and with the distributed and partly volunteer-led development model for these browsers, perhaps there just isn't time for more comprehensive reviews and high-level planning/architectural work and so on, but there is a reason those things have value. As for decreasing the bug count, I fear that what is actually happening is that there are so many now and the target moves so fast that many of us have just given up on reporting them. Certainly for our project bug trackers here, Chrome used to be worse by a factor of several for browser bug issues, but Firefox spiked after going to rapid releases and has held the dubious honour of being the worst major browser for quality more often than not since that time. Several of these projects use modern but mainstream features from HTML5 and CSS3, but I'm not talking about bleeding edge or obscure features here. It is, however, unequivocally false that no one cares about the quality of browsers or that all the focus is on features. Please notice that I was very careful not to claim that. I do, however, feel that as a matter of priorities, quality of implementation often takes a back seat to ticking feature boxes and to performance at the moment. Again, this is based directly off our own bug trackers. I could have described canvas and SVG issues not far from the kind of things simonsarris mentioned earlier in this thread, for example, and others in CSS3 backgrounds and borders, CSS3 animation, HTML5 media elements, and so on. In contrast, while the pace of development for IE is slower, it has far fewer blunder-type issues in basic functionality these days, in our (obviously anecdotal) experience. As a final comment, I'm interested in these W3C tests you've mentioned a couple of times. I had no knowledge of anything like that before this thread. If you can share some sort of authoritative/definitive link to how that works, I'd appreciate that, as it sounds like I should learn about it.

by jgraham 13y ago

Silhouette 13y ago

On the other hand, if you do web development work professionally (and I appreciate that you personally might not), you should seriously consider running at least the beta versions of browsers on a day-to-day basis, and consider testing in something even newer. If you don't, it's much more likely that a browser change/bug will regress your site for tens or hundreds of millions of users. Again, I appreciate what you're saying, and it seems a reasonable request. However, I think it's unrealistic for a lot of developers. For example, I work on multiple web-based projects in various capacities. Right now, just to test changes on one of my sites before releasing them, I run manual tests on about half a dozen desktop browsers and several mobile devices. (Automated testing is great, but there is only so far you can get with scripting, particularly when it comes to things like rendering glitches and responsiveness.) If I put a change live on one of my sites, or worse, hand it over to a client or ship a web app embedded in network-enabled hardware, I shouldn't have to be constantly checking for blatant regressions in basic functionality every few weeks. In fact, I didn't have to, until the teams behind various popular browsers and plug-ins decided that the kind of major/minor/point releases the entire industry had been using successfully for decades weren't sufficient and pushing out whatever was ready at an arbitrary six-weekly interval was a better idea. As for the rapid release cycle, my feeling is that process dramatically decreases the bug count. It forces everyone to keep the code in a roughly shippable condition all the time, and provides early feedback if a problem is introduced. That's an admirable intent, but it clearly doesn't always work in practice. For a vast and complicated piece of software like a modern web browser, slipping in an occasional regression in a corner case is all but inevitable, but there have been your-browser-doesn't-work-at-all scale howlers that have been pushed out sometimes too. I suspect what this really exposes is an over-reliance on automated tests and a false sense of security if all of the tests pass. With development at this kind of pace, and with the distributed and partly volunteer-led development model for these browsers, perhaps there just isn't time for more comprehensive reviews and high-level planning/architectural work and so on, but there is a reason those things have value. As for decreasing the bug count, I fear that what is actually happening is that there are so many now and the target moves so fast that many of us have just given up on reporting them. Certainly for our project bug trackers here, Chrome used to be worse by a factor of several for browser bug issues, but Firefox spiked after going to rapid releases and has held the dubious honour of being the worst major browser for quality more often than not since that time. Several of these projects use modern but mainstream features from HTML5 and CSS3, but I'm not talking about bleeding edge or obscure features here. It is, however, unequivocally false that no one cares about the quality of browsers or that all the focus is on features. Please notice that I was very careful not to claim that. I do, however, feel that as a matter of priorities, quality of implementation often takes a back seat to ticking feature boxes and to performance at the moment. Again, this is based directly off our own bug trackers. I could have described canvas and SVG issues not far from the kind of things simonsarris mentioned earlier in this thread, for example, and others in CSS3 backgrounds and borders, CSS3 animation, HTML5 media elements, and so on. In contrast, while the pace of development for IE is slower, it has far fewer blunder-type issues in basic functionality these days, in our (obviously anecdotal) experience. As a final comment, I'm interested in these W3C tests you've mentioned a couple of times. I had no knowledge of anything like that before this thread. If you can share some sort of authoritative/definitive link to how that works, I'd appreciate that, as it sounds like I should learn about it.