5 ms·

> How is a web developer supposed to test the website when he and user might have different browser behaviour? So that is always going to be the case with a ch

by arp242 2y ago

> How is a web developer supposed to test the website when he and user might have different browser behaviour? So that is always going to be the case with a change like this, simply because people use older versions, use different browsers, etc. If this change breaks popular-site.com then it will continue work fine for 95% of people, and it breaks for "only" 5%, one of whom will (hopefully) report it. This allows the Firefox people to test the waters to make sure it's not going to horrible break things and break things for too many people.

lynx97 2y ago
Yay, "our users are our beta-testers" hard at work. How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? I agree with parent, this is an awful approach.
- anon7000 2y ago
  The scale is far too big. You cannot take into account the millions of permutations of websites and browser extensions when testing changes like this.
- frosted-flakes 2y ago
  They're not testing the browser, they're testing the Web. Testing in the real world with real users is the only way to adequately probe the effects of potentially breaking changes like this. This is not new either. Many such changes have been reverted after discovering that they broke more things than expected. One example is the Object.groupBy static method, which was initially Array.prototype.group.
  - lynx97 2y ago
    > They're not testing the browser, they're testing the Web. That sounds like a pretty lame excuse. So if a self-driving car manufacturer does testing, they are not testing the car, they are testing the environment? Sounds like a pretty neat trick, maybe marketing should adopt this attitude.
- arp242 2y ago
  You can't test against the entire web, especially internationally. Do you know what the most popular Vietnamese sites are? I don't either. Do that for >200 languages. Never mind of course there's tons of non-public stuff.
  - bryanrasmussen 2y ago
    [flagged]
    - tomalbrc 2y ago
      1. Your “could” is nonsense. 2. Better not to get salty about some downvotes
      - bryanrasmussen 2y ago
        >1. Your “could” is nonsense. quote: Do you know what the most popular Vietnamese sites are? I don't either. Do that for >200 languages. was the example given, it's true I do not know what the most popular Vietnamese sites are, and the person who posted that and pretty much anyone on this site does not know what the most popular Vietnamese sites are or the same thing for 200 plus languages, but there are a few companies out there for which finding that out is pretty much child's play. One of those companies makes the Chrome browser. Also testing against the entire web, obviously asking people to report problems is not really testing against the entire web, it is testing against a subset of the web. This should not really need to be pointed out to the H part of HN, but here I am. A similar thing would be if you had a big index of the web and you tested against that. >2. Better not to get salty about some downvotes your and my definitions about saltiness seem to vary a great deal. on edit: Noting really that the thing to determine is how many sites actually use the UA styles for H1 instead of overriding, which would also be child's play for Google, and determining the most popular sites or pages that do so. I believe that the sites that do so are few and far between and not exceptionally popular, as well as probably very simply styled, if this belief is true it would also be relatively simple for someone to figure out what side effects were likely - if they had the data and processing power to do so. But Mozilla does not, therefore they must ask people to tell them if they have problems.
- eru 2y ago
  > How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? Presumably they have done as much of that evaluation as is reasonable already.
  - ozim 2y ago
    Second that - I assume that they did asnmuch as they could and this is just defense in depth one additional step.
    - codedokode 2y ago
      That is not defence in depth, that is causing annoyance to web developers.
  - joquarky 2y ago
    In my experience, that's probably not a good presumption anymore. It feels to me like usability testing and quality assurance have both joined security and accessibility on the back burner now.
- hulitu 2y ago
  > How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? "Who would pay for it ?" Sincerely, Microsoft™
- mvdtnz 2y ago
  Yeah why didn't the developer simply test every website on the internet.
- zcorpan 2y ago
  Some testing has already happened before starting the rollout. The change has been shipping in Firefox Nightly for a year. I have analyzed impact of affected pages in the HTTP Archive dataset (about 12,000,000 pages), twice: https://github.com/whatwg/html/issues/7867#issuecomment-1977647444 https://github.com/whatwg/html/issues/7867#issuecomment-1977... https://github.com/whatwg/html/issues/7867#issuecomment-2595987424 https://github.com/whatwg/html/issues/7867#issuecomment-2595... The next step is either rollout or ship directly.
- chuckadams 2y ago
  > Yay, "our users are our beta-testers" hard at work. That is kind of part of the bargain when you download a version of the browser explicitly labeled as a beta.
selcuka 2y ago
> simply because people use older versions, use different browsers, etc. Yes, but in this case asking the user to update to the latest version won't fix the problem either. I agree it's a terrible idea.
- aarond0623 2y ago
  You would direct the user to submit a bug to Firefox if the latest version doesn't fix the issue. That's how they learn if this change causes issues.
  - codedokode 2y ago
    You should first investigate the cause of the bug before spamming Firefox developers with potentially invalid reports. And it is difficult to investigate when your browser and user's browser behave different.
- chuckadams 2y ago
  It's a beta version. Betas don't roll out evenly. Asking the user to update to a beta version has never been a good idea.
  - selcuka 2y ago
    They are doing it for the stable release as well: > The plan is to roll out to 5% of users on the Firefox 138 stable release, ramp up to 50% of users, then ship on all platforms in Firefox 140.
    - chuckadams 2y ago
      Yeah that makes no sense. Versions are meaningless if they don't represent standard defaults. But then, evergreen software makes versions something like an implementation detail anyway.
827a 2y ago
Remember: The people who would supposed to benefit from a rollout like this are the website operators, not Firefox. The change is happening; I doubt Mozilla is going to reverse course because a couple websites look a little wonky (totally reasonable). How would they report it? "Hey, your website looks weird. Yeah I'm running Firefox 138 stable and I'm in the 5% experimental group who received the default h1 styling change"? What is more likely to happen is: A website might get a report, the dev goes to reproduce it, and they're in the control group, so they can't; and because it was only one or two reports, it gets closed.
- zcorpan 2y ago
  Firefox also has a built-in "broken website" reporter in the Help menu, which sends a report to Mozilla. The WebCompat team analyzes those reports and can decide to report a Gecko bug if it has a bug, contact the website if they have a bug, or apply a sitepatch. We're also monitoring the broken website reports for regressions from this change in particular. I understand that a rollout makes reproduction harder, and not everyone will be aware of the change. It's a tradeoff when deciding whether to do a rollout or ride the trains normally.
- madeofpalk 2y ago
  > I doubt Mozilla is going to reverse course because a couple websites look a little wonky (totally reasonable). This is totally possible. Many changes to the web platform we abandoned or changed significantly because a couple of websites look a little wonky.
crumpled 2y ago
Usually we can use feature flags. Browser makers also have beta versions or their browsers available for devs.
piokoch 2y ago
Even Microsoft managed to figure that out and if Teams or Outlook get new look, there is an option to use it or switch back to the old one.

by arp242 2y ago

lynx97 2y ago

Yay, "our users are our beta-testers" hard at work. How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? I agree with parent, this is an awful approach.

anon7000 2y ago
The scale is far too big. You cannot take into account the millions of permutations of websites and browser extensions when testing changes like this.
frosted-flakes 2y ago
They're not testing the browser, they're testing the Web. Testing in the real world with real users is the only way to adequately probe the effects of potentially breaking changes like this. This is not new either. Many such changes have been reverted after discovering that they broke more things than expected. One example is the Object.groupBy static method, which was initially Array.prototype.group.
- lynx97 2y ago
  > They're not testing the browser, they're testing the Web. That sounds like a pretty lame excuse. So if a self-driving car manufacturer does testing, they are not testing the car, they are testing the environment? Sounds like a pretty neat trick, maybe marketing should adopt this attitude.
arp242 2y ago
You can't test against the entire web, especially internationally. Do you know what the most popular Vietnamese sites are? I don't either. Do that for >200 languages. Never mind of course there's tons of non-public stuff.
- bryanrasmussen 2y ago
  [flagged]
  - tomalbrc 2y ago
    1. Your “could” is nonsense. 2. Better not to get salty about some downvotes
    - bryanrasmussen 2y ago
      >1. Your “could” is nonsense. quote: Do you know what the most popular Vietnamese sites are? I don't either. Do that for >200 languages. was the example given, it's true I do not know what the most popular Vietnamese sites are, and the person who posted that and pretty much anyone on this site does not know what the most popular Vietnamese sites are or the same thing for 200 plus languages, but there are a few companies out there for which finding that out is pretty much child's play. One of those companies makes the Chrome browser. Also testing against the entire web, obviously asking people to report problems is not really testing against the entire web, it is testing against a subset of the web. This should not really need to be pointed out to the H part of HN, but here I am. A similar thing would be if you had a big index of the web and you tested against that. >2. Better not to get salty about some downvotes your and my definitions about saltiness seem to vary a great deal. on edit: Noting really that the thing to determine is how many sites actually use the UA styles for H1 instead of overriding, which would also be child's play for Google, and determining the most popular sites or pages that do so. I believe that the sites that do so are few and far between and not exceptionally popular, as well as probably very simply styled, if this belief is true it would also be relatively simple for someone to figure out what side effects were likely - if they had the data and processing power to do so. But Mozilla does not, therefore they must ask people to tell them if they have problems.
eru 2y ago
> How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? Presumably they have done as much of that evaluation as is reasonable already.
- ozim 2y ago
  Second that - I assume that they did asnmuch as they could and this is just defense in depth one additional step.
  - codedokode 2y ago
    That is not defence in depth, that is causing annoyance to web developers.
- joquarky 2y ago
  In my experience, that's probably not a good presumption anymore. It feels to me like usability testing and quality assurance have both joined security and accessibility on the back burner now.
hulitu 2y ago
> How about devs evaluate a change by doing some testing, instead of using users as guinea pigs? "Who would pay for it ?" Sincerely, Microsoft™
mvdtnz 2y ago
Yeah why didn't the developer simply test every website on the internet.
zcorpan 2y ago
Some testing has already happened before starting the rollout. The change has been shipping in Firefox Nightly for a year. I have analyzed impact of affected pages in the HTTP Archive dataset (about 12,000,000 pages), twice: https://github.com/whatwg/html/issues/7867#issuecomment-1977647444 https://github.com/whatwg/html/issues/7867#issuecomment-1977... https://github.com/whatwg/html/issues/7867#issuecomment-2595987424 https://github.com/whatwg/html/issues/7867#issuecomment-2595... The next step is either rollout or ship directly.
chuckadams 2y ago
> Yay, "our users are our beta-testers" hard at work. That is kind of part of the bargain when you download a version of the browser explicitly labeled as a beta.

selcuka 2y ago

> simply because people use older versions, use different browsers, etc. Yes, but in this case asking the user to update to the latest version won't fix the problem either. I agree it's a terrible idea.

aarond0623 2y ago
You would direct the user to submit a bug to Firefox if the latest version doesn't fix the issue. That's how they learn if this change causes issues.
- codedokode 2y ago
  You should first investigate the cause of the bug before spamming Firefox developers with potentially invalid reports. And it is difficult to investigate when your browser and user's browser behave different.
chuckadams 2y ago
It's a beta version. Betas don't roll out evenly. Asking the user to update to a beta version has never been a good idea.
- selcuka 2y ago
  They are doing it for the stable release as well: > The plan is to roll out to 5% of users on the Firefox 138 stable release, ramp up to 50% of users, then ship on all platforms in Firefox 140.
  - chuckadams 2y ago
    Yeah that makes no sense. Versions are meaningless if they don't represent standard defaults. But then, evergreen software makes versions something like an implementation detail anyway.

827a 2y ago

Remember: The people who would supposed to benefit from a rollout like this are the website operators, not Firefox. The change is happening; I doubt Mozilla is going to reverse course because a couple websites look a little wonky (totally reasonable). How would they report it? "Hey, your website looks weird. Yeah I'm running Firefox 138 stable and I'm in the 5% experimental group who received the default h1 styling change"? What is more likely to happen is: A website might get a report, the dev goes to reproduce it, and they're in the control group, so they can't; and because it was only one or two reports, it gets closed.

zcorpan 2y ago
Firefox also has a built-in "broken website" reporter in the Help menu, which sends a report to Mozilla. The WebCompat team analyzes those reports and can decide to report a Gecko bug if it has a bug, contact the website if they have a bug, or apply a sitepatch. We're also monitoring the broken website reports for regressions from this change in particular. I understand that a rollout makes reproduction harder, and not everyone will be aware of the change. It's a tradeoff when deciding whether to do a rollout or ride the trains normally.
madeofpalk 2y ago
> I doubt Mozilla is going to reverse course because a couple websites look a little wonky (totally reasonable). This is totally possible. Many changes to the web platform we abandoned or changed significantly because a couple of websites look a little wonky.

crumpled 2y ago

Usually we can use feature flags. Browser makers also have beta versions or their browsers available for devs.

piokoch 2y ago

Even Microsoft managed to figure that out and if Teams or Outlook get new look, there is an option to use it or switch back to the old one.