5 ms·

I've removed some feature from the web in Chrome after a long deprecation phase with warnings in devtools, proper announcements in ALL the relevant mailing list

by Orphis 5y ago

I've removed some feature from the web in Chrome after a long deprecation phase with warnings in devtools, proper announcements in ALL the relevant mailing lists and release notes. Still, many major websites broke as they failed to implement the very simple required changes in their products. Did they break in Chrome? No, we landed the change at a later date than announced. But Firefox did the same removal and it landed to stable a few weeks before Chrome. They also advertised for the removal extensively and were hit by all the "Major site X and Y are broken" bugs unfortunately. A couple days later, everything was fixed though, when Chrome's side of thing reached stable, no sites were really impacted. The replacement API had been available for many years before, people used the old non-standard and deprecated way still. So no, websites owner will not be proactive and fix their products no matter what you do unless they break. We do our best to be upfront with the changes, but both sides have to be willing to communicate for it to happen.

danShumway 5y ago
> We do our best to be upfront with the changes, but both sides have to be willing to communicate for it to happen. What processes did Chrome follow in this case to identify sites that might break, and did it reach out directly to any of the people who would be affected? I'm seeing comments from people like Chris Coyier that they were caught off guard with this change: https://twitr.gq/chriscoyier/status/1420027533005836298#m https://twitr.gq/chriscoyier/status/1420027533005836298#m If the creator of CodePen didn't realize this was coming, and if it didn't register to anyone in charge of this change that he should be contacted about it and included in the discussion before late July, then to me it really doesn't seem like the communication breakdown here is happening because of website developers.
- Orphis 5y ago
  There are archives you can sample to measure the impact of a change to some degree. We can use them to have an estimation of how common some patterns are. It's tricky to query so it may not work for advanced use cases or identify cross origin issues. And it's not possible to reach out directly to those people. We can't just send deprecation notices to random contact emails (if there's any) on websites. We can't just open a "support request" to L1 customer support of websites telling them their site might not work soon. We're not going to send dead-tree mail to addresses listed on websites either. If you want to support specific browsers, as part of your ongoing maintenance, you should test your site with canary and beta versions of browsers to catch issues before they go live at a minimum. You can also read the relevant mailing lists or blog posts for the major browsers to have an idea of the direction they're taking. Chrome 91 went stable July 15. But the first beta was April 22. The change was probably in Canary before that as well. Quite a long time to catch issues with the right automation. Personally, I wouldn't want to have to read the mailing lists and other articles. I would want a dashboard that tells me everything is tested and working fine automatically. And when it doesn't, I know I have many months to fix the issue before it hits stable and find the relevant documentation about the changes that might be causing the issues. Or just open an issue to the browsers, sometimes people are using APIs in very unexpected ways, or you know, a bug sneaked in despite the testing we have in place.
  - danShumway 5y ago
    > And it's not possible to reach out directly to those people. We can't just send deprecation notices to random contact emails (if there's any) on websites. To be clear, we're talking about Chris Coyier right now. The Chrome team doesn't have any way to contact him? The team had to know that REPLs would be impacted by this, right? You can blame people for not having the right automation and dev tools lined up and for not reading the right blogs, but it still seems to me like this is covering up the fact that the Chrome team does not view it as their job to reach out to web developers or put serious effort into marketing changes. If Chris didn't know about these changes, then I seriously doubt that the Chrome team was getting any kind of a representative sample of developer feedback during this process. > Personally, I wouldn't want to have to read the mailing lists and other articles. I would want a dashboard that tells me everything is tested and working fine automatically. What view of the web does the Chrome team have if they think the majority of site operators have this kind of testing setup running? It just feels so ridiculously out of touch with the reality of where web developers congregate and how news gets disseminated. I know it's a difficult problem, but the responsibility of Chrome is to work with the web community as it exists, not as the professional fully-automated community they'd like it to be. Chrome couldn't send out some Twitter DMs? Chrome team couldn't post a heads up to HN? Any effort to post anything to the popular web dev communities on Reddit? Did the team reach out to any any popular bloggers or members of the community to ask them to spread the word? When the Chrome team is making changes to the dominant browser for what is (imo) the most important platform on the planet, it just is not enough to say "we put it in the mailing list, and you weren't testing well enough." The expectations for the Chrome team here are higher, because Chrome matters more than other products. Yes, developers need to meet Chrome halfway on communication, but frankly, Chrome is not coming halfway right now to meet us. It should not have been a surprise to some of the biggest REPL sites on the web that their products were about to break. I don't know what to say about this, I'm not trying to be mean, but if the Chrome team's perspective is that developer communication isn't at least partially their problem to solve, and they think that if they roll out a feature and nobody knows about it that developers are the problem, then those people shouldn't be in charge of the dominant web browser.
    - Orphis 5y ago
svieira 5y ago
I read chromestatus every week or so and the intended landing date for this change caught me off guard.
- Orphis 5y ago
  But do you use Chrome Canary, Dev or Beta for development purposes? Would you have then noticed the change? Do you have continuous testing on the websites you maintain with those versions? I presume you're only human, so it's almost expected you will skip over a change as you don't think it's relevant or might even take days off work, in which case, it's easy to forget to read it. ChromeStatus is just one of the many signals you should use to filter issues before they reach customers.
  - svieira 5y ago
    Yes, and I used to use it as my daily driver. But I switched to Firefox a year ago to make sure I wasn't ignoring the minority experience. Yes, we have CI (700+ machine hours for runs-on-master integration tests). No, we don't have a test that would have caught this (we are an app building platform and some of our features which our users use to deliver features to their users use iframes)
gpm 5y ago
> proper announcements in ALL the relevant mailing lists and release notes. I can't emphasize enough that mailing lists are worthless for this. Your average web developer reads no mailing lists, I'd be surprised if even 5% did, and those 5% are going to be clustered. Release notes (that anyone reads) come too late, since people only read them once users are already installing the broken browser. Announcements that you actually expect to be read need to be in a much higher profile place, it looks like this is an appropriate blog: https://www.blog.google/products/chrome/ https://www.blog.google/products/chrome/ - even then it should be written with the goal of it being picked up by places like reddit/HN/twitter/news. Even then you should expect that most web developers won't find out, and most who do find out won't change anything in regards to your announcement that your new version of the browser is going to break compatibility with their perfectly functional website, incentives just won't align for many of them to fix it pre-emptively. That last part is why I advocate for indicating the issue in a user visible but non-breaking way first, that's going to be the only way to get funding/time to update to your new version for many websites.
- danShumway 5y ago
  I'll add onto this, putting an announcement out needs to be followed up by checking to see if the announcement has gained any traction. Announcing breaking changes is not a checklist that can be filled out and then ignored. Put out an announcement, see if any other sites or prominent developers pick up on it, if they haven't picked up on it then the job isn't done and the change needs to be advertised more. Maybe for some things posting it to the blog might be enough, and the major stakeholders that need to know will pick it up. Then you can move on to the next steps. But if they don't see it, and you don't see any reaction to the blog posts, then don't just move onto stage 2 because "they had their chance, if it breaks now it's their fault." :) Have some standards of what level of conversation and preparation you expect to see from web developers before you consider the community adequately informed.
- Orphis 5y ago
  Here's an example of a major bank website with some user visible non-breaking issue I saw recently: https://youtu.be/GFvaRgFf4LU?t=482 https://youtu.be/GFvaRgFf4LU?t=482 . It seems that it's been visible for a while, yet is not fixed. It's actually quite common to see errors unfixed in software as you may know. No matter what you do, some websites will only be fixed a long time after very obvious warnings are displayed. Some will never be fixed until they actually stops working. Some will just never be updated at all. Maybe you're one of the people who take their car to the repair shop when the engine light is on, but there are also many others who will just ignore it as the car still drives "fine" (for now at least). Regarding your suggestion of using Reddit, HN, Twitter, news, it's not always possible or relevant. I've seen a lot of very interesting publications just not getting any traction on those are they're boring in nature or cater to experts mainly. They're also read only by a fraction of users. Unfortunately, if you're hit by any issue, you shouldn't be only asking what others should do for you, but also what you can do to catch those issues in the future yourself. All sides can maybe do better at broadcasting information about some changes, or voice their opinion better and sooner, but it's not an easy problem with an easy solution.

by Orphis 5y ago

danShumway 5y ago

> We do our best to be upfront with the changes, but both sides have to be willing to communicate for it to happen. What processes did Chrome follow in this case to identify sites that might break, and did it reach out directly to any of the people who would be affected? I'm seeing comments from people like Chris Coyier that they were caught off guard with this change: https://twitr.gq/chriscoyier/status/1420027533005836298#m https://twitr.gq/chriscoyier/status/1420027533005836298#m If the creator of CodePen didn't realize this was coming, and if it didn't register to anyone in charge of this change that he should be contacted about it and included in the discussion before late July, then to me it really doesn't seem like the communication breakdown here is happening because of website developers.

Orphis 5y ago
There are archives you can sample to measure the impact of a change to some degree. We can use them to have an estimation of how common some patterns are. It's tricky to query so it may not work for advanced use cases or identify cross origin issues. And it's not possible to reach out directly to those people. We can't just send deprecation notices to random contact emails (if there's any) on websites. We can't just open a "support request" to L1 customer support of websites telling them their site might not work soon. We're not going to send dead-tree mail to addresses listed on websites either. If you want to support specific browsers, as part of your ongoing maintenance, you should test your site with canary and beta versions of browsers to catch issues before they go live at a minimum. You can also read the relevant mailing lists or blog posts for the major browsers to have an idea of the direction they're taking. Chrome 91 went stable July 15. But the first beta was April 22. The change was probably in Canary before that as well. Quite a long time to catch issues with the right automation. Personally, I wouldn't want to have to read the mailing lists and other articles. I would want a dashboard that tells me everything is tested and working fine automatically. And when it doesn't, I know I have many months to fix the issue before it hits stable and find the relevant documentation about the changes that might be causing the issues. Or just open an issue to the browsers, sometimes people are using APIs in very unexpected ways, or you know, a bug sneaked in despite the testing we have in place.
- danShumway 5y ago
  > And it's not possible to reach out directly to those people. We can't just send deprecation notices to random contact emails (if there's any) on websites. To be clear, we're talking about Chris Coyier right now. The Chrome team doesn't have any way to contact him? The team had to know that REPLs would be impacted by this, right? You can blame people for not having the right automation and dev tools lined up and for not reading the right blogs, but it still seems to me like this is covering up the fact that the Chrome team does not view it as their job to reach out to web developers or put serious effort into marketing changes. If Chris didn't know about these changes, then I seriously doubt that the Chrome team was getting any kind of a representative sample of developer feedback during this process. > Personally, I wouldn't want to have to read the mailing lists and other articles. I would want a dashboard that tells me everything is tested and working fine automatically. What view of the web does the Chrome team have if they think the majority of site operators have this kind of testing setup running? It just feels so ridiculously out of touch with the reality of where web developers congregate and how news gets disseminated. I know it's a difficult problem, but the responsibility of Chrome is to work with the web community as it exists, not as the professional fully-automated community they'd like it to be. Chrome couldn't send out some Twitter DMs? Chrome team couldn't post a heads up to HN? Any effort to post anything to the popular web dev communities on Reddit? Did the team reach out to any any popular bloggers or members of the community to ask them to spread the word? When the Chrome team is making changes to the dominant browser for what is (imo) the most important platform on the planet, it just is not enough to say "we put it in the mailing list, and you weren't testing well enough." The expectations for the Chrome team here are higher, because Chrome matters more than other products. Yes, developers need to meet Chrome halfway on communication, but frankly, Chrome is not coming halfway right now to meet us. It should not have been a surprise to some of the biggest REPL sites on the web that their products were about to break. I don't know what to say about this, I'm not trying to be mean, but if the Chrome team's perspective is that developer communication isn't at least partially their problem to solve, and they think that if they roll out a feature and nobody knows about it that developers are the problem, then those people shouldn't be in charge of the dominant web browser.
  - Orphis 5y ago

svieira 5y ago

I read chromestatus every week or so and the intended landing date for this change caught me off guard.

Orphis 5y ago
But do you use Chrome Canary, Dev or Beta for development purposes? Would you have then noticed the change? Do you have continuous testing on the websites you maintain with those versions? I presume you're only human, so it's almost expected you will skip over a change as you don't think it's relevant or might even take days off work, in which case, it's easy to forget to read it. ChromeStatus is just one of the many signals you should use to filter issues before they reach customers.
- svieira 5y ago
  Yes, and I used to use it as my daily driver. But I switched to Firefox a year ago to make sure I wasn't ignoring the minority experience. Yes, we have CI (700+ machine hours for runs-on-master integration tests). No, we don't have a test that would have caught this (we are an app building platform and some of our features which our users use to deliver features to their users use iframes)

gpm 5y ago

> proper announcements in ALL the relevant mailing lists and release notes. I can't emphasize enough that mailing lists are worthless for this. Your average web developer reads no mailing lists, I'd be surprised if even 5% did, and those 5% are going to be clustered. Release notes (that anyone reads) come too late, since people only read them once users are already installing the broken browser. Announcements that you actually expect to be read need to be in a much higher profile place, it looks like this is an appropriate blog: https://www.blog.google/products/chrome/ https://www.blog.google/products/chrome/ - even then it should be written with the goal of it being picked up by places like reddit/HN/twitter/news. Even then you should expect that most web developers won't find out, and most who do find out won't change anything in regards to your announcement that your new version of the browser is going to break compatibility with their perfectly functional website, incentives just won't align for many of them to fix it pre-emptively. That last part is why I advocate for indicating the issue in a user visible but non-breaking way first, that's going to be the only way to get funding/time to update to your new version for many websites.

danShumway 5y ago
I'll add onto this, putting an announcement out needs to be followed up by checking to see if the announcement has gained any traction. Announcing breaking changes is not a checklist that can be filled out and then ignored. Put out an announcement, see if any other sites or prominent developers pick up on it, if they haven't picked up on it then the job isn't done and the change needs to be advertised more. Maybe for some things posting it to the blog might be enough, and the major stakeholders that need to know will pick it up. Then you can move on to the next steps. But if they don't see it, and you don't see any reaction to the blog posts, then don't just move onto stage 2 because "they had their chance, if it breaks now it's their fault." :) Have some standards of what level of conversation and preparation you expect to see from web developers before you consider the community adequately informed.
Orphis 5y ago
Here's an example of a major bank website with some user visible non-breaking issue I saw recently: https://youtu.be/GFvaRgFf4LU?t=482 https://youtu.be/GFvaRgFf4LU?t=482 . It seems that it's been visible for a while, yet is not fixed. It's actually quite common to see errors unfixed in software as you may know. No matter what you do, some websites will only be fixed a long time after very obvious warnings are displayed. Some will never be fixed until they actually stops working. Some will just never be updated at all. Maybe you're one of the people who take their car to the repair shop when the engine light is on, but there are also many others who will just ignore it as the car still drives "fine" (for now at least). Regarding your suggestion of using Reddit, HN, Twitter, news, it's not always possible or relevant. I've seen a lot of very interesting publications just not getting any traction on those are they're boring in nature or cater to experts mainly. They're also read only by a fraction of users. Unfortunately, if you're hit by any issue, you shouldn't be only asking what others should do for you, but also what you can do to catch those issues in the future yourself. All sides can maybe do better at broadcasting information about some changes, or voice their opinion better and sooner, but it's not an easy problem with an easy solution.