5 ms·

Windows still has well over 80% of the desktop market and IT WASN'T TESTED? How can that be? It's not like this is an obscure Windows issue. View any page on yo

Windows still has well over 80% of the desktop market and IT WASN'T TESTED? How can that be? It's not like this is an obscure Windows issue. View any page on your site and it would have been glaringly obvious... Thanks for the fun write-up though. Overboard rage aside, it's a good reminder that we all should double-check our QC processes.

JoeAltmaier 11y ago
Microsoft recently folded their test engineers into the general population. I.e. everybody is supposed to test their own code. Which means, testing dropped to negligible levels. That's why our IT dept has to vet every update now; some of them break your machine.
- zaphar 11y ago
  It's not on Microsoft to test Mediums web font rendering changes for them. Medium should have tested on a windows desktop with a multiple browsers. If they had they would have caught this before shipping to consumers.
  - JoeAltmaier 11y ago
    Ah I see.
- arcticbull 11y ago
  Not to mention, I have no idea why folding test engineers into the general population would have you give up writing tests? How can you know your code is ... right? How could you not try the thing you wrote yourself? Or did you mean QA?
- Chlorus 11y ago
  "That's why our IT dept has to vet every update now; some of them break your machine." Uh, what? That's what your IT dept should have been doing from day one. Are they also in the habit of just immediately applying kernel updates on production boxes?
deleted 11y ago
[deleted]
jwmerrill 11y ago
Yes, we should all test our software better. But let's look at the cost/benefit here for a second. Back in the days when "system" was a reasonable font choice, fixing a bug after shipping would take weeks, months, or years, and could cost $10^4 to $10^8. You might have had to pull actual boxes off actual shelves somewhere! In this story as it actually happened, the bug was fixed and the fix was deployed in a single day, at the cost of a few laughs. I'm sure they tested on Windows when they added Segoe to the list, so the problem is that they didn't test again after fiddling with other fonts that they (incorrectly) thought would be irrelevant to Windows. And remember, nothing was broken in a way that a machine would notice, so automated acceptance tests wouldn't have caught this regression. I think it's worth considering the possibility that this is a story of Medium acting more or less in their economic interest (maybe even in the economic interest of their users?) rather than a story of a bunch of hipsters with a hilariously broken QC policy.
- tvararu 11y ago
  > And remember, nothing was broken in a way that a machine would notice, so automated acceptance tests wouldn't have caught this regression. I'd like to nit here that visual regression test harnesses for web application development do exist. [1] [2] [3] Using something like Sauce Labs you could conceptually have these tests run on Windows machines in the cloud. However, having implemented visual regression testing in the past in various projects, I can personally vouch that it's a huge waste of time. Your tools will break all the time, your tests will break all the time, and you'll generally have a bad time. [1] https://github.com/chenglou/node-huxley https://github.com/chenglou/node-huxley [2] https://github.com/Huddle/PhantomCSS https://github.com/Huddle/PhantomCSS [3] https://github.com/bradgignac/janus https://github.com/bradgignac/janus
  - jwmerrill 11y ago
    Thanks for this perspective. I've never tried this kind of automated visual regression testing except in very specialized circumstances, but I've sometimes thought that I should. In analogy with scientific modeling, there's a risk of over-fitting your tests against the exact circumstances of your product as it exists today. You want tests to capture the important aspects of your product, while having a little bit of flexibility for the "noise" related to differences between platforms, and changes to mostly but not entirely unrelated parts of the product.
hbosch 11y ago
It's important to also consider that while Windows may represent ~80% of the desktop market, it may not represent that much of a majority for the Medium market. I'd be surprised, honestly, if 80% of Medium readers used Windows...
- Dylan16807 11y ago
  Hang on, "well over 80" and "about 80" are not the same thing. Why would you be surprised by 80% of Medium readers using Windows? Even 20% non-Windows would represent a massive anti-Windows bias.
  - twoodfin 11y ago
    Not if a large fraction of their traffic is mobile.
    - Dylan16807 11y ago
      I expect everyone on the site to own a real computer too. I interpret the question as "people that use X as their desktop OS" because of the context.
      - deleted 11y ago
        [deleted]
  - hbosch 11y ago
    To note: I wrote "~80%" because I was too lazy to look up current statistics and accepted the parent comment at face value, while also allowing a slushy figure. The general presupposition of my comment remains the same regardless: it's unwise to assume that the desktop computer market reflects a website's visitor segmentation.
merrywhether 11y ago
Testing windows? As basically the lone user of Safari on the various teams I've worked on, I've learned that you're lucky if people test on anything but Chrome for Mac.
- 100k 11y ago
  "My company's CTO just told me we need to support more browsers than just Chrome. I put in my two weeks notice." https://twitter.com/hipsterhacker/status/274693354804084736 https://twitter.com/hipsterhacker/status/274693354804084736
- sangnoir 11y ago
  Sadly Apple stopped updating Safari for windows a while back, and there never was a Safari for Linux. I am certain a sizable fraction of organizations/people are not prepared to buy a Mac for the sake of testing in Safari (depends on size of org & target demographics)
bitserf 11y ago
Overall web browsing is already 60/40 in favour of mobile though, and mobile is high 90% using WebKit or some flavour of it. Not that this justifies skipping testing on Windows, but it's not impossible to see how it's no longer a top priority, since all of the growth is sitting on the mobile side.
- fishanz 11y ago
  I'm sure a couple years ago almost all mobile ran WebKit, but aren't a huge chunk of new android phones running some flavor of chromium these days? I'd be surprised if WebKit was still in the high 90% range.

JoeAltmaier 11y ago

Microsoft recently folded their test engineers into the general population. I.e. everybody is supposed to test their own code. Which means, testing dropped to negligible levels. That's why our IT dept has to vet every update now; some of them break your machine.

zaphar 11y ago
It's not on Microsoft to test Mediums web font rendering changes for them. Medium should have tested on a windows desktop with a multiple browsers. If they had they would have caught this before shipping to consumers.
- JoeAltmaier 11y ago
  Ah I see.
arcticbull 11y ago
Not to mention, I have no idea why folding test engineers into the general population would have you give up writing tests? How can you know your code is ... right? How could you not try the thing you wrote yourself? Or did you mean QA?
Chlorus 11y ago
"That's why our IT dept has to vet every update now; some of them break your machine." Uh, what? That's what your IT dept should have been doing from day one. Are they also in the habit of just immediately applying kernel updates on production boxes?

deleted 11y ago

[deleted]

jwmerrill 11y ago

Yes, we should all test our software better. But let's look at the cost/benefit here for a second. Back in the days when "system" was a reasonable font choice, fixing a bug after shipping would take weeks, months, or years, and could cost $10^4 to $10^8. You might have had to pull actual boxes off actual shelves somewhere! In this story as it actually happened, the bug was fixed and the fix was deployed in a single day, at the cost of a few laughs. I'm sure they tested on Windows when they added Segoe to the list, so the problem is that they didn't test again after fiddling with other fonts that they (incorrectly) thought would be irrelevant to Windows. And remember, nothing was broken in a way that a machine would notice, so automated acceptance tests wouldn't have caught this regression. I think it's worth considering the possibility that this is a story of Medium acting more or less in their economic interest (maybe even in the economic interest of their users?) rather than a story of a bunch of hipsters with a hilariously broken QC policy.

tvararu 11y ago
> And remember, nothing was broken in a way that a machine would notice, so automated acceptance tests wouldn't have caught this regression. I'd like to nit here that visual regression test harnesses for web application development do exist. [1] [2] [3] Using something like Sauce Labs you could conceptually have these tests run on Windows machines in the cloud. However, having implemented visual regression testing in the past in various projects, I can personally vouch that it's a huge waste of time. Your tools will break all the time, your tests will break all the time, and you'll generally have a bad time. [1] https://github.com/chenglou/node-huxley https://github.com/chenglou/node-huxley [2] https://github.com/Huddle/PhantomCSS https://github.com/Huddle/PhantomCSS [3] https://github.com/bradgignac/janus https://github.com/bradgignac/janus
- jwmerrill 11y ago
  Thanks for this perspective. I've never tried this kind of automated visual regression testing except in very specialized circumstances, but I've sometimes thought that I should. In analogy with scientific modeling, there's a risk of over-fitting your tests against the exact circumstances of your product as it exists today. You want tests to capture the important aspects of your product, while having a little bit of flexibility for the "noise" related to differences between platforms, and changes to mostly but not entirely unrelated parts of the product.

hbosch 11y ago

It's important to also consider that while Windows may represent ~80% of the desktop market, it may not represent that much of a majority for the Medium market. I'd be surprised, honestly, if 80% of Medium readers used Windows...

Dylan16807 11y ago
Hang on, "well over 80" and "about 80" are not the same thing. Why would you be surprised by 80% of Medium readers using Windows? Even 20% non-Windows would represent a massive anti-Windows bias.
- twoodfin 11y ago
  Not if a large fraction of their traffic is mobile.
  - Dylan16807 11y ago
    I expect everyone on the site to own a real computer too. I interpret the question as "people that use X as their desktop OS" because of the context.
    - deleted 11y ago
      [deleted]
- hbosch 11y ago
  To note: I wrote "~80%" because I was too lazy to look up current statistics and accepted the parent comment at face value, while also allowing a slushy figure. The general presupposition of my comment remains the same regardless: it's unwise to assume that the desktop computer market reflects a website's visitor segmentation.

merrywhether 11y ago

Testing windows? As basically the lone user of Safari on the various teams I've worked on, I've learned that you're lucky if people test on anything but Chrome for Mac.

100k 11y ago
"My company's CTO just told me we need to support more browsers than just Chrome. I put in my two weeks notice." https://twitter.com/hipsterhacker/status/274693354804084736 https://twitter.com/hipsterhacker/status/274693354804084736
sangnoir 11y ago
Sadly Apple stopped updating Safari for windows a while back, and there never was a Safari for Linux. I am certain a sizable fraction of organizations/people are not prepared to buy a Mac for the sake of testing in Safari (depends on size of org & target demographics)

bitserf 11y ago

Overall web browsing is already 60/40 in favour of mobile though, and mobile is high 90% using WebKit or some flavour of it. Not that this justifies skipping testing on Windows, but it's not impossible to see how it's no longer a top priority, since all of the growth is sitting on the mobile side.

fishanz 11y ago
I'm sure a couple years ago almost all mobile ran WebKit, but aren't a huge chunk of new android phones running some flavor of chromium these days? I'd be surprised if WebKit was still in the high 90% range.