19 ms·

Ask HN: How relevant is cross-browser testing anymore?

• 2rsf 4y ago
  Yes, to some degree. We use usage data to decide on a strategy, it usually ends up in testing on the big names on big platforms the latest and one or two versions back. Of course we are risking a self fulfilling prophecy since we rarely test Firefox for example.
• CM30 4y ago
  Well you don't need to test on lots of versions, since updates are usually automatic now (or at least highly encouraged on startup). But you definitely need to test on Chrome and Firefox in the browser (maybe also Safari?), and both Chrome and Safari on mobile, since there are JavaScript APIs and CSS features not supported by all of them. The latter is especially true when you're trying to make a PWA or 'web app', since Apple does not like letting websites have similar abilities to native apps on their devices...
• Venezio 4y ago
  True
  • sachinjain 4y ago
    We might be wrong here and missing some valid point. I see companies doing automation testing beyond a point but do they really test beyond 2 browsers? One line of thought could be if we have already invested in automation testing then why not run in 80 browsers too when we can. Its just additional hardware cost or cost of platforms like BrowserStack/Lambdatest.
• raxxorraxor 4y ago
  Not a web developer, but I usually just test with the latest or almost latest version of FF and perhaps Chrome. I personally don't like or use any of the frameworks you presented. If I had to pick one I would choose Vue. jQuery did a lot for inter-browser compatibility, so I don't understand the argument. On the contrary, it was much more difficult to manage diverging code without it. 99% of my web programming is displaying some data, a website to interface XY. I don't like modern websites at all. Probably more a design descision than a framework issue, but I try to minimize any browser scripts in the first place and just haven't felt the need for somthing like React or Angular.
  • sachinjain 4y ago
    Hey @raxxorraxor, I am not against using jQuery. What I meant was initially back in 2012-2013 days when jQuery used to be a default DOM query/manipulation library people used, the same code used to break across browsers and we had to use some polyfills. Like you said you usually test with the latest version of Chrome/Firefox you are partly agreeing to the statement I made. I am trying to find someone who's strongly in favour of testing their apps across browsers and multiple versions.
• sagarsoni 4y ago
  Most of the cross browser issues arise either from CSS or JS, and both of these occur because of older browsers. Since we don’t have control over people as to which browser they use, devs generally tackle this issue by using different stylesheets for different browsers conditionally. Modern tools like lambdatest, browserstack generate multiple previews easily.
• PaulHoule 4y ago
  I develop w/ Firefox and our tester tests with Chrome. I use our product quite a bit with an iPad. If problems are reported with Safari we fix them. I've also worked at places where our product was specified to work with Chrome, I would try to run it with Firefox and no dice and they would not accept any ticket to fix bugs in Firefox. There's an assumption today that browsers get updated frequently. My work computer is a managed Windows machine that I had administrator access to, the one complaint I have is that the Firefox is a long-term support version which is often long as in "in the tooth". Bloomberg would complain about it incessantly until I got the new LTS version. Other than that I think browsers are usually up to date.
  • sachinjain 4y ago
    Fixing any issue specific on Safari is a whole challenge on another level. Hope you had an easy time debugging issues on Safari.
    • PaulHoule 4y ago
      At work we've never had one that was that bad. In one cases it was third-party cookie issues that affected Safari before everyone else. Personally Safari is the reason why some of my image-based sites still use JPG in parallel with WEBP. I think they fixed it just recently, but if you have an older mac (i do) Apple couldn't be bothered to include a WEBP codec with the OS. Firefox, Chrome and every other browser for mac supports WEBP but not Safari. I think they resolved this in the last month, but I think for better or first Apple wants to roll out new SIMD instructions in their libraries much faster than Microsoft does. So at some point your CPU is too old. Of course the flip side to that in the Windows World is that people are stuck using a 10 year old instruction set. Intel doesn't seem to get it that by not supporting the latest AVX in their latest chips that people just "don't use AVX". If the latest SIMD instructions were widely supported you'd have a reason to buy a new computer, but when they aren't Intel has lackluster sales and wonders why.
      • warrenm 4y ago
        Honestly, I thought pretty much everyone had moved to PNG from JPG by now
• tgv 4y ago
  Depends on how critical it is for you/your company. I work on products that have internal tooling, and if they run on the latest version of Chrome, it's good enough (although I personally test them on Firefox, too, and someone used an older version of Safari the other day, and found a weird bug). However, some of our tools are used by the general public (I've found the user agent for a coupon app as well as some PlayStation browser in the logs; might have been spoofed, of course). Those are written to the lowest common HTML/CSS/JS standards, so we don't try fancy features, and Babel or similar can be helpful. IE11 is out of grace, but we still support years old versions of most browsers, because it costs money and goodwill when too many fail.
• Izkata 4y ago
  Nowadays the biggest issues are with CSS, and those frameworks won't help with that. However, there are other CSS frameworks (Bootstrap/Tailwind/etc) that do. Even using those, I'd still give other browsers a look over to see if anything looks odd. One weird one we had was, on Chrome/Edge a React dropdown looked fine, but on Firefox it was somehow positioned like 80px too high. I don't remember exactly what the fix was, just that it was a CSS problem and the original developer was relying on a Chrome default they didn't realize was there and not in other browsers. Something akin to "position: relative; top: 20px;" just happened to look vertically centered, but only in Chrome-based browsers.
• jfk13 4y ago
  > Do we really need to test our code on 50ish Chrome versions, 25is Firefox versions and so on. Probably not, unless you're a really big outfit with a vast user base. But you should test on at least a recent Chrome version and a recent Firefox version, and probably Safari. And unless your product is irrelevant to mobile users, on both iOS (Safari) and Android (Chrome, Firefox), with both phone and tablet form factors.
  • TAForObvReasons 4y ago
    Painful as it is to admit, even Firefox is not worth testing. In our B2B site, more people are using IE11 than FF. https://gs.statcounter.com/browser-market-share https://gs.statcounter.com/browser-market-share at ~3% of overall share / 7% desktop-only share it's hard to justify putting extra effort.
    • patrakov 4y ago
      Maybe they are not using Firefox, because your site is broken in it, and they have no other way to tell you this?
      • helij 4y ago
        This. There are some websites that I have to login that are completely broken when using Firefox. I login with Brave on those and am on a constant lookout for competitors that work on Firefox. As soon as I find them I jump ship and take my money elsewhere.
      • kfrzcode 4y ago
        Survivorship bias! Yes, this is something I've experienced directly from end-user perspective as well as debugging/user-analytics perspective.
      • l0b0 4y ago
        Yep, it's the classic self-fulfilling prophecy. Even a lot of non-techies are aware that there are multiple browsers, and will just switch to whichever gets the job done. So if you're seeing less % of any specific browser than the overall web your site is probably broken on that browser. If it's not obvious by now, that means you need to do more testing in that browser, not less.
• yeputons 4y ago
  I have no idea about how relevant it is, but every few months I see a website which seem to be tested in Chrome only, because it blatantly don't load in Firefox or Safari. Like, at all, blank page. Or have some buttons like "Purchase" not working.
  • chunk_waffle 4y ago
    My retirement account and healthcare provider both have this problem. In one case I had to manually delete a disabled attribute on a button to get signed in because something broke halfway through a multi-step workflow.
  • WoodenChair 4y ago
    When you see this, complain and let them know. I've had some success complaining about this to medium sized companies and getting updates.
  • thepra 4y ago
    Can't tell you my distress when government websites use Google captchas and on Firefox you're stuck and can't go through your business...
    • BenjiWiebe 4y ago
      Google CAPTCHAs work in Firefox for me, on multiple devices and OS's.
      • withinboredom 4y ago
        Do you work at Google maybe? They don’t work for me either.
  • NoboruWataya 4y ago
    The Irish tax authority's login portal is completely broken in Firefox. It's the only website I need to have a separate Chromium install for.
• Arubis 4y ago
  I consult, frequently jumping between web application codebases. Very often, I bring cross-browser testing with me by sheer nature of using Firefox when _everyone_ has been using Chrome exclusively for years. I end up finding issues; about half the time, clients don't care, because all their customers use Chrome. This direction worries me (we did this with MSIE and it took a decade+ to heal those wounds).
  • thepra 4y ago
    Just tell them Chrome is going to become the next IE if everything only works on Chrome
    • warrenm 4y ago
      id est? That's the response you'll get from most people nowadays ... especially if they're under 40
    • Macha 4y ago
      More and more of them have never had to deal with web development in the era of IE
• squeegee_scream 4y ago
  PeacockTV has problems on Safari but just works on Firefox. This is just the most recent example I have of large companies whose site doesn’t work well on all browsers. So yes I think we should test on multiple versions of multiple browsers on multiple OSes
• andrewmcwatters 4y ago
  Regardless if you're doing small personal projects or doing commercial work, personally I don't find it too time consuming to just breakout the latest versions of Safari, Chrome, and Firefox and do some local testing. Any experienced software developer will have some experience testing against those. I don't think you need to go out of your way to test against specific builds of those browsers, with maybe the exception of Safari and Mobile Safari since I have noticed standards-compliant differences over time with older iOS devices. That is to say, there isn't anything wrong with what Safari and Mobile Safari do, but the WHATWG living standards have changed over time just enough that older Apple devices may render web pages differently. Try not to just limit yourself to Chrome though. And if you're using vendor extensions, don't rely on them, just use them to pretty up from your baseline designs.
  • Macha 4y ago
    Safari does require a Mac, so a little more time consuming unless you're already a Mac shop
• politelemon 4y ago
  It's still relevant - there are enough differences between the browsers to render incompatibilities and bugs. It does also depend on how serious you are about your product, as some kinds of products not working in a browser can leave a very bad or frustrating experience. It is of course painful to test in so many variants, so a decent compromise is often to just test in the latest version of all browsers. There's enough coverage there to help you find any potential problems.
  • thepra 4y ago
    That's why I use only features that are cross compatible between browsers, even though I feel left behind on some new features only available on Chrome -_-
• golemiprague 4y ago
  I have noticed that there are still vast differences in the printing functionality, both in terms of how things render and performance. So if you need to do any kind of printing in the app and you know your clients will use an array of different browsers it is worth checking it. Even within the same browser things tend to go back and forth when it comes to printing, Chrome for example tends to break things in one version and then they come back in the next one. It is a bit of a mess.
  • croisillon 4y ago
    all your comments are flagged, please write an email to the admins to sort this out
• aww_dang 4y ago
  Feels like everything just works on Chrome and Firefox, but Safari is the new IE.
  • timack 4y ago
    Absolutely, feels like almost 100% of browser specific issues we deal with are related to Safari.
  • yakubin 4y ago
    Here is a bug I've recently discovered, which is present only in Chromium-based browsers: <https://bugs.chromium.org/p/chromium/issues/detail?id=1382987 https://bugs.chromium.org/p/chromium/issues/detail?id=138298...>. Doesn't happen on Safari, nor Firefox.
• n8cpdx 4y ago
  Please test with Safari on desktop at least. I routinely find things that are just completely broken. And not small names, big names like Hulu, Clover, green chef, and others have just completely broken on Safari. Integration with Google’s captcha seems to be a common failure point. Also please consider writing your apps in a way such that they don’t completely shit the bed if your 600th ad tracker script fails to load. I hate having to disable pihole so that Google can track me while I pay my utility bill.
  • chadlavi 4y ago
    Occasionally I run into innocuous enough websites that just don't work at all unless I let them track me. Note to all fellow fe devs: don't fucking make your call to analytics a blocking process!
  • drummer 4y ago
  • Raed667 4y ago
    Safari is a pain with bugs, unimplemented standards and just overall unpleasantness. I can't see why anyone would still use it today when FireFox for Mac has become so good.
    • codetrotter 4y ago
      Safari is probably better optimised for power consumption on macOS than Firefox is. Meaning you’ll probably get more hours of use with Safari on macOS than with Firefox on macOS, when you are running on battery.
      • lotsofpulp 4y ago
        Battery life, speed, and integration with Keychain is why I use Safari.
    • lelandfe 4y ago
      The simple answer is that Safari ships with the platform and few people choose browsers based on standards adherence. For daily use, it’s a pretty normal experience.
• Bilal_io 4y ago
  Not much anymore, at least from my experience. When writing Typescript, I know what I am targeting and it'll transpire to that. When writing CSS, I rely on caniuse.com when not certain.
• pacifika 4y ago
  I encountered a cross browser mixed implementation bug only today, affecting react components. https://jsfiddle.net/svandragt/236Lhkub/3/ https://jsfiddle.net/svandragt/236Lhkub/3/ So I’d say yes
• dutchCourage 4y ago
  I don't think it's worth testing on old versions of Chrome and Firefox since they're updated automatically. iOS Safari on the other hand, can't be updated on older phones since it's tied to iOS updates. This makes any Safari bug linger for a few years. It's also the only rendering engine available on iphone so some people are truly stuck using an old version. So I'd say, test on Chrome because it's ubiquitous and test on iOS Safari like it's the new Internet Explorer. I'm not sure about Firefox. I personally use it and test on it, I think it's the right thing to do for the web. And at least with that you've covered most rendering engines.
  • justsomehnguy 4y ago
    > I don't think it's worth testing on old versions of Chrome and Firefox since they're updated automatically https://support.mozilla.org/en-US/kb/firefox-esr-release-cycle https://support.mozilla.org/en-US/kb/firefox-esr-release-cyc...
  • s1mon 4y ago
    I don't disagree that most versions/instances of Chrome are updated regularly, but implying that this is universal is questionable. Chrome is only updated if the user lets it happen. I can't count the number of times I've watched some co-worker share their screen on a conference call and the Chrome update button is lit up, but apparently that's not a trigger to these people (I update pretty much as soon as I notice the indication since Google has done a remarkably good join of updating in place in a way that most SaaS things and web sites just reopen without trouble). I'm sure the next time they restart/reboot it will get updated, but if there was some hideous Chrome-related bug which is from a release a month ago, you might want to think about that. It's also entirely possible to turn off updates if you or some misguided IT person is really so inclined.
    • dutchCourage 4y ago
      Good point, it's definitely not universal. Maybe it's more fair to think of it in terms of priority. I don't expect a smaller team to test on many versions, but the bigger the team and the more users they have, the more diligent they should be with their testing. So in my opinion, in order, I'd test on: - current Chrome - current Safari - older Safari - older Chrome & current Firefox Not sure what comes next, IE 11 maybe? I guess at some point it's good to look into automating some testing. And it's worth noting, as with most things, that "it depends", I don't expect the same browser support from something like Figma and from a government website for example.
• Mikeb85 4y ago
  Chrome and Firefox auto-update so no.
• hebrox 4y ago
  We're using Playwright [0] to do e2e testing. Locally I just run Chrome, but on CI FF and Safari are used as well. Using the official Docker image, this was really easy to setup. Because all browsers are automatically updated nowadays, we only test the most recent versions. A recent issue we found, was that Safari was responding different to some HTTP headers. [0]: https://playwright.dev/ https://playwright.dev/
  • sachinjain 4y ago
    Do you use any tool like Browserstack or Lambdatest or Saucelabs for running your tests across browsers? My question is if you have already invested in end to end testing automation then why not run in 10 browsers to be on safe side than running in only 3. Price?
    • hebrox 4y ago
      If it adds value, I think it's a good idea. But less is more, so right now I'm not inclined to add more to our CI/CD pipeline. In the future our application will need to be able to run on mobile devices. Then I think I want to test it on a recent iPad, iPhone and Android device. We'll probably be using one of the tools you mentioned.
• bryanrasmussen 4y ago
  > Do we really need to test our code on 50ish Chrome versions, 25is Firefox versions and so on. The only time I've ever seen this intense level of testing was on a government project that was producing someone that everyone in the country would be required to use. So if that is what your situation is yes, you need to do that much testing because if it is mandated by law that citizens use it, it better be able to handle whatever combination of citizen and tech it encounters. Otherwise I think you are probably being somewhat hyperbolic here. Aside from that where accessibility is concerned there can be significant differences between various combinations of browsers and screen readers.
• polygot 4y ago
  It depends on your risk tolerance and how many DAU/WAU/MAU you have (and whether you're willing to have an untested experience for X users.) For example, I would test really old browser versions, and if there is a bad experience, you could let the user know to use a different browser or to upgrade. My intuition says you'd get better value out of testing different browsers shallowly rather than 50 old versions of Chrome, though.
• helsontaveras18 4y ago
  I experienced a DOM issue where a horizontal scrollbar was being rendered on some versions of Chrome running on Windows, some versions of Firefox on Mac, but I couldn't reproduce it myself (Chrome / Safari on Mac). It happens. I don't QA for these things beforehand, but when they do occur, it helps to have a way to perform cross-browser testing to fix the bug.
• deleted 4y ago
  [deleted]
• mindcrime 4y ago
  Not just cross-brower, but its still important to test unique client operating systems as well. Else you get weird crap like this: https://www.reddit.com/r/linuxquestions/comments/yfjaa1/disney_stopped_working_on_each_of_my_linux/ https://www.reddit.com/r/linuxquestions/comments/yfjaa1/disn... I just ran into this bug myself like two days ago, when Disney+ just up and quit working. And even better, if you apply the UA string hack to work around that, it breaks Netflix! I had to, for the moment, install a browser extension that allows persistent site-specific UA string overriding. And sure, that works as such, but c'mon... this is bullshit. People, test your dang websites on more than Chrome on Windows for FSM's sake.
• jinushaun 4y ago
  Most cross browser bugs are CSS and yes, you still have to test that. The problem is not JS
• spinlock 4y ago
  Look at your analytics and see which browsers your users are using. Sometimes, I’ve worked on apps without a big userbase but we had to support IE because a big percentage used it. You’ll probably find out that you don’t have too many browsers that actually matter.
• comprev 4y ago
  I worked with a client last year who had a firm requirement of IE6 being 100% compatible. No fancy CSS just plain old hand coded HTML. Lightning fast performance mind! It was an e-commerce site which sold _very_ grey area products (most were banned from Amazon/eBay/FB Marketplace/etc.) and the ancient IE spec gives you an idea of the demographic they ~exploited~ sold products to ... The market for selling questionable medicinal pills is much much larger than I could have ever imagined. Absolutely mind blowing!
• radiojasper 4y ago
  The other day I came across a website where the signup button didn't work on Firefox alone. So yeah, that's reason enough to keep on testing on all major browsers.
• leephillips 4y ago
  I made a website that did some slightly unusual things with text inside SVGs. It rendered correctly on Gecko but not on Blink. I submitted a bug report to the Blink project, and they fixed it pretty quickly. Who knows what other rendering bugs lurk in the various engines. So at least test on recent versions of the three major engines.
• brundolf 4y ago
  It's still important to test a bit on Firefox and Safari, or you'll have the occasional UI bug. I found one earlier today where Firefox behaved differently than the other two and needed a quick fix. I'd say this happens once every few months Times have changed in that you won't have layers and layers of branching logic to work around browsers that aggressively disagree with each other. But Chrome tends to fudge the web standards in some of their dark corners, Firefox tends to stick to the letter of the web standards rather than the spirit, and Safari tends to have random little gaps and quirks Your UI will probably be usable without cross-browser testing (assuming you're not doing anything exotic), but it will probably have little things here and there that aren't quite right
• pipeline_peak 4y ago
  You have to test on Safari because Apple was smart enough to box out Chrome from iOS. We’ll probably see Gecko used in more embedded applications but Firefox is a joke at this point.
• r_singh 4y ago
  Never had to do it since I use autoprefixer and safari is my main browser
• jpgvm 4y ago
  Depends if you care about Safari. Real browsers like Chrome, Firefox and Edge generally "Just Work", Safari on the other end seems to be an endless tire fire that either implements things poorly or not at all. Doubly true for the iOS versions which generally tend to be older and worse than desktop Safari.
• cheri9 4y ago
  [dead]