12 ms·
Ask HN: Best way to test accessibility of a website?
Is there a bot/crawler akin to Google's Webmaster Tools that will test, score, and identify accessibility issues on a website?
If not, what are some best practices people use to maintain accessibility? Does anyone write tests for this functionality? Would love to learn more, top resources, checklists people use, etc.
- ioddly 8y agoLighthouse, available in Chrome devtools, includes an a11y audit although how thorough it is I can't say.
- mxadnam 8y agoHekar in i,
- winrid 8y agoI've used the Wave extension for Chrome. Also, being able to use the website with a screen reader (with your eyes closed) is a good metric.
- zczc 8y agoWave also has firefox extension http://wave.webaim.org/extension/ http://wave.webaim.org/extension/ and webapp http://wave.webaim.org/ http://wave.webaim.org/
- sfusato 8y agoI'm also looking for something similar: you give it a list of routes and it will score each of them. Doing this manually with Chrome's Audit tool for each route is time consuming.
- bazza451 8y agoWe had to do this a few years ago for a major bank. Automated tools like a11y will only get you a certain amount of the way. Your tests will say 100% but your app may still be completely useless to blind people especially if you’re using an SPA as all screen readers perform differently. Having a partially sighted consultant come in and point out the deficiencies and how they use devices was extremely helpful. A lot of sight impaired people use Apple devices as their voiceover support is excellent, Android is getting better but you will find a lot of differences between their implementations in versions which is extremely frustrating. For desktop on windows JAWS seems to be the most used which again has its own way of parsing. I don’t know if it’s changed but my advice would be plenty of manual testing, there’s really not much else than actually using the devices with voiceover enabled to test if things actually work properly. And it’s also a case of compromise and picking what devices to support as fixing something on one reader will inevitably break another. Hope that helps :)
- gtirloni 8y agoHere's one: https://achecker.ca https://achecker.ca
- xorand 8y agoI used your checker on https://news.ycombinator.com/newcomments https://news.ycombinator.com/newcomments and it gives 38 known problems and 384 potential problems.
- throwaway857384 8y agoPay me to do it... But in all seriousness, I use aXe-Coconut[1], headingsMaper[2]. Then tab through the page and make sure you can get from top left to bottom right. The cursor[focus state /ouline] should be visible the whole time and you should never lose your place. When you are done that, if you have a Mac turn on voiceover with Command+F5 and use Safari or Chrome (more aria-support). On Windows, download NVDA[3] and FireFox. Again use the keyboard to go through the page, but listen to the screen reader and make sure everything is spoken like it looks. So a button should say, "Submit Form Button" etc. Anything that doesn't say anything, ie, " button" or say something different from what you see needs work. The general consense of the community is that automated testing only finds 30% of issues, so you've got to test manually. [1] https://chrome.google.com/webstore/detail/axe-coconut/iobddmbdndbbbfjopjdgadphaoihpojp?hl=en https://chrome.google.com/webstore/detail/axe-coconut/iobddm... [2] https://chrome.google.com/webstore/detail/headingsmap/flbjommegcjonpdmenkdiocclhjacmbi?hl=en https://chrome.google.com/webstore/detail/headingsmap/flbjom... [3] https://www.nvaccess.org/ https://www.nvaccess.org/
- x0x0 8y agowhat are your rates and why are you qualified?
- throwaway857384 8y agoI don't want the extra work. But I do accessibility as my full-time job.
- x0x0 8y agoDo you have a recomended service? Part of our service has us hosting a form for our customers that is exposed to their users, so accessibility is a priority. Thanks for any pointers!
- throwaway857384 8y agoI've worked with people from Level Access[1] and Deque[2], there is also TPG[3]. I'm not sure how much money they cost, but they are the real experts in the industry. Some people on Fivver offer services, but I don't know how qualified they are. [1] https://www.levelaccess.com/ https://www.levelaccess.com/ [2] https://www.deque.com/ https://www.deque.com/ [3] https://www.paciellogroup.com/ https://www.paciellogroup.com/
- oh_boy 8y agoUse aXe for automated tests and test everything manually with JAWS + Firefox (most popular screen reader Desktop combination) and iOS VoiceOver (most troubling mobile issues in my experience). If you can afford in your project, use a external service that tests with people with different disabilities.
- throwaway857384 8y agoJAWS + IE is the most popular combination [1] [1] https://webaim.org/projects/screenreadersurvey7/#browsercombos https://webaim.org/projects/screenreadersurvey7/#browsercomb...
- jscholes 8y ago> JAWS + IE is the most popular combination It's worth keeping a few points in mind when reading these results: 1. The survey was carried out in October 2017. Things can and do move quite rapidly in this area, especially given the increased number of users who will be running Windows 10 now who may not have been back then. 2. IE won't be around forever, and once the accessibility of Microsoft Edge improves (which is likely to happen pretty soon when they adopt Chromium), IE will become less and less relevant. 3. NVDA with Firefox is a much more common combination than JAWS with Firefox, primarily because the NVDA developers recommend it. Freedom Scientific (makers of JAWS) have historically pushed IE perhaps more than other browsers, so it will be interesting to see how that changes as Edge becomes more usable.
- throwaway857384 8y agoMy only comments to that would be WebAim is the highest authority on accessibility. I'm well aware that these combinations will change over time. The biggest thing is that JAWS costs a lot of money, so people buy, get used to it, and stick with their setup. I tend to say, 'The only people who still use IE have no choice" or something like that.
- jscholes 8y ago
- dingus 8y ago- Use blind accessibility tools with your website. If you're on iOS/macOS enable VoiceOver, figure out how it works, close your eyes and use your website. - Disable JavaScript and use your website. This may not be relevant for web apps, but regular websites should work normally. - Try to use your website with only the keyboard. - Try to use your website with extreme zoom levels and font size overrides. - Monitor your resource sizes. Anyone with a limited connection (a LOT of people) will abandon your website because it loads slowly, or are paying actual money for the wasted bandwidth. I'm not aware of automated methods that give better results than using your website with the actual accessibility tools yourself.
- warent 8y ago> "Use blind accessibility tools with your website. If you're on iOS/macOS enable VoiceOver, figure out how it works, close your eyes and use your website." I doubt this is going to be very effective. Sure, it's better than nothing at all, but this is akin to sighted person with zero blindness experience testing the effectiveness of a newly designed white cane, only a website is much more complicated. You should always find someone with personal experience to test it themselves and provide feedback if you want a clear understanding of how it will actually work for that community.
- uberswe 8y agoEven if someone with personal experience is better I would still highly recommend testing these tools as it's not always clear how they work if you haven never used them. I was really surprised the first time I tested one of these tools and now I know a few dos and don'ts when developing web apps.
- jitl 8y ago>> Use blind accessibility tools with your website. If you're on iOS/macOS enable VoiceOver, figure out how it works, close your eyes and use your website. > I doubt this is going to be very effective. You are wrong. The only way for a primarily sighted team to build accessible software is to test and iterate on accessibility as part of day-in, day-out development practices. Think of it the same as localization for RTL languages or cross-browser testing: you should test in each environment you support before merging a change. Training and user-testing with blind community are important, but it doesn’t take a native Arabic speaker to notice broken layout in Arabic, nor does it take an Android device owner to notice usability problems in Android Chrome. The best front-end engineers I know all have a VoiceOver hotkey on their dev machines and phones, and I regularly see them testing sites they visit for keyboard navigation, focus management, and VoiceOver usability.
- ximm 8y ago- If you have little time and money, use any of the automated tools mentioned in the other comments - If you want the best results, do extensive user testing - If you are somewhere in the middle, do manual testing yourself. Simply turning on a screen reader and covering the screen with a sheet of paper does wonders. - Think about accessibility from the start, even before you write the first line of code (just like with security). It is very hard to fix it after the fact.
- dbjorge 8y agoMicrosoft recently open sourced[1] one of the main tools they use for web accessibility testing. Accessibility Insights for Web (https://accessibilityinsights.io https://accessibilityinsights.io) is a Chrome extension that guides you through doing an accessibility assessment of a website. It includes a combination of automated checks (using aXe[2]) and assisted/guided manual checks. Its rules are based on the W3C Web Content Accessibility Guidelines[3]; passing a full assessment in the tool amounts to meeting all the "A" and "AA" WCAG 2.0 requirements. Like other commenters have mentioned, automated checks (like aXe) are a great start, but aren't really sufficient on their own. Accessibility Insights will lead you through how to test different aspects of screen reader usage (and other types of accessibility issues, too!). It will also help you track issues as you go, and produce a pretty report at the end that you can attach to a bug. As with most types of UI testing, no matter what tools you're using, there's really no substitute for watching a real user try to use your product. Disclaimer: I am an engineer that works for Microsoft on the Accessibilty Insights team. [1] https://cloudblogs.microsoft.com/opensource/2019/03/12/microsoft-open-sources-accessibility-insights/ https://cloudblogs.microsoft.com/opensource/2019/03/12/micro... [2] https://www.deque.com/axe/ https://www.deque.com/axe/ [3] https://www.w3.org/WAI/WCAG21/quickref/ https://www.w3.org/WAI/WCAG21/quickref/
- scrollaway 8y agoHey I just tried this out, it's pretty awesome. Hadn't heard of it at all before. Love the tab-index mapping. Have you folks talked to the Lighthouse team to get your work integrated there; or at least possibly referenced there? I know lighthouse has a couple of accessibility recommendations, would be nice to have visibility on it.
- jscholes 8y agoThere is a great Chrome extension called aXe[1], as well as tools like Tenon[2] which also has a rest API so you can potentially set it up as part of a CI workflow. It's worth remembering that "accessibility" is a broad term. If you want a tool which can identify some common antipatterns and the WCAG guideline(s) they violate, the tools I've mentioned are a good place to start. But if you want to create a truly accessible website which is also usable by people with special needs, you'll need experts to come in to carry out audits and specialised user testing. Please feel free to drop me an email using the address in my HN profile. I'm a usability and accessibility professional, and disabled to boot. Happy to answer any questions you might have and connect you with other great resources, tools and people. [1] https://chrome.google.com/webstore/detail/axe/lhdoppojpmngadmnindnejefpokejbdd https://chrome.google.com/webstore/detail/axe/lhdoppojpmngad... [2] https://tenon.io/ https://tenon.io/
- raverbashing 8y agoDon't forget to check your website with a colorblindness simulator. This is a manual process unfortunately Also with things like high contrast, high zoom level or increased font size, etc
- jscholes 8y agoAs an additional comment for people coming across this thread in future and anyone else it may help. Please make sure that your accessibility tools are high quality, as you would when assessing any other dependency or service used in the development of your product. In particular, they should always assess the rendered DOM of any page, rather than a string of HTML. And if they make bold claims about the increased accessibility of your website once their recommendations are followed, or try to sell you some ridiculously cheap consultancy services, stay away. Whether you buy their consulting services or not, if they're too cheap, their tools probably are too. And if they tell you that automated testing is a route to complete usability for people with special needs, they're deluded.
- jedberg 8y agoHire an engineer who needs accommodation. I mean that in jest of course, but only sort of. For a while I had an engineer on my team who was dyslexic and while not color blind themselves, were very in tune with the color blind community. They made huge improvements on not only our front end UX, but also on our codebase. For example, apparently camelCase is really bad for dyslexics, and interstitial underscores are always_preferred.
- cwbrandsma 8y agoThere are multiple forms of dyslexia. For instance, I have one form and I have a daughter with a completely different variety. I have no problems with camelCase, but I am very particular about my fonts (Consolas for life). I don’t like _ because of the keyboard placement. I don’t trust my pinkies at those angles.
- jedberg 8y agoMy understanding from my coworker was that while some forms of dyslexics can process camelCase no problem, the underscore will never be worse than camelCase and will sometimes be better, so it's better to just use underscore. I'm certainly no expert though.
- sha666sum 8y agoDisable javascript and see if it still works.
- alangpierce 8y agoOne tool, not mentioned so far, is tota11y: http://khan.github.io/tota11y/ http://khan.github.io/tota11y/ It gives a bunch of inline annotations for accessibility problems and points of concern. Not sure how it compares with the others suggested, but might be useful to try.
- baroffoos 8y agoFirefox recently added an accessibility inspector in the dev tools which may be of use.
- austincheney 8y agoIn your test environment send HTML as type application/xhtml+xml, block all CSS and images, and add JavaScript that locks the mouse cursor to the first pixel on the screen. Then test the site for content and usability as you normally would. Vision impaired persons do not benefit from CSS or images. All images should have proper corresponding alternate text. Changing the media type of the HTML does require a bit more strict syntax, but it also is a cheap trick to ensure the HTML tags and structure are arranged in a valid way. I do not recommend disabling JavaScript. JavaScript is for interaction and blind persons need JavaScript just the same as the rest of us to navigate through content on complex sites. Inserting code that locks the mouse cursor to a single location forces QA to navigate the site/content using a keyboard. Doing these things is the cheapest way to test for very basic compliance. In order to achieve AA WCAG 2.0 conformance you actually have to understand accessibility and classes of disabilities. You might be able to get some quick wins with tool automation, but manual testing is much better. There is one exception, color contrast. You absolutely need to use a tool to perform color contrast validation, because people eyes aren't good at accounting for both luminescence and contrast at the same time. EDIT: In FireFox (Windows) you can turn off CSS by clicking the ALT keyboard key to bring up the main menu then View -> Page Style -> No Style
- qwerty456127 8y ago> send HTML as type application/xhtml+xml What does this change?
- skibz 8y ago> Changing the media type of the HTML does require a bit more strict syntax, but it also is a cheap trick to ensure the HTML tags and structure are arranged in a valid way.
- jitl 8y agoThis seems a bit bananas to me because there’s plenty of valid HTML5 is NOT valid XHTML or XML. For example, HTML5 allows common tags such as <p> and <li> to be unclosed, which is not valid in XHTML. HTML5 written in this style is not “wrong” - it’s valid, and correctly presented by tools like VoiceOver. Screen readers will be a-okay with HTML5; this advice is going to send people down wild goose chases with no need. XHTML is an alternative, dead-end branch of HTML evolution that time has forgotten. To validate HTML, use an HTML validator.
- longnow 8y agoYou can use chrome dev tools. Right click the window and click 'inspect'. In the 'Audits' tab, you can run an audit for accessibility. It runs your website and checks it against common accessibility criteria, giving you a score out of 100. On a side note, running this page through the accessibility audit, gives a score of 33. Perhaps HN has some work to do on accessibility.
- pbhjpbhj 8y agoHN it's very obviously bad for accessibility, small click/touch targets, grey-on-grey text with v. low contrast in some situations (by design), poor indent-level visibility. Using 'shade of grey' to convey information is never going to be great. Table-based layout can screw up screen-readers, I assume it's still 'tabulated but non-tabular'. Way back screen-readers used to assume the first tr contained th even if it wasn't marked up that way. It would be interesting to hear from people who use assistive tech for HN how much of a problem it causes them in practice.
- jscholes 8y ago> It would be interesting to hear from people who use assistive tech for HN how much of a problem it causes them in practice. I use a screen reader and am a frequent reader of HN. Obviously, much of the site's content is textual - you can't include images in comments AFAIK and it's extremely rare for people to link to graphical content as people often do on Reddit. So from that point of view, the site is an oasis away from the often visual nature of the modern web. On the other hand, there is absolutely no way for my screen reader to gauge whether a comment is a child of another, quickly jump past a child thread that I'm not interested in, quickly move up to the parent or grandparent of the current comment, etc. TL;DR: the content is accessible because it's text, but accessible navigation is non-existent.
- JimDabell 8y agoYou've received a lot of recommendations for software, but one of the most important things to bear in mind is that software cannot ensure accessibility. It can discover some problems with accessibility, but it has a lot of blind spots. If you care about accessibility, you need to incorporate manual human testing. This is a highly specialised field. The best thing for you to do is to fix the low-hanging fruit by plugging automated tools into your CI system, then get an expert to do a manual test so that you're aware of what other issues there are.
- hikergirl1234 8y agoSwitch the language option if one exists and try using the website in a language you don't understand. Has helped immensely for my side projects in improving UI clarity and ease of navigation.
- darekkay 8y agoFor automated testing, try pa11y[0] or Lighthouse (intergrated in Chrome, using web.dev or as a standalone NPM package - (it uses axe-core under the hood)). Tools will only get you so far, though. To make your site fully accessible, you will need to actually test it yourself. Luckily, most issues can be solved/prevented by following some simple guidelines [1]. Recently, I've reviewed the accessibility of 21 popular CSS UI frameworks [2], so this might be also helpful to you. [0] - http://pa11y.org/ http://pa11y.org/ [1] - https://moritzgiessmann.de/accessibility-cheatsheet/ https://moritzgiessmann.de/accessibility-cheatsheet/ [2] - https://darekkay.com/blog/accessible-ui-frameworks/ https://darekkay.com/blog/accessible-ui-frameworks/
- deleted 8y ago[deleted]
- bookofjoe 8y agoAsk me to look at it: 39 days ago, on this very site, I learned that I am what is called by cognoscenti a "breaker": https://news.ycombinator.com/item?id=19153810 https://news.ycombinator.com/item?id=19153810
- majewsky 8y agoI have this, but with typos. Give me a page full of text with a single typo, and somehow, against all odds, I'll immediately spot it.
- bookofjoe 8y agoToo bad copyreaders are a dying profession 'cause you'd be a natural.
- majewsky 8y agoEh, it's not like I crave copyreading. So I don't mind being automated away by spell checkers in that department.
- bookofjoe 8y agoFWIW,since the Washington Post effectively fired all its copyreaders, there have been significantly more errors, as noticed by this close reader/subscriber.
- ficklepickle 8y agoMe too! I just got an email from Cory Doctorow re: a typo I found in his article. It"s my only super power
- philo23 8y agoNot seen anyone mention it yet, but the Accessibility feature in Firefox (might only be available in Nightly/developer edition, not sure) built into the developer tools are pretty good. You need to go into the dev tools settings to enable it from memory. It gives you access to what is essentially an "accessibility DOM" that shows you how the page's structure will look to a screen reader, and you can also check the contrast of an elements text against its background. It might not help you score any better on things like the webmaster tools tests directly, but it might help you identify things to change/fix.
- lioeters 8y agoJust checked in Firefox dev tools: there's a top-level menu for Accessibility, but the panel displays a grayed-out message, "Turn On Accessibility Features" which can't be clicked. Thanks for pointing out this (future) feature, it sounds very useful. I'll keep an eye out for when it's available.
- philo23 8y agoJust had a quick read, looks like it's available since Firefox v63, https://developer.mozilla.org/en-US/docs/Tools/Accessibility_inspector#Accessing_the_Accessibility_Inspector https://developer.mozilla.org/en-US/docs/Tools/Accessibility... It should work though once you click on that button, weird that it doesn't seem to be working for you. Edit: Just read through more of that link, it's got quite a good breakdown of the features on it.
- lioeters 8y agoThank you for the link, I bookmarked it to read more in detail. You're right, that's a nice overview of the features. I've always felt the need to learn more about accessibility (and regularly apply it to what I build). This looks great for the purpose. I found in the browser preferences that I'd checked "Prevent accessibility services from accessing your browser". I had to uncheck it to be able to enable the Accessibility Inspector. Will be exploring it more.
- 8y ago
- sureaboutthis 8y agoThere are maybe 10 well known tools for doing such things, easily Google-able, including Google's tools, that web developers everywhere have been using for ages but I'm on my phone at an airport so I'll try and post them tonight.
- ykevinator 8y agoPa11y
- SpaceL10n 8y agoA lot of good suggestions here already, but in terms of screenreaders, JAWS has a majority stake in the market. They make a tool call JAWS Inspect which gives you reports (and by extension, version control) over your screenreader presentation. It's not free though.
- zxxc 8y ago1. Install NVDA 2. Read part 4 from user guide https://www.nvaccess.org/files/nvda/documentation/userGuide.html?#GettingStartedWithNVDA https://www.nvaccess.org/files/nvda/documentation/userGuide.... 3. turn off your monitors 4. try to do simple smock test
- T3RMINATED 8y agoTesting what? use a framework that is known to be Accessibility Friendly, and you are done!
- Favic0n 8y agoCheck out http://www.sitespeed.io http://www.sitespeed.io
- netrap 8y agoTry to browse it with NetSurf!
- walrus01 8y agoTry viewing it in lynx and navigating with keyboard commands only in lynx, to see how it will work for a screen reader tool.
- tmaxxcar 8y agoI can think of only one tool that will test, score and identify accessibility issues, as well as track progression/regression over time. A testing tool provided by User1st. uTester That said, automated testing can test anywhere from 30-40% of WCAG, leaving the rest to be found by a manual review. If you are not familiar with JAWS, NVDA, VoiceOver or TalkBack then a large portion of WCAG on your site(s) will go un-tested. http://www.user1st.com/services/ http://www.user1st.com/services/
- KineticLensman 8y agoAlso, please don't forget people with hearing impairment. You don't need a special tool, just remember to include text captions or similar if your page includes spoken content. I'm partially deaf so sensitive to this issue. With hearing aids my hearing is 'fine'. But I recently completed some online training (that had spoken information) with the sound turned completely down to see how bad it would be for a profoundly deaf person. There were two questions where a correct answer required having heard a spoken (but not captioned!) statement.
- throwaway5752 8y agoPay Knowbility or another specialist (https://news.ycombinator.com/item?id=19475842 https://news.ycombinator.com/item?id=19475842) - no affiliation with either - to do it for you. It's a much deeper domain than you think it is and people have real expertise worth paying for, particularly if you need section 508 compliance for a contract and money is on the line.
- inevitable2 8y agohttps://www.appbeat.io/ https://www.appbeat.io/
- gnicholas 8y agoWCAG has historically focused mostly on accessibility for people with physically-manifesting needs (blindness, deafness), but has recently begun working on addressing usability for neurodiverse populations (dyslexia, ADHD, autism). IMO, this is long overdue, especially considering how large these populations are. Things to consider here relate to text presentation and overall site layout. Think about text sizing/contrast, line spacing, column width, and paragraph height, for example. And consider how each of these things changes when the user is on mobile (paragraphs are now much taller, creating the "wall of text" that many people dread).
- Sumner19 8y agoThere is a new tool from Microsoft/Axe partnership. Microsoft is one of the leaders in Accessibility. Partnering with Axe (Deque), the tool not only has a fast checker, it shows both on the site and in the report where the problems are, links to exact WCAG rules and has guidelines how to fix. All Open Source. Chrome extension: https://chrome.google.com/webstore/detail/accessibility-insights-fo/pbjjkligggfmakdaogkfomddhfmpjeni?hl=en https://chrome.google.com/webstore/detail/accessibility-insi...
- dta5003 8y agohttps://www.compliancesheriff.com/ https://www.compliancesheriff.com/ Compliance Sheriff helps identify compliance issues with WCAG and Section 508 standards, and their scan will give output with visual indicators of the non-compliant areas. Haven't seen that in any other tool. They have a free test for up to 10 publicly facing URLs too.
- pcmaffey 8y agoJust wanted to say thanks to everyone for all the super helpful responses in here!