Web Accessibility Testing Explained for Product Owners

Web accessibility testing aims to ensure that each and any person can use a product without any obstacles. It expands the target audience of applications by making them accessible to people with disabilities — hearing loss, vision problems like a weakening of color perception, physical, cognitive, or other disabilities. However, product owners should discover and enhance web accessibility not only to be inclusive of people with disabilities but to keep the application in line with current legislation and to gain other competitive advantages. Typically, non-compliance results in incrementing web accessibility lawsuits, as in the UsableNet research indicates 319 lawsuits filed in September 2022. 

So, in this article we’re going to highlight legal provisions related to web accessibility, scrutinize real life examples, explain what tools are used to test a software product for web accessibility and what benefits are reaped with their help.

Web Accessibility Importance Explained with Examples

The digital journey starts when a user opens a web application. Further running into hurdles while making a purchase, watching a video, or navigating a website means that the user will close the tab and leave for a competitor. With web accessibility barriers, it’s a traditional scenario. According to WebAIM Million 2021, 97.4% of the world’s top one million websites didn’t offer full accessibility in 2021. It hasn’t changed much in 2022, though. And because of that inability to accommodate users, companies are losing out potential customers. Is it possible to make a difference?  

The best way to get the most out of web accessibility is to understand the type of difficulties that people are grappling with and the type of approaches we can use to mitigate these challenges. Hence, let’s sort out the types of disabilities and dive deeper into digital exclusion examples.

People with vision impairments

Blindness, color blindness, low or blurry vision are all disabilities of one group that may significantly differ in terms of visual perception of web content. For instance, blind people can’t get along without screen readers and might face difficulties if the navigation isn’t consistent or there is no description for visual elements and multimedia.

At the same time, people with low vision need screen magnification or text reading software to use a web application efficiently. They can’t read inscriptions or deal with UI elements if content isn’t resizable or the quality of the magnified text and images is poor. Low contrast text is recognized  as a widespread accessibility issue common to 86.4% of home pages. The light gray text and light desaturated background are anything but an ideal marriage: the content becomes unreadable for many users.         

As for common challenges affecting all people with vision impairments, these are inconsistency of navigation, lack of time for completing particular tasks, and unexpected action on the page including a redirect.

People with hearing disabilities

Deafness and impairments related to hearing cause complexities in perception of audio content and multimedia presented in web apps. By 2050, 2.5 billion people will have some degree of hearing loss, which means that web accessibility is becoming of paramount importance. Captions and subtitles, transcripts, and sign language translations seem to be useful for any web content. A lot of platforms, including Youtube and TED, have already introduced transcripts and captions for videos. However, automatically generated captions aren’t that accurate, and we can see a great room for improvement of web accessibility for people with hearing impairments not only through the use of  assistive technologies. 

There are also other less obvious steps to be taken for web accessibility enhancement:

• Provide different contact options
• Give preference to plain English 
• Implement intuitive navigation
• Review captions manually                                                            

People with physical, literacy, and cognitive disabilities

If a person has physical disabilities, using a keyboard or a mouse may be challenging. Literacy disabilities, in turn, imply challenges related to reading and finding appropriate words. Complex language and unstructured content on the website are common barriers for people with literacy disabilities. And as for cognitive disabilities, they cause memory impairment and difficulties in expressing thoughts. 

As we have already gotten acquainted with some types of disabilities, it’s time to grasp the significance of web accessibility testing and find out barriers that prevent people from using a web application. 

Here are widespread examples of such barriers:

1. Low-contrast colors (the text blends with a background, thus becoming unreadable for people with vision impairments )
2. Color as the only  feature to provide clues about the web app’s functionality (color-blindness hinders people from distinguishing colors, and screen readers don’t address this challenge, so there might be a problem if the UI text color is intended to convey specific information)
3. No tags for the images (any picture or chart should have so-called “alt text” description, otherwise people with vision impairments won’t understand their meaning)
4. No subtitles or captions for the video content (people with hearing impairments won’t catch the intent and the content of the video)
5. Sophisticated online forms (without labels for fields and indicators of errors, the form looks unclear to people with disabilities who can’t fill it out or submit) 
6. Navigation limitations or inconsistency (when your web application doesn’t support keyboard navigation, those who can use only a trackpad or mouse will face difficulties)

People with disabilities can partially overcome barriers by using specific assistive technology that’s gaining traction these days. Assistive technology includes speech recognition modules, screen readers, magnification systems, and specialized keyboards—all software that is beneficial for particular groups of users. However, without being tested for accessibility, web applications won’t be fully available to people with disabilities regardless of what software they use. So, let’s figure out how to ensure that the web application you are building is accessible to people with disabilities and elaborate on what benefits it may bring.

Read also:

Software Product Testing Strategy and Best Practices

Web Accessibility Guidelines, Standards, and Benefits                 

Digital inclusion and web accessibility are governed by WCAG (Web Content Accessibility Guidelines), Section 508 Standards of The Americans with Disabilities Act (ADA), and other documents.     

WCAG 2.2 consists of global regulations on web accessibility. Not only has WCAG set standards in the field of content accessibility, but it has proved to be a reliable source that doesn’t change after publishing. And Gartner reveals that by 2023, digital products in full WCAG compliance are expected to outperform their market competitors by 50%.

WCAG includes three assessment levels estimating success (compliance with WCAG): A, AA, and AAA. Taking into account WCAG 2 criteria, WebAIM has tested a top one million homepages and discovered that 96.8% of them have failures.

Section 508 Standards of The Americans with Disabilities Act (ADA) establishes requirements that aim at improving the accessibility of software apps, websites, mobile apps, and electronics for people with disabilities. These standards are relevant for federal agencies, companies that act as their contractors, and educational institutions.

MobiDev’s experience in custom software development proves that only those who want to go beyond expectations and take a step ahead of their competitors consider web accessibility testing in accordance with WCAG and ADA compliance. Nevertheless, the accessibility importance will grow and bring the following benefits:

• Alignment of the web product with legal requirements
• Potential growth of business through attracting a new audience
• High positions in search results
• Improved user experience

Since the significance of web accessibility testing is clear, it’s about time to deal with relevant tools and explain how to test a website for accessibility.

Web Accessibility Testing Approaches and Tools

To perform accessibility testing, we can use two approaches:

Manual Testing

QA engineers manually test the application with the help of  keyboard and screen readers, while applying different scenarios and finding potential barriers for people with disabilities. Manual testing implies review of content and its scaling, UI elements, styles disablement. It’s notable that to test a website or an application, QA engineers mostly do not use a mouse and may rely on additional tools that increase accuracy and reduce the time for testing. However, human WCAG testing is essential for ADA compliance since automated testing detects issues only partially. But what does automated testing actually stands for?            

Automated Testing

Automated testing tools allow engineers to catch errors across applications and web pages. So it’s an appropriate tool to analyze image contrasting, design styling, and an app’s functionality. Automated testing can foresee testing on physical devices to check how assistive technology works in particular cases and whether the web accessibility is sufficient enough. To get acquainted with testing tools, you don’t need to look further than the next section.         

Read also:

Crowdsourced Testing Vs Outsourced Testing: The Difference and What to Choose

Case Study: Testing Web Accessibility With 10 Tools

The MobiDev team was supposed to check web accessibility of the HR portal & automated employee management system. This solution represents a corporate social network with company structure. We decided to perform web accessibility testing using the most popular testing tools, and then evaluate the outcomes.             

Actually, accessibility checks are available even in Chrome DevTools: under the Lighthouse panel, aXe extension, or the Accessibility pane, you can view the accessibility tree, ARIA attributes, and computed accessibility properties of DOM nodes. However, we’ve investigated 10 tools and browser extensions to do away with some of them, choose the best ones, and apply them for real accessibility testing on the project.

Let’s take a closer look at the following web accessibility tools that were investigated:                      

1. WAVE
2. AXE
3. JAWS
4. Web Accessibility Toolbar (WAT) for IE
5. aViewer (Accessibility Viewer)
6. Functional Accessibility Evaluator (FAE)
7. Tenon
8. NVDA
9. Color Contrast Analyser (Chrome extension)
10. Color Contrast Analyser (tool)

WAVE Evaluation Tool

Web accessibility evaluation tool (WAVE) enables us to seek out and eliminate accessibility issues. With WAVE, you can easily get visual feedback on the content. So what it does is  facilitate the evaluation process rather than completely automate it. The process takes place in the Chrome browser (security measures related to passwords and sensitive information are taken into account). 

WAVE issues are aligned with WCAG 2.1 failures. Evaluation is simplified due to its user-friendly interface, which allows checking the compliance with Web Content Accessibility Guidelines and Section 508.

This is how it looks like on the real project.

As you can see, WAVE highlights different types of  accessibility issues, also contrast errors, some alerts, etc.

Axe DevTools 

Axe DevTools is an extension available for Chrome and Firefox. It’s free and lightweight and intended for testing whole pages or their parts. Accessibility issues here are graded according to their severity, while code snippets facilitate debugging. Compared to other tools, Axe DevTools helps to spot more issues because of the Intelligent Guided Tests feature which identifies areas to be tested. Axe-core is the crucial component under the hood of Axe DevTools. That’s why you are supposed to use Axe DevTools separately from tools with a similar tech stack, to avoid double-scanning. 

The tool clearly shows the results: 76 issues in total. All problems are categorized by severity: Critical, Serious, Moderate, and Minor.

JAWS

JAWS is known as one of the most sought-after screen readers for Microsoft Windows. The primary purpose of JAWS is to allow people with vision problems to understand the text displayed on the screen with the help of speech synthesizers or braille displays. JAWS provides speech outputs for websites, documents, applications, and emails. Also, it simplifies navigation and form filling for those who use Windows computers. As for the distribution of browsers among JAWS users, most of them prefer Chrome, FireFox, or Internet Explorer. In the U.S., a JAWS Annual License costs from $90 per year.

Web Accessibility Toolbar (WAT) for IE

Web Accessibility Toolbar (WAT) is designed for manual testing of web pages and can be added to Internet Explorer (IE) as a toolbar. It was developed by the Web Accessibility Tools Consortium, but currently isn’t considered as an undergoing development project. In general, we can consider it as an obsolete tool for the outdated browser.  

Accessibility Viewer (aViewer)

The Accessibility Viewer (aViewer) shows the accessibility API information which can be analyzed by assistive technology devices and operating systems. A web browser exposes the following API data:   MSAA, IAccessible2, UI Automation, ARIA, HTML DOM. 

It’s notable that aViewer UIA properties are displayed only if the browser supports them. For instance, IE doesn’t support IA2. The last version of aViewer available on GitHub is dated by May 2017. Therefore, this tool is also outdated and hasn’t been used in web accessibility testing on the project.

Functional Accessibility Evaluator (FAE)

The Functional Accessibility Evaluator (FAE) 2.0 checks the compliance of web pages with the W3C Web Content Accessibility Guidelines 2.0 Level A and AA Success Criteria.

FAE uses the OpenAjax Evaluation Library rules and rulesets. Its primary purpose is to be used in conjunction with AInspector WCAG for Firefox and Accessibility Bookmarklets to help people understand the accessibility features, issues, and potential issues of a web resource.

There is a direct interrelation between rules of FAE 2.0 and WCAG 2.0 Success Criterion requirements. Moreover, these rules imply the support of usable web design for people with disabilities. The latest version 2.3 is available on GitHub.

Tenon

Tenon is an automated accessibility testing tool that can be integrated with third-party apps to run unit testing, acceptance testing, system testing, and issue tracking. With Tenon, you can fix any accessibility issues within the website or application. It evaluates web accessibility for WCAG 2.0 and Section 508 compliance.

Tenon is a paid tool with a free trial period. It provides the possibility to test a URL without registration, yet the result seemed to be a bit alarming to us. So, we haven’t used this tool for real web accessibility testing in production.

NVDA

NVDA is a free screen reader for Windows. Nearly 40% of users prefer NVDA, treating it as the primary screen reader. Having NVDA at their disposal, people with visual impairments can navigate Windows or interact with third-party apps—from chat software to music player and office apps.         

NVDA includes a built-in speech synthesizer supporting over 50 languages. It also supports braille displays, can be run on Windows logon or other secure screens, and includes paid services like telephone support and training materials.  

Color Contrast Analyser (Chrome extension)

Crafted as a Chrome extension, Color Contrast Analyser verifies whether a webpage has issues related to the text color contrast and aligns with WCAG 2. The functionality allows users to assess entire web pages or any visible content of tabs. Also, it can check images and PDF documents that are easily opened in Chrome. Users decide on the type of conformance (AA or AAA) and specify the size of the text to be analyzed. In output, you can see a map consisting of edges identifying color contrasts which meet requirements. This output is available as a PNG file—you can download it on your computer. Before starting to use Color Contrast Analyser, you should be aware that the extension can access data related to your websites, tabs, and browsing activity.

Color Contrast Analyser (CCA) (tool)

CCA is a free color contrast checker for determining the contrast ratio of two colors by using an eyedrop tool. It enables content optimization, both images and text, in such a way that people with vision impairments easily explore it. CCA offers rich functionality, including indicators of compliance with WCAG 2.1, color blindness simulator, and different to select colors.

The tool allows you to see clean results of the color contrast for AA or AAA conformance. On the screenshot, the contrast validation for particular foreground and background colors has “Pass” status.

How We Run Web Accessibility Testing at MobiDev  

1. Ask about the particular scope of requirements-the product must match (WCAG 2.1, levels of testable success criteria, etc.)
2. Approve test environments and non-functional requirements for Accessibility.
3. Discuss and get approval for test tools, which will be applied for the testing (!). This could be very important because of different Accessibility policies in different organizations.
4. Create your estimates and approve the deadlines.
5. Keep in touch to discuss some edge cases and /or if you have different results from different test tools.
6. Get some cool headphones.
7. Kill the mouse! 

Which Testing Tools Really Work?

In most cases, 5-6 tools will be enough to provide complete and sophisticated testing and assure a high level of website accessibility. Here what we have used in our case:

1. WAVE — useful and convenient tool, which facilitates human evaluation of many aspects of accessibility and Web Content Accessibility Guidelines and Section 508 compliance.
   2. AXE — convenient tool for finding additional issues if WAVE doesn’t suit you as the only one source of web accessibility information. AXE and WAVE may work well together and be a great combo.
   3. JAWS — one of the best screen readers. It’s not free, though. We used 40-min JAWS sessions to double-check if there was anything that hadn’t been found with NVDA.
   4. NVDA — screen reader, which is almost as good as JAWS, but absolutely FREE.
   5. Color Contrast Analyser — Chrome extension for most general contrast testing, allowing us to see the big picture.
   6. Color Contrast Analyser (tool) — tool for precise Colors and Contrast analysis.

Avoiding pitfalls

1. Be aware that different test tools could produce different test results. So it’s better to double-check issues in some cases (e.g. NVDA can read elements in a different way than JAWS).
2. Always discuss all the edge cases with the client team. Sometimes it could be better to change some requirements than fix them.
3. Correct your primary estimates if needed,  because you never know how many issues you’ll find and how many times you’ll need to re-test them.
4. Document all the issues to provide a customer with a complete report of testing results.

Summing Up Results of Web Accessibility Testing

We leveraged different tools to perform accessibility testing for the product and find out trust-and-true solutions. With these tools, the Mobidev team can keep web applications in line with existing web accessibility requirements, ensuring zero critical or major issues. So this approach could be robust and effective for solving real tasks in accessibility testing. Should you need to develop a web product from scratch or just ensure its web accessibility before the release, feel free to drop us a line — the MobiDev web development team and QA engineers will be backing you up and directing your path.