Automated Accessibility Testing with A11ym

The A11y Machine or a11ym (pronounced as ‘alym’) is an open-source automated accessibility tool for web applications which crawls and tests web pages against accessibility conformance standards to provide detailed reports.
 
A11ym validates the web pages against following accessibility standards:

• W3C Web Content Accessibility Guidelines (WCAG) 2.0, including A, AA and AAA levels (understanding levels of conformance),
• U.S. Section 508 legislation
• W3C HTML5 Recommendation

Why Accessibility?

Accessibility is not just about helping disabled people. It’s about making the web more inclusive and enabling people around the globe with different abilities to access web content with greater ease. Accessible websites not only yield good SEO reports but also avoid potential lawsuits. Hence, making websites accessible is beneficial in every way.

Why A11ym?

One of our clients, a renowned insurance company in Australia, wanted to perform accessibility checks on their current site. Imagine the amount of manual effort in terms of time, resources, and money required to perform the accessibility run for a complete website! To overcome this challenge we decided to look for an appropriate automated tool, which brought us to a11ym. 
 
It had a couple of compelling features which sealed the deal for us -

• Can run locally 
• Can test private code 
• Can test auth-required parts
• Can crawl all web pages
• Can test single URL as well as multiple URLs
• Team can focus more on fixing the code rather than finding errors

A11ym Installation 

Pre-requisite:

• Java ( to allow a11ym to validate web pages against the HTML5 recommendation)
• Node Package Manager (NPM): Open terminal and run following command: $ npm install -g the-a11y-machine. A11ym will be installed in your machine in no time.

To ensure successful installation of a11ym, open terminal and run command: $a11ym

 
You will see options list for a11ym

 
A11ym Usage: 

Let’s start with typing the base URLs of website

Hit command : $ ./a11ym http:/test.org/

All URLs accessible from http://test.org/ will be tested against the WCAG2AA (default accessibility standard defined in a11ym) standard. In order to reduce the number of URLs to test, --maximum-URLs option will be used.

For example: if the only URL needs to be tested use --maximum-URLs 1 or -m 1
 

Testing list of URLs

What if the client wants to perform accessibility check on business critical pages instead of crawling complete website? In order to achieve the same, multiple URLs can be provided in two ways -
 

1. Compute several URLs by adding them to the command-line, like this: $ ./a11ym http://test.org/Ahttp://test.org/B http://test.org/C
2. Read URLs from STDIN, as follows: $ cat URLs.lists | ./a11ym -

URLs.lists is the file containing a list of web pages. It can be of other extensions e.g. URLs.txt. When reading several URLs, the --maximum-depth option will be forced to 1 be the default.
 
Note the  -: It means “Read URLs from STDIN please”.
 
Standards (WCAG2A, WCAG2AA, WCAG2AAA) cannot be combined with each other, except HTML5 that can be combined with any other. 
 
So for instance, to run WCAG2AAA:
$ ./a11ym --standards WCAG2AAA http://test.org/
 
To run WCAG2AA along with HTML:
$ ./a11ym --standards WCAG2AA,HTML http://test.org/

How a11ym works?

Following are the intricacies of a11ym

• The node-simple crawler tool is used to crawl a web application based on the given URLs
• Two type of tests are performed on each URL.

1. Accessibility: PhantomJS runs and HTML_CodeSniffer is injected in order to check the page conformance. This step is semi-automated by the help of pa11y, which is a very thin layer of code wrapping PhantomJS and HTML_CodeSniffer,
2. HTML: The Nu Html Checker (v.Nu) is run on the URL to validate web page against HTML conformance.

• Results from different tools are collected and produced

A11ym Reporting

“a11y_output” folder will be formed in the same directory from where the run was executed, once the test run is complete. 
To view complete report open a11ym_output/index.html residing in same folder.
 
Report of all URLs will look like this:
User can view detailed report of each URL separately by clicking on specific URL link.

 
 A11ym report bifurcates all the issues in three categories i.e. error, warning, and notice. 
A CSS selector is provided with each reported issue to select and analyse the buggy element. In addition to it, description and a link to the W3C recommendation is also provided.
 For example:

• Notice: This button element does not have a name available to an accessibility API. Valid names are: title attribute, element content
• Code: H25,
• Rule name: #WCAG2AA.Principle2.Guideline2_4.2_4_2.H25.2.
• Selector: html>head>title
• Code extract: <title>Career With Test - Scrum Mast...</title>

 
You can follow the GitHub link to download the a11ym project and explore it further.
 
If you work with QA and test automation, you can also explore our posts on Behat installation and mobile testing automation tools.