OpenAccess

Automating automated accessibility audits

Callum McMenamin — 15 May 2025

Many of us run browser extensions for automated accessibility testing. But how do we make this process easier for developers?

Layered defence against accessibility failure

Delivering accessible web content reliably is all about using multiple layers of defence. If one layer fails to catch a problem, the next layer hopefully will. No single tool or process will catch every issue, or be appropriate in every situation.

For example, it’s not practical for many organisations to pay for a manual accessibility audit every day, however, manual audits are absolutely essential for catching issues that automated tools simply cannot detect.

Relying only on automated tools also isn’t practical, as you’d be lucky to catch even a quarter of the issues that a manual audit would catch.

Sometimes even automation is too manual!

If you’re relying on automated accessibility testing tools that run via a browser extension, even that can become a burden. Browser extensions typically test one page at a time, which severely limits their usefulness in development. Sometimes you might forget to run the extension on a new page, or after you make a change. Maybe your organisation’s development team encounters some turnover, and the informal practice of using accessibility browser extensions vanishes from the organisation.

Luckily, there are ways to further automate this process. Using browser extensions is perhaps the slowest way to perform automated testing. Embedding automated testing into developer workflows vastly improves developer experience, therefore resulting in higher compliance with testing strategies.

How OpenAccess does it

The OpenAccess website is a static site built with Eleventy.

I have created a large number of automated tests to help me maintain high quality standards throughout the site, such as:

• automated spell checking (via cspell)
• automated accessibility testing (via axe-core and IBM equal-access)
• automated link health checking (via a custom node script)
• automated visual regression testing (via playwright and pixelmatch)

All of the above testing methods are run on every single page of the site, using a web crawler implemented in a node script, which uses playwright to render each page in a headless instance of Chromium.

These tests are run automatically, either on every commit, or when I push changes to the remote git repository. I often forget to run tests, so having them automatically run when committing or pushing changes is a great way to ensure I don’t forget.

I use husky to create git hooks — these ‘hooks’ basically enable me to run scripts at certain points in the git workflow.

There are two main husky files I use:

• .husky/pre-commit — this runs before I commit changes to my local git
• .husky/pre-push — this runs before I push changes to the remote git repo

pre-commit is really useful for running tests that don’t take much time, because commits occur frequently, I don’t want compute-intensive tests to slow down my workflow. Spell checking, for instance, is very quick, so I run that.

pre-push is more useful for long-running tests, such as visual regression testing (as it’s computationally expensive). As people generally push less frequently than they commit, longer running tests are less of an annoyance.

So, any time I make changes to the site, I can be confident that every single page has undergone multiple layers of automated quality testing.

Continuous integration and continuous deployment (CI/CD)

While git hooks might not be CI/CD in a traditional sense, the approach of using git hooks to run automated tests achieves similar outcomes.

Using git hooks to run automated accessibility tests is fairly efficient as it can run locally, and take full advantage of the local machine’s compute. I’ve found that running tests locally, rather than through GitHub Actions, generally results in less debugging, waiting time, and frustration.

One potential downside of using git hooks, is some developers in a team might not install the hooks correctly, meaning when they push changes, the tests won’t run. This can be solved by using a CI/CD system. However, you can also ensure git hooks are installed via your package.json file, for instance, so when a developer runs npm install to get set up on a project, the hooks automatically get installed.

Code to auto-install git hooks

To ensure git hooks are installed automatically when someone runs npm install, add this to your package.json file:

"scripts": {
    "prepare": "husky"
}

Conclusion

Automated accessibility testing needs to be integrated further into the development workflow for it to be maximally effective.

The core advantage of automated testing is that it can be run frequently, inexpensively, and across thousands of pages within seconds. If organisations are only using browser extensions, they are missing out on the true power of automated testing.

And finally — while I am a huge advocate for automated testing (could you tell?), I have to reiterate that it is not a replacement for manual testing. Automated testing only catches a small fraction of all accessibility issues, and should be used in conjunction with manual testing. But, that small fraction of issues is detected for free, so why not use it? The combination of both layers of testing — automated and manual — is the best way to reduce the risk of publishing accessibility failures to the web.

— Callum