Accessibility is Usability. If your app or site isn’t accessible, it isn’t
usable – at least by some segment of the population. But also, if your app
or site fails to meet accessibility criteria, it’s probably unusable by a
larger segment of the population than you think. Just anecdotally I think
that if an audit of your website shows a lot of accessibility-related
defects, it’s rather likely that it’s crappy to use for fully-abled people
Let’s look at how we can build accessibility into our development process.
Like security, quality, and testability, accessibility (a/k/a “a11y”) isn’t
something you can “bolt on” at the end of a development cycle. It needs at
least a little bit of up-front planning so that we ensure we’re sticking to
guidelines and don’t have to do a bunch of rework. And like those other
“-ities”, if we pay attention to the guidelines they will actually help us
avoid having an app that sucks – in more ways than just being unusable by
people with disabilities.
If you’re clueless about accessibility, the W3C has a good
I’m not going to talk about what it means or the technical requirements.
I’m also not going to try to convince you it’s necessary any more than the
sales pitch above. Click the link above the for the why and what. This
article is about how to make it part of your day-to-day routine so that you
can simply produce great software.
Where Does Accessibility Testing Belong?
As noted above, we should be thinking about accessibility from the
beginning of the project. Part of this is just practical: if we wait until
the end and discover that we’ve built an app with 50 pages and we use a
scanner that comes up with 20 usability errors or warnings on every page,
then we’ve got 1000 fixes to make! It’s like waiting to turn up the
compiler warning level until the end of the project.
This means we should integrate some sort of accessibility testing into our
early in the life of the project. In the version of the pyramid I show in
that article, a11y scan is in the middle of the pyramid as something that
runs in CI – individual developers don’t have to run it on every compile.
CI-only is a reasonable choice. This ensures that developers can’t merge
accessibility-related regressions in their MRs. But it doesn’t slow down
their level 1/2 fast-feedback test reports.
What might be ideal is if there were a Go library that could take a web
page (and linked CSS and whatever other resources are needed). Then we
could write “unit tests” that fetched pages similar to some of our other
tests and passed the page
bodies into this validation library to check for violations. But I haven’t
been able to find anything like this. (Please let me know if this exists!)
An interesting possibility is to write semgrep rules to warn on accessibility
defects. I think it would be impossible to catch all categories, but these
could at least catch some of the simpler structural warnings.
A hybrid approach is probably a good choice: add semgrep rules to catch the
things that are easy to catch with that tool, especially if the
accessibility scanner tends to catch certain violations a lot. Add unit
tests for some things if there are patterns that are easy to catch that
way, and you want to catch those things earlier rather than in an MR. And
that have one or more accessibility scanners running in CI (and/or higher
in the pyramid).
There are tons of testing tools
available. Unfortunately many of these
are online services that want to do scans of a live website. This doesn’t
really help us to remove errors during the development process, before a
site is live.
There are also a number of audit services available, and while I think it’s
very reasonable for a business to pay for an occasional outside
accessibility audit, I’m mostly interested in making accessibility testing
available to the masses so that (for example) open source projects are more
usable by more people. Thus “Call for Pricing” automatically rejects a
set of tools and services.
Another category of tools are intended to be run manually in the browser.
This is better – at least we can run these on our local dev or staging
apps and have a chance to fix problems before deployment. An example of
this type of tool, and one that is especially low-impact to integrate is
Khan Academy’s tota11y.
This is great because it’s just a bookmarklet, and a couple of clicks gave
me the information I need to fix the specific problems that were found. But
I still find this to be a rather slow feedback loop, and it doesn’t fit
well into a CI pipeline.
Two tools that I found that are specifically built for CI are
and pa11y-ci. The BBC tool specifically
has a docker image that I’ve had good luck with so far. (I haven’t tried
pa11y yet.) The examples below use bbc-a11y but should be adaptable to
It’s worth noting that I used bbc-a11y and tota11y on Aklatan, and they found
a small number of errors that were not completely overlapping – so there
is definitely value in using multiple tools for this testing.
Supporting Individual Developer Workflows
The encapsulation of bbc-a11y in a docker image makes it trivial to run the
tool against a single url. Assuming you’ve got the app running in a
separate terminal session, and it’s listening on port 3000 of your laptop,
which has IP address 10.10.1.250:
docker run --rm --tty bbca11y/bbc-a11y-docker http://10.10.1.250:3000
This will generate a list of issues found in the main page of the app. Note
that we use the external IP of the dev machine to keep things simple and
avoid having to jump through any hoops with docker.
We can also pass a series of URLs on the command line and each will be
checked in turn. Individual developers can check whatever url(s) they are
actively working on to get fast feedback.
We can also maintain a config file with all of the urls in the app in
bbc-a11y-config.js. One complication is that we need to do this in a way
that allows each developer to customize the ip address where their app is
running. We’ll do that by embedding an environment variable in the config
file and then running a trivial bit of preprocessing. Here’s an example
And we can process that with envsubst, which substitutes the values of
environment variables in its input. And then we can mount the current
directory as a volume in the docker container and tell bbc-a11y to use that
And then, because that’s just a little bit complicated to type all the
time, we can add a Makefile rule that performs the substitution and runs
the tool. Developers still have to make sure their app is already running,
but mostly they can just run make a11y and the tests will run. And manual
runs of single urls aren’t very complicated either.
@if [ -z "$$APP_PUBLIC_IP"]; then echo "Must set APP_PUBLIC_IP"; exit 1; fi envsubst < bbc-a11y-config.js > a11y.js
docker run -v $$(pwd):/mnt/ --rm --tty bbca11y/bbc-a11y-docker --config=/mnt/a11y.js
Integration with CI
Once you’ve got a Makefile rule that works locally, integrating with a CI
pipeline is only a little bit of extra work. Different CI services will
require slightly different approaches, but here’s a sketch of how it would
work on GitLab’s CI:
add a build step that creates a docker image with the app
add an a11y job that:
includes services to run the app
sets the APP_PUBLIC_IP to the “hostname” of the service
runs make a11y
Later this week we’ll add a backend command to import data from CSV into
A fully-worked example of the pipeline described above will be in a
subscribers-only Special Bonus edition next week.
Next Friday we will dive into pagination (this will be really nice to have
after importing a whole bunch of books with our handy importing tool).