If something can be automated, developers will try to build it to save them time. In the field of accessibility, some automated tools out there can catch up to 50% of accessibility defects (like axe-core). On top of that, there are a lot of free accessibility rules libraries out there already for developers to choose from. This perfect storm of conditions can make it extremely appealing to take on the task of building your own accessibility testing solution to integrate into their existing test environments.
While this is a great solution for some teams, there are many challenges to building your own automated accessibility solution. In this post, I’ll discuss my experience of trying to build an automated accessibility solution at a Fortune 50 company. I’ll also cover when it’s better to buy an enterprise solution.
What It Takes To Build Your Own Automated Accessibility Solution
The Groundwork Before Building Your Own Tool
Before I worked at Deque, I was at a large Fortune 50 insurance company. My company was actively trying to address accessibility, but in the past had mainly done audits manually in test environment links and used a monitoring tool for issues that were pushed into production. My accessibility team was trying to help development teams test for accessibility earlier in the development life cycle, so we tried to build an automated accessibility testing solution.
I knew the solution needed to test the content at build time, give the accessibility violations back in a way that all users could understand the results, and possibly have a graphical user interface (GUI) to scan content automatically. When I first started researching the requirements for this tool, I assumed that other developers in my company were using similar tech stacks and standards that I had used on my previous team within the company.
After doing a lot of research, I realized each development team was working in a multitude of different tech stacks and testing frameworks. Identifying all these types of frameworks was time-consuming and I knew that these frameworks were subject to change with new trends. To add to that, the requirements for the tool grew as I needed to build a tool that worked in multiple types of build processes and tech stacks.
I also knew that the tool I was going to build needed CI/CD pipeline support. At most large companies, a DevOps team controls your CI/CD pipeline support such as Jenkins, Bamboo, or Azure. This team is usually very busy and difficult to meet with in order to get the proper approvals.
Before you build a tool, it’s also very important to identify teams that will use the tool. At this point, you’ve outlined the technologies and testing frameworks that teams are using. The next step is about managing change and getting development teams to use the tool and implement a culture of accessibility that creates developers that are accessibility aware as they create new content.
At my organization, I was lucky that we had an existing central accessibility team that could help spearhead this effort, while also giving me access to accessibility experts that helped identify features for the tool. However, if your company doesn’t have this team to help drive buy-in, it may be difficult to handle this change management process on your own. This may add more time and research to your project timeline, but my recommendation is to find advocates wherever you can to act as evangelists of accessibility.
Resources
After laying the groundwork and performing research before building the tool, we started to look into resourcing for the project. As I previously mentioned, we were developers that were a part of an accessibility team, but we needed outside resources to help as well.
When finding resources for building this tool you’ll need developers who are familiar with Java, JavaScript (NodeJS), C#, and Ruby. Advocating for resources to cover all these tech stacks was difficult to achieve. On top of this, we had to advocate for doing this project on top of our regular work and a pilot team’s development work. In an agile development world, this can be hard to do as products and priorities are rapidly changing.
Support
Ultimately, we were able to get the resources we needed to drive the project. We had some buy-in from teams and the necessary resources to build the tool. It’s important to mention that progress was slow because the requirements for the tool were constantly changing and developing throughout the process. Eventually, our team did get to a point where we had the tool built and running and being used by our pilot team, but it needed support.
This is the point in the project where things started to fall flat. Once the tool was in use by development teams they started to find bugs and ask for specific feature enhancements. If you use an open-source library like axe-core, there are updates that are regularly pushed. An update may cause something to break in your built solution.
It was difficult in our situation to determine if the development teams themselves should fix the bug, or an open-source team that sponsors it, or if our accessibility team should fix it. In my example, the dev teams wanted my accessibility team to fix it, which quickly became a full-time job that we couldn’t support properly.
In the end, my management didn’t want to support our internally-built accessibility solution. After this point, you’re back to square one. Looking back, I can say it was a difficult process to build this accessibility solution. These struggles are unique to large organizations, so if you work at a small organization, it may be much easier to build an internal solution.
One last thing to keep in mind is that turnover and trade out of people who built or are supporting this built solution may cause future problems. Knowledge transfer can theoretically prevent this from being an issue, but this approach is rarely successful in practice.
The Pros of Buying An Automated Accessibility Solution
Getting Your Organization At a Better Starting Point
Deque’s tools are often packaged with professional services that will help you cut down on the time it takes to implement a new tool. This professional support provides access to a team that has a wealth of experience integrating our automated accessibility solution, axe DevTools, into any type of build system or tech stack.
Our developer services team has dealt with a lot of different companies across the board, and they understand the unique challenges that development teams are facing when implementing new tools. If your organization’s build process and tech stacks are a special case, you’ll need this extra support to implement a new tool.
Another important aspect that our team helps with is a strategy for how to properly use this tool so it works in practice. As I previously mentioned, accessibility is a change management problem. Buying an automated accessibility tool saves time and energy and comes with helpful documentation, but it still requires some strategy and process to make sure the tool is adopted, successful, and withstands turnover. Our developer services team can help instill those processes and strategies with your team.
Out-Of-The-Box Integration
Axe DevTools comes with out of the box framework integrations that you don’t have to build yourself. Below are the types of integrations that exist so your team can hit the ground running:
Continuous integration environments include Maven, Jenkins, Saucelabs, CircleCI.
Native Mobile integrations are appium, xcode, and Android. Jest, React, Circleci
Immediately Start Working Towards Accessibility Compliance and Remediation
Some organizations need to buy an automated tool because they have a compliance need for accessibility and need to stop the bleeding ASAP. Buying an automated solution instead of building one saves time and can get you fully integrated in your testing environment faster than starting from scratch. Your team can quickly catch 50% of accessibility bugs with automation alone.
What is axe DevTools?
In Summary
There are pros and cons to buying or building an automated testing solution for your team. This post isn’t meant to discourage developers who are trying to do the right things and be proactive about accessibility– it’s very likely that your situation could be different than the one I described.
Rather, the purpose of this post is to shed light on the effort, time, and overall magnitude of what it takes to build an automated testing solution and what challenges may occur, especially if you’re at a large organization. This story is something I wish I would’ve heard when I first started my accessibility journey in my previous role.