Juggling competing priorities, tight deadlines, and endless iterations can make it easy to deprioritize accessibility considerations. In our work, we've seen how neglecting accessibility can lead to frustrating user experiences and other consequences. That's why we decided to treat accessibility issues with the same urgency as any other bug.

Enter the Accessibility Bug Classification Matrix, which we created to help us quickly prioritize issues based on their impact. It provides a clear framework for promptly identifying and addressing the most critical accessibility barriers.

Features of our bug classification matrix

• Adapted from industry best practices
• Covers a wide range of Web Content Accessibility Guidelines (WCAG) criteria
• Prioritizes issues based on user impact severity
• Provides clear action steps for each priority level

Overview

As our team grows and projects become more complex, we've realized the value of having a centralized process and systemic approach to prioritizing accessibility bugs as they are discovered.

Inspired by the impact classifications developed by accessibility experts at Deque, we adapted their framework to fit our workflows and priorities. The result is a helpful tool that helps our teams quickly assess and prioritize accessibility issues based on their true severity and impact on users.

This matrix isn't just for accessibility experts. We designed it to be a resource for all team members, regardless of the accessibility knowledge. By providing clear definitions and examples for each priority level, we're empowering everyone on our team to make informed decisions about what needs to happen when a bug is discovered.

It's important to note that this isn't meant to be a rigid, exhaustive list of every possible accessibility issue. We recognize that the field of accessibility is constantly changing, and new challenges emerge.

Accessibility as a team sport

We're fostering a culture of shared responsibility by making this resource available to the whole team. Accessibility shouldn't be limited to a few experts. Whether you're a project manager triaging a backlog of issues, a QA team member doing release testing, or a designer who has noticed something odd when reviewing how their work was implemented, our Accessibility Bug Classification Matrix provides a common language and clear guidance.

Assessing failure levels and issue priorities

We've divided the matrix into five levels, each with its own set of criteria and corresponding remediation priority.

Level 1: Page-level interference (Single A WCAG criteria)

Think of this as the "drop everything and fix it now" category. Issues at this level are the most severe, as they fundamentally block users with disabilities from accessing essential content or functionality.

See the full table of criteria in the Accessibility Bug Classification Matrix.

When we encounter issues like these, our priority is to address them immediately, even if it means bypassing a regular release cycle. The user impact is simply too high to defer resolving the issue.

Level 2: Primary component failure

Next up are issues that significantly degrade the user experience for key components or content. While not as severe as Level 1 issues, these still present substantial barriers for users with disabilities.

See the full table of criteria in the Accessibility Bug Classification Matrix.

The goal is to address these level 2 issues in the next regular release cycle. They're high priority but with a bit more flexibility (compared to Level 1).

Level 3: Component interference

At this level, we're dealing with issues that, while not completely blocking access, still present significant usability challenges for assistive technology users.

See the full table of criteria in the Accessibility Bug Classification Matrix.

Our target for Level 3 issues is to resolve them within the current quarter as part of regular backlog refinement.

Level 4: Page or component degradation

These issues, while still important to address, have a lower impact on overall usability. They may hinder compliance or present minor annoyances but typically have workarounds.

See the full table of criteria in the Accessibility Bug Classification Matrix.

We prioritize these for remediation as we're doing related development work or as time allows.

Level 5: Missing best practices

Finally, we have issues that might not be strictly related to compliance but represent opportunities to enhance accessibility and user experience. For these, we rely on ongoing accessibility education and awareness among our teams to consistently apply best practices and bake these in from the start.

See the full table of criteria in the Accessibility Bug Classification Matrix.

By breaking issues down into clear, actionable levels, we can ensure that the most impactful issues are addressed promptly while still being pragmatic about making improvements.

Advancing accessibility, one bug at a time

Project realities like tight deadlines and limited resources often force teams to make tough choices about where to focus their efforts. By using tools like the Accessibility Bug Classification Matrix and developing a culture of responsibility, teams can make meaningful progress without compromising other project goals. A clear, actionable framework lets us be strategic about where to spend our time to provide the biggest impact. The key is to approach accessibility pragmatically and as an ongoing process.

Prioritizing accessibility bugs for maximum impact was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

What 21st Century IDEA Means to CivicActions

Our team has embraced many of the key elements of The 21st Century Integrated Digital Experience Act (IDEA), which became a US law 5 years ago (December 2018). This was a bipartisan act to encourage government agencies to build a framework and requirements for a digital-first public experience, with an emphasis on accessibility, building mobile-friendly user experiences, digitizing services and forms, and improving the overall customer experience.

The legislation was designed with specific recommendations for a variety of senior executive agency roles including Chief Information Officers. At a high level, the guidance was designed to push agencies and the overall administration to modernize the federal government. Included in the IDEA act is a handful of strategic goals calling on agencies to:

• Modernize their websites (which includes being mobile-friendly)
• Digitize services and forms
• Transition to standardized and centralized shared services
• Improve overall customer experience (CX)

Additionally, the Office of Management and Budget (OMB) Memo M-23–22, Delivering a Digital-First Public Experience, came out this past fall (September 2023), which expanded on the 21st Century IDEA act. The OMB memo clarified what the 21st Century IDEA act meant by the concept of modernization. The OMB memo affords greater insight into what OMB and the federal government expects agencies to deliver to the public.

This OMB Memo gives specific mechanisms by which agencies should be improving their CX. Specifically, agency sites should:

• Provide a feedback mechanism for users to report satisfaction or dissatisfaction with each web page or piece of web content
• Test online content with the intended target audience before and after publishing
• Examine websites and digital services to ensure content is written and implemented so that Limited English Proficiency (LEP) users can meaningfully access those services

CivicActions has extensive experience working within these strategic goals found in the IDEA act and the OMB memo, and we have brought this expertise to agencies such as the US Department of Veterans Affairs (VA), Centers for Medicare and Medicaid Services (CMS), the National Science Foundation (NSF), and the Department of Education (ED). The work that we do is accessible, responsive, mobile performant, and frequently exceeds the requirements of Section 508, thereby ensuring the solutions create the best customer experience while adhering to the needs of the public.

Modernizing websites

We understand that a modern digital experience fully incorporates smartphones. Our team works with our clients to build a good mobile first experience. We know from the US Census data that millions of Americans do not have a desktop computer. Data from Analytics.USA.gov shows that more than half of the traffic to the sites monitored is coming from either iOS or Android users.

To ensure we're building the best mobile-first experience, CivicActions works extensively with Drupal and the US Web Design System (USWDS). We maintain the Drupal USWDS Theme, and also implement it for our clients. This allows us to confidently build modern digital experiences for multiple devices, screen sizes and assistive technology, through a proven mobile-first strategy. By leveraging the USWDS federal design system and open source platforms such as Drupal, we improve the customer experience (CX) of our clients' sites.

In addition, we bring content strategy, user research, and visual design into our processes to ensure they meet the needs of the public and the agencies we serve. Our practitioners are well-versed in using human-centered design (HCD) approaches and we balance this with addressing the needs of each agency and their goals. Finally, our focus on accessibility allows us to build sites that are future compatible, providing an experience that works across all of the public.

Digitizing services and forms

There is an ongoing challenge to digitize government services. Agencies are struggling to move beyond legacy PDF forms. At CMS and VA, we have done extensive work on building customized web forms. These forms follow best practices, and provide a user-friendly interface for citizens to provide information to the government. Such approaches also help to ensure information gathering is done in an accessible and usable manner, creating time-saving data collection methods for agencies and reducing mistakes and processing time.

Transitioning to standardized and centralized shared services (including open source)

CivicActions has extensive experience in open source communities, like the USWDS. We are connected into the USWDS community, and our teams contribute to improving the USWDS framework by actively engaging with the community through collaboration in GitHub and Slack as well as through annual conferences, trainings, newsletters, podcasts, webinars and other opportunities. Where we can, we bring our deep Drupal experience and solutions to our client's challenges back into the USWDS. This is a critical part of seeing that we continue to provide robust digital services in the future.

This work with Drupal supports the goal of future transitions to centralized shared services. Building open source solutions with Drupal create structured content which lends itself to standalone and/or federated systems. Content can be published once, and blended into the pages of multiple sites, thus reducing the cost of maintaining repetitive content. Governments around the world are seeing the benefits of centralizing on mature open source technology like Drupal.

Working with Drupal also provides us an advantage both in accessibility and mobile support. Drupal has been built to exceed the needs of Section 508 for both the public facing and author facing interfaces. With 25% of the population having a disability, we know that authors also need accessible interfaces. Because of our work in the Drupal community, we are able to engage with global experts on a range of topics.

Improve overall customer experience (CX)

As mentioned above, we have an experienced team with an understanding of user research, content design and service design work.

CivicActions understands the need for user research and testing. Our human- centered and data-driven design team works to align with the USWDS and the specific design systems and style guides of our government clients. CivicActions' content design team is also working to establish plain language text that is easy to understand and more accessible to the average user.

We know people come to government websites in order to find information and we work with our clients to build an experience that allows users to quickly find what they need. The OMB's policy guidance (M-23–22) embraces the idea that search is an important piece of a digital first public experience. It is important that government pages are optimized for search engines, but also that they have a meaningful internal search capacity. Our team has experience customizing the search experience in Drupal for our clients. There is a lot of customization available within Drupal, but we have also leveraged Search.gov services.

We are working with our clients to find ways to embed dynamic experience for our users in their sites. We understand that government often requires users to complete complex tasks and we continue to utilize best practices that align to the IDEA act and the OMB's memo in order to simplify their workloads.

Accessibility and Digital-First Public Experiences

Finally, for the first time, per the OMB memo, agencies are directly encouraged to do the following:

• "Apply the most current Web Content Accessibility Guidelines (WCAG) published by the World Wide Web Consortium (W3C) to websites and web applications, where possible."
• "Include automated scanning, manual testing of websites, and usability testing with people with disabilities, as well as testing with users of adaptive technologies."
• "Incorporate the needs of individuals with disabilities into the design and development of websites and digital services, and should include individuals with disabilities in usability testing of new tools or features."

The guidance from the OMB re-enforces the need for government agencies to focus more on accessibility. The Spring 2023 Section 508 Program Maturity Report, demonstrated accessibility is not being consistently well managed across government agencies. Citizens with disabilities should be able to expect that they can engage with government agencies. This has been understood as a right for citizens for over 30 years, yet it is still a challenge. The OMB Memo provides very specific guidance on how agencies should be addressing accessibility.

Accessibility is at the heart of CivicActions mission; our team is taking a leadership role in Drupal's accessibility, and is engaged in several other open source projects that focus on accessibility as well. Our staff includes people with disabilities, and through our Champions Program and Accessibility Onboarding, we ensure that this value is understood by every team member.

Where We Are Going

Our team is continuing to innovate in digital government. Our work with the open source communities, like the USWDS & Drupal, allow us to regularly engage with other teams, projects, and agencies. By engaging with others, we learn and contribute to the development of best practices. We can then bring these approaches back to our clients, improving the experience for all. Reach out if you are interested in learning more about our approach — we'd love the opportunity to talk about what we can do together.

Delivering Digital First: Turning 21st Century IDEA into Action was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

When we talk about testing websites for accessibility, one of the most frequently cited statistics is that automated testing discovers 30% to 60% of the errors we need to address. (Aside: that discrepancy seems to be due to how you define the number of errors, by type or frequency.) No matter which number is cited, this means a significant number of accessibility violations, around 40–70%, still require an actual human user to verify.

This past year, from November 2022 to now, the accessibility practice for the Web Experience and Content Management Services (WECMS) team, which supports the Centers for Medicare and Medicaid (CMS), has increased our manual testing efforts. With all of this in mind, I thought it would be interesting to quantify how much effort we spent verifying accessibility issues via automated testing, manual testing, or a combination of both — especially as this relates to concerns raised in the recently published Government-wide Section 508 Assessment. A few of the criteria directly address manual testing efforts:

13/ If your agency uses a manual and/or hybrid ICT accessibility test methodology for web content, which manual test methodology or methodologies does your agency use?

47/ Indicate how often your agency conducts comprehensive manual conformance validation testing for web content (internet and intranet) prior to deployment to address all applicable Section 508 standards.

68/ How many public internet web pages were evaluated by a combination of both automated and manual testing in the reporting period?

Assessment methods

We can determine which form of testing — automated, manual, or both — was required for our project's tickets by looking at the Web Content Accessibility Guidelines (WCAG) success criteria that apply to each issue/ticket. A very small number of WCAG criteria can be verified by automated testing alone; the vast majority require some degree of manual review. For example, an ARIA label that satisfies 1.3.1 Info and Relationships can be detected by an automated test, but the value must be reviewed manually by a person to confirm that it makes sense.

First, I added labels to all accessibility issues in Jira resolved between November 2022 and 2023 that reference the relevant WCAG criteria (i.e., "WCAG-2.5.3"). While this exercise was somewhat time-consuming —we logged around 300 accessibility issues total across 7 websites — it was well worth the time as it will greatly simplify updates to our upcoming annual reporting process. So, win-win.

Then I mapped the testing method to each criterion in this matrix (see Appendix A: Required testing methods by WCAG criteria), based on a similar matrix published by Usablenet and the axe-core rule definitions. Each axe-core rule is assigned an Issue Type, which notes whether a rule can result in failure (a clear accessibility violation), or if it needs review, which Deque defines as, "there may or not be an accessibility issue and more investigation is required." If an axe-core rule's issue type includes "needs review," that rule, and by extension the WCAG criteria tested, may require manual testing.

Results

We logged around 300 accessibility issues total, however only 230 tickets represented WCAG violations (the rest related to best practices or discovery). Of those:

• 146 required both automated and manual testing
• 72 could be verified with manual tests only
• 12 could be verified with automated tests only

Summary

The WECMS team made great strides with manual testing in 2023, and testing in general:

• Most of the WCAG violations we fixed were verified by a person using a keyboard or screen reader, and/or by considering context, meaning, purpose, logical placement, and allowed exceptions.
• Code written by our engineers consistently has few objective errors that can be discovered with automated testing. I can vouch for our design and development teams using semantic markup, including text labels, checking color values, and overall creating templates and experiences that will pass automated checks.

This exercise showed that manual testing is not only necessary, it accounts for most of the time we spend verifying accessibility conformance for the WECMS websites. We can now use these results to inform how much effort we put toward, for example, team training on screen reader use, and it can fine-tune how we prioritize time and resources for remediating issues.

It's important to remember that while we're growing more proficient with manual testing tools and methods, usability testing with disabled people — the true experts when it comes to assessing a website's accessibility — should continue to be a top goal for our accessibility practice.

Measuring the Impact of Manual Accessibility Testing was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

At CivicActions we have pledged to advance our accessibility maturity to provide critical human services to as many people as possible so that they can get the help they need, as well allowing service providers to readily engage with their users. The commitment here is twofold (at least). Firstly, allow users of the site to not only access information the site provides, but to make that process as easy as possible regardless of their accessibility challenges. Secondly, allowing service providers to reach as many users as possible while keeping those users engaged, and keeping the site first and foremost in the users' minds for subsequent times they need information.

Reaching this higher level of maturity includes, but is not limited to:

• Integrating the highest standards of testing into our continuous integration processes
• Automating site-wide testing and historical analysis
• Implementing site-wide rankings and improvement targets into the agile process
• Include manual testing to catch issues that automated testing cannot evaluate

To that, the team that works for the Defense Security Cooperation Agency (DSCA) supporting the Department of Defense (DoD) Regional Centers has spent the last year (2022 Q4 to 2023 Q3) monitoring accessibility issues on the platform and working to improve them. We use browser extensions, WAVE and axe DevTools to check for issues on individual features we work on. To capture issues across many pages, we have set up Pa11y CI running axe-core and HTMLCS and Cypress with cypress-axe plugin. Axe-core is an industry standard automated accessibility testing engine. The tools are all run automatically using GitLab CI/CD pipelines.

Our efforts to improve accessibility have not produced any reported site disturbances, slow-downs, or errors; the feedback from our users has been overwhelmingly positive:

"As far as accessibility goes, this system is way ahead of other DoD systems that I've used." — Dwayne Wood Ed.D., Curriculum & Training Developer, Ted Stevens Center for Arctic Security Studies

While it is always nice to receive kudos, praise like this makes us aware that accessibility is also important to our stakeholders, and that our efforts have not gone unnoticed.

Issues explored and URLs Scanned

The team's goal was to, at the minimum, attain Section 508 and WCAG 2.0 AA standards and our plan is to work on reported issues in quarterly epics (an epic is a large chunk of work that is segmented into smaller tasks, used to organize tasks and create hierarchy in the development process). While automation only catches about 33% — 50% issues, those are still important to fix. Following the axe issue impact, the team focused first on critical and serious issues. The team also focused on moderate/minor issues that are common to many pages where possible.

We configured each tool to assess a set of URLs with our Pa11y scans. We focused on the URLs that are the main pages and also represent common features. With Cypress-axe the focus also includes URLs for organization manager and member roles. The URLs cover:

• Home Page
• Login
• Organization Management
• Support Form
• Policies
• Organization/Groups/Courses
• Account Management
• Search
• Guide
• 404 Page

The common features are:

• Headers
• Links
• Tabs
• Buttons
• Breadcrumbs
• Images
• Text/Link Colors
• Form Fields
• Various Text Displays
• Tables

Reduction of issues over each quarter

Here is our progression as we started fixing the system's accessibility issues as reported by the two different tools.

Pa11y-CI

Run against 15 production URLs:

• Start of Q1, number of issues: 118
• Start of Q2, number of issues: 66
• Start of Q3, number of issues: 12
• As of August 15th: 0*

Cypress

Run against 33 URLs (the additional URLs are tested with authenticated users who have different roles):

• Start of Q1, number of issues: 156
• Start of Q2, number of issues: 68
• Start of Q3, number of issues: 33
• Start of Q4, number of issues: 5
• As of Oct 5th: 0*

Additional Criteria

In our analysis of accessibility issues reported by the tools and our efforts to fix them, we identified the following issues that went unaddressed:

• Color-contrast: we skipped this because the system theme uses background images in CSS. So we get a lot of reports of color contrast reports requiring manual review from both tools.
• Nested-interactive: we skipped this because of the use of jQuery UI tabs that rely on nested implementation to work. We did confirm manually that the tabs are accessible with a keyboard.
• Aria-required-children: similar to the previous one, jQuery UI accordions creates HTML elements with non-standard aria roles, throwing this error. We did confirm manually that the tabs are accessible with a keyboard.
• Duplicate-ID, duplicate-id-aria: we skipped these on a few pages because in some cases making the ids unique caused existing features to not work as expected.
• Heading-order: in a future ticket, we will tackle adjusting header order for multiple pages (see challenges for details).
• We also skipped checking accessibility of third-party widgets such as Google Translate as we cannot control the HTML they render.

Challenges

Challenge : CSS used across multiple element types: changing the element to be accessibility compliant sometimes means updating the CSS, which in turns breaks another element sharing that CSS styling.

Possible solution(s) : Create smaller CSS "classes" so that elements which need modification can have their CSS attributes updated with minimal effects on other elements. Elements that need a number of CSS attributes can implement groups of these smaller CSS classes. If, while fixing an accessibility issue, a conflict arises because two or more elements are sharing a class, we can create another small CSS class to swap out for the pre-existing class to resolve the conflict.

Challenge : When attempting to have our header hierarchy meet accessibility standards, we found out header elements were tightly coupled with CSS and JavaScript functionality. For this reason, it is not as simple as swapping out h2 for h3, because the combination of styling and JavaScript leads to misalignment, or broken functionality i.e. a menu heading that does not open when clicked.

Possible solution(s) : This is similar to the CSS challenge. Break down the CSS/JavaScript so that the header elements share as few attributes as possible.

Challenge : Using an image and teaser text within a link to an article caused accessibility issues. In this case the image, which has its own descriptor text when a user hovers, causes a context error since it is in the scope of the link, but does not describe to what it is being linked.

Possible solution(s): To rectify this issue, we decided to create a teaser card element in which the image and teaser text would reside and the card containing those elements would provide context to the linked page.

Challenge : Color contrast adjustments to meet WCAG AA standards is not problematic, however getting to WCAG AAA standards will be a bit more challenging as the range of colors to achieve the correct contrast ratio is a bit more narrow, and one must also consider the link color contrasts in addition to the background color, and foreground color contrasts.

Possible solution(s) : If you already have a comprehensive style guide, then this should not be too much of an issue. However, if you don't, then the creation of a style guide would go a long way to helping to keep color contrasts meeting accessibility standards. Also, it's not enough for the guide to have one foreground color A contrasts well with one background color B, but maybe include a range of colors that are acceptable for the site.

Challenge : Dealing with third party issues. For example, when trying to clean up the AXE accessibility tool error attribute IDs must be unique , we found that the methodology the Date module uses to construct the wrapper IDs was the cause of the issue.

Possible solution(s) : Making changes to third party software can be problematic in that a modification can solve your problem, but runs the risk of causing problems for someone else. The recommended course of action is to create a patch file which solves your problem, and share the patch with the overall community for review and testing.

Manual testing and fixes

In our progress to fix accessibility issues, we also did manual testing using a keyboard and found certain elements of the site didn't work. These are issues that automated tests will not catch. We fixed those as well to improve the keyboard navigability of the platform.

What's next

Overall, we were able to improve the accessibility of our site through

• Having guidelines to direct our efforts
• Making a plan based on how effective our implementations would be
• Making use of automated testing
• Integrating manual testing/checking with the automated testing
• Keeping an eye on which issues could put on the back burner

However, accessibility is not a one and done aspect of any project. It requires constant vigilance. To that end, we are looking forward to continuing our work with these high-level to-do items:

• Add more URLs to each list. Cover more areas of the site that might not be covered in the examples above.
• Share results of the Pa11y CI with users who primarily provide content, like our organization managers. They can use the results to make sure they are not introducing accessibility issues with their content.
• Cypress accessibility tests run whenever we push code changes; we are waiting to get down to a few remaining issues to switch it to running automatically instead of on demand. We will also switch the pipeline from allowing failure to requiring the pipeline to pass, this should prevent accessibility issues from being introduced whenever we push code changes.
• Add more dynamic element testing like menus and interactive components. This is to make sure when a menu is open or some element is interacted with that the results are also accessible. This can be accomplished with both Pa11y-CI and Cypress testing.

In conclusion, while our process might not be appropriate for every use case, however, it is important for everyone to put some effort into increasing the accessibility of their site. Our efforts have not only increased the accessibility of the government site we support, but has also made us more cognizant of accessibility challenges. This awareness contributes to the raising of the best practices bar for our developers and other members of our team. In addition to the success of incorporating accessibility into our coding and design process, success can also be measured by what we are not hearing; namely bug reports and complaints regarding usability. In that regard, to date, our efforts have proven to be successful.

* Zero accessibility automated issues doesn't mean the site is without barriers, but it does mean 30% to 50% of potential issues on the URLs we test are clear.

How We Helped a Government Website Achieve Zero Accessibility Errors* was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

Mike had this article published with the Federal News Network celebrating the approach taken by the Office of Management and Budget, U.S. Department of Justice and the GSA to collect accessibility information from government agencies.

Seeing results every six months will also be so important to be able to track progress on meeting the Seciton 508 goals, but also achieving the government's Diversity, Equity, Inclusion and Accessibility (DEIA) goals.

Mike had this article published with GitHub's ReadME Project looking back over what worked with Drupal, and how it could apply to more open source projects.

Drupal is probably the most accessible content management system in the world. This didn't happen over night, and it has taken the input of hundreds of people. Still there are other software communities that can learn from this to make their code more accessible.

Mike had this article published with the Federal News Network on how the USA government can learn from best practices adopted in Europe with the Web Accessibility Directive.

There is such a great opportunity to learn from and build on the work done by the European Commission. Right now they are leading the world in digital accessibility.

We spoke to CivicActioner Daniel Mundra, Associate Director of Drupal, about the benefits of Drupal, community contributions, and why working at CivicActions has been so rewarding for him . He is among a team of 60+ Drupal engineers with nearly 20 Acquia certifications and 20+ edX a11y (Accessibility) certifications.

CivicActions is known for our Drupal expertise. To you, why is Drupal such an important tool in the work that we do for our government clients?

Drupal is a complex tool but it is the best tool to help deliver complex concepts for our government clients. Drupal also delivers for our clients while still being free, open source, having a growing community, and also not afraid to change/adapt. I don't believe other similar products could check all those boxes.

I have been working with Drupal for 12+ years, first with Drupal version 6 and now with 10. Since my early days, Drupal has been a veritable Swiss army knife for web development. It helped me deliver a lot of value to users in a quick amount of time. Earlier I would produce many sites with a lot of modules but not consider the maintenance costs, reliability you get with writing tests, giving back to the community, and so on. It left my clients with a functional site but a maintenance burden. Nowadays I focus more on contributing back, writing more tests, reducing maintenance burden and thinking of long term solutions than quicker output of sites. Just like how the Drupal platform and community has evolved, so have I. Drupal has given me the space to do that and I am grateful for that.

What are some community contributions to Drupal from your team that you are especially proud of?

CivicActions contributes quite a bit to Drupal. Last year CivicActions was credited with contributing 90+ contributions and that is only growing. I am particularly proud of the modules we are adding to the community, like Media Link Enhancements (from Cameron Prince), Google Programmable Search JSON API (from Timo Zura and Sam Lerner), and recently Content Model Documentation (from Steve Wirt).

You are very passionate about accessibility — why is accessibility so important to you and how do you work to make projects more accessible at CivicActions?

Accessibility is important to me because to truly be human centered we have to make technology accessible to all. The primary ways I work to make projects more accessible is by first building from the beginning with accessibility in mind. The second way is to add layers of accessibility testing to ensure the site remains accessible. Layers like automated testing, manual testing with keyboards/screen readers, and testing with disabled users (when possible). Accessibility is a long term commitment and I can say that at CivicActions we are all contributing to that and we are making a difference.

What initially drew you to a career in civic tech, particularly at CivicActions?

After spending 10 years working in higher education, in particular as an in- house developer for the University of Oregon, I wanted a change. As the university was going away from in-house development, I wanted to continue growing my skills while still supporting similar mission-driven organizations. I found both at CivicActions.

What is the most rewarding part of your job?

Being able to give back to the community and also take part in the company outside of the project. I am an associate director now but I wouldn't have been able to reach that role without the support of everyone at CivicActions. They made space for me to take part in company activities like OKRs, guidebook updates, accessibility practice area, and so on. I am successful because of the culture of the team to make space and make most parts of the company accessible to discussion, debate and continuous improvement.

Any advice for someone looking to follow a similar career trajectory?

Ask questions and get involved where you can, either at the company you work for or in your local community. Find that company and community that energizes you and bring your whole self to them.

On a lighter note, what is a fun fact that people may not know about you?

I like statistics so much that for 3 years I was the official statistician for the Oregon Cricket League. Lots of data entry and review but I was able to see how local players and teams evolved.

Daniel Mundra: Diving into Drupal was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

"We lost our vision due to our service…my vision gets blurry," says a Veteran over Zoom during a user research session.

We were testing new designs for a web-based app helping Veterans check into a medical appointment.

This Veteran's experience is not uncommon: about 30% of Veterans live with a disability (but that number is likely much higher due to military culture). Almost 7,000 Veterans become newly blind or visually impaired every year. And disabilities can vary in type and severity — the most common of which include hearing loss, post-traumatic stress disorder (PTSD), scars, back and neck pain, and migraines.

Figure 1: A few days before an appointment, a Veteran receives an SMS text with a link to start pre-check-in. After selecting the link, the Veteran sees the login page (first image). When they complete pre-check-in, they see the submission page (second image). Figure 2: When the Veteran arrives on the day of their appointment, a poster with instructions to check-in is placed in the facility. The Veteran scans the QR code or texts '5309' to check into their appointment using their phone. The Medical Support Assistant (MSA) receives a notification that the Veteran has checked-in to their appointment. The Veteran waits until called in.

Checking into a medical appointment with a smartphone has barriers

For example, it requires a Veteran to:

• Own a phone (or have access to one)
• See the check-in poster at the facility
• Have reasonable tech-savviness to use SMS or a QR code
• Understand the steps within a reasonably short time-frame (which can be difficult for some Veterans with cognitive disability or PTSD)

We wanted to design digital and non-digital solutions that helped make it more usable for disabled Veterans to check into medical appointments.

But we had a lot of questions.

• Are we able to design for the range of types and severity of disabilities?
• Would some solutions only benefit some Veterans, while introducing barriers to others?
• How do we decide on those trade-offs?

We created an Accessibility Use Case Map

An accessibility use case map is a framework to evaluate proposed solutions across multiple different types of disabilities affecting Veterans. We considered both digital and non-digital solutions for each phase of the check- in process.

A spreadsheet makes it accessible

As we note later in the article, we worked with an accessibility specialist who uses a screen-reader. We chose to design the map in a spreadsheet. This made it easy for our colleague to offer feedback.

Figure 3: Example of accessibility use case map. Solutions are indicated on the x-axis. All nine disabilities are listed on the y-axis. Each solution is assessed for degree of impact: impactful, helpful, it depends, not relevant, and inaccessible. This criteria is discussed later in this article.

We wanted our map to help prioritize solutions that maximized benefits for Veterans living with a variety of disabilities, while minimizing the chances of introducing barriers.

Here's how we created the accessibility use case map.

Step 1: Developed proto-personas for each type of disability

We chose nine of the most common types of disabilities reported by Veterans:

• Blindness
• Low vision
• Hard of hearing/deafness
• Cognitive disability
• PTSD
• Motor disabilities
• Military Sexual Trauma (MST)
• Deafblindness

Time and resources were limited. So we used secondary sources and "best estimates" to develop light-weight personas of each disability type ( proto- personas). Each described the behavior, symptoms, needs, and tools used to manage the condition. This allows for quicker, iterative research.

Figure 4: Example of a proto-persona for a Veteran living with low-vision. We used secondary research sources to validate our assumptions on behaviors, needs and challenges. Details of the post-it notes are intentionally not-visible, and are shown for example purposes only.

Step 2: Created user journey maps to identify pain points and possible solutions

Based on the proto-personas, we created user journey maps across the medical appointment experience:

• Before checking into an appointment
• During the appointment, and;
• After the appointment

We proposed solutions to help address pain points across the journey for each type of disability.

Figure 5: Example of user journey map for a Veteran living with low vision and may need caregiver support to prepare for the medical appointment. Details of the post- it notes are intentionally not-visible, and are shown for example purposes only.

Step 3: Mapped each solution across multiple disability types, creating our accessibility use case map

But what might benefit someone with one type of disability, might be a barrier for others. For instance, text-to-speech software is helpful for a blind Veteran trying to access their appointments. But it may be inaccessible to a Veteran who is hard of hearing.

Due to this challenge, we decided to map each proposed solution across the 9 most common disabilities affecting Veterans.

We listed recommendations along the x-axis. We then listed the nine disabilities along the y-axis.

For each type of disability, we assessed if the solution was:

• Impactful (positive and we believe will enhance the experience for this disability)
• Helpful (positive and it's not specific to this disability)
• It Depends (it may depend on the degree or type of disability, or the recommendation is too complex for us to weigh in at this point)
• Not Relevant
• Inaccessible (it's a barrier for a person with this disability)

Step 4: Iterated with various accessibility experts

Time constraints didn't allow us to validate with users (which would've been ideal). Instead, we iterated on our map with various accessibility specialists:

• Josh Kim (Accessibility Designer)
• Martha Wilkes ( Accessibility Strategist at the VA Office of the Chief Technology Office)
• Melissa Manak (Senior User Experience Mobile Designer)
• Angela Fowler (Senior A11Y Specialist and Designer)
• Vanessa Luxen (an IAAP certified professional and former Audiologist at CivicActions)

Some of the experts we consulted also lived with a disability. This helped ensure we were including the lived experiences of the very people we were designing for.

Step 5: Developed an "Accessibility Impact Score" for each recommendation.

Working with our accessibility experts, we assigned a value depending on how helpful or unhelpful the solution was for each type of disability. We called this our "Accessibility Impact Score."

• 3 for Impactful (positive and we believe will enhance the experience for this disability)
• 2 for Helpful (positive and it is not specific to this disability)
• 1 for It Depends (it may depend on the degree of type of disability, or the recommendation is too complex for us to weigh in at this point)
• 0 for Not Relevant
• -2 for Inaccessible (it is a barrier for a person with this disability)

We then tallied the number of scores for each solution to calculate the final "Accessibility Impact Score."

This helped us assign a numerical value to evaluate the impact each recommendation would have across multiple disabilities. For instance, improvements like "sending an SMS notification with a link to pre-check-in" was one of our highest impact solutions. This meant it was impactful for the greatest number of disabilities.

Step 6: Prioritized recommendations based on the Accessibility Impact Score

Using a prioritization matrix, we prioritized each recommendation based on its total Accessibility Impact Score.

We refined it further with our product team based on scope and feasibility, so we could eventually choose the recommendations we wanted on our roadmap.

We took some risks and made many assumptions

There are many caveats with our framework, for instance:

• It favors specific solutions that are impactful for a few disabilities and may end up with a lower score vs. general solutions that help a variety of disabilities.
• We did not capture the complexity within each disability type. For example, cognitive disabilities can include a range of disabilities that vary in severity, symptoms and more. We recognize disabilities are not a monolith and the experiences are complex and varied.
• It doesn't take user preferences or familiarity with the technology into account
• It's not intersectional — it doesn't consider people who live with more than one disability
• We didn't test it with users living with disabilities.

This is our first attempt at designing for disabled Veterans when resources are limited, time-frames are tight, and priorities conflict.

Ideally, disabled Veterans would be part of our process

What would we have done differently without those constraints? We would have included disabled Veterans throughout all six steps of the process. This way we'd design with (not for ) the disabled Veteran community.

What's next?

This framework is dynamic. We plan to continue to iterate as we address our many assumptions. We hope to start conducting user research sessions with Veterans living with disabilities to validate (and invalidate) our proposed improvements.

Authors: Nira Datta, Senior Content Designer at CivicActions; Ya-ching Tsao, Product Designer at CivicActions

Helping disabled Veterans Check-In: Designing an Accessibility Map was originally published in CivicActions on Medium, where people are continuing the conversation by highlighting and responding to this story.

Full post: How I do automated accessibility testing for my website