Building testing tools for ARIA live regions
ARIA live regions inform users who use assistive technologies, such as screen readers, about dynamic content changes in applications. Typically these dynamic changes happen to a part of the page which the user is currently not focusing on. While sighted users are able to see these changes happen on the page, assistive technology users are unaware of them. In a case where the screen reader is reading a text block in the middle of the page, it simply cannot detect changes happening on top of the page. ARIA live regions are used to inform assistive technologies about these changes.
In this blog post I'll go through a bug case we encountered when building live regions with Javascript frameworks. After analyzing the root cause we identified our @testing-library driven tests to be false positive and ended up building a new assertion library and live region detection tool.
Live region use case examples
In practice ARIA live regions can help with various use cases. With aria-live="assertive" we can inform users about critical alerts requiring user action. Because these announcements have high priority, they will interrupt the assistive technologies’ current reading. For example, assertive messages can be used when a user tries to submit a form without filling all the required fields.
Typically assertive live regions are implemented by using role="alert". This role has implicit aria-live="assertive". Assistive technologies may also trigger a special system alert sound before announcing contents of alert messages and include other indicators to describe them, for example by adding "alert" in the beginning of the announcement.
With aria-live="polite" regions we can inform users with non-critical messages, which are often used to give user feedback about their actions. For example, in a multiselection listbox users can be informed about their selections: "Selected apples, four selections active".
Polite live regions informing users about the state of the page can be implemented by using role="status". This role has implicit aria-live="polite". Assistive technologies may include other indicators to describe these as status messages, for example by adding "status" at the end of the announcement.
Bug report: Initial content of live regions are not announced
To fulfill requirements of WCAG 3.3.1 or 4.1.3, applications are likely required to utilize live regions. ARIA 1.1 states that user agents and assistive technologies should react to live regions updates — the initial content of the live region is not considered as an announcement. This small detail is often forgotten by implementers.
There are differences in how the most common screen readers announce live regions. While Apple's VoiceOver announces basically anything wrapped in a live region, NVDA strictly follows the ARIA specification. NVDA doesn’t announce initial contents of live regions — except role="alert" ones.
This issue is demonstrated below. Example 1 contains a search form which fetches resources from the server. Items from a successful response are appended to the table below the search and the user is notified of this content change using a status live region.
Example 1: Incorrectly defined live region
<!-- Render #1 --><form><input type="text" value="Unicorns" /><button>Search</button></form><!-- User clicks Search button --><!-- Action is successful and feedback is provided to user --><!-- Render #2. Status message is not announced ❌ --><div role="status">Five matching results added to table below.</div><form><input type="text" value="Unicorns" /><button>Search</button></form>
In example 1 the live region and its content are appended to DOM at the same time. Assistive technologies which only track updates of the live regions do not detect this as there are no updates. If we changed the contents of the live region now, assistive technologies would detect it.
Example 2: Correctly defined live region
<!-- Render #1 --><div role="status"><!-- Note: Live region is present in DOM already --></div><form><input type="text" value="Unicorns" /><button>Search</button></form><!-- Render #2. Status message is announced ✅ --><div role="status">Five matching results added to table below.</div><form><input type="text" value="Unicorns" /><button>Search</button></form>
In example 2 the live region is present in DOM before the user triggers a search. Assistive technologies are able to start tracking the live region and can respond to its updates. Message is announced correctly.
These broken live regions come up typically when using component driven dynamic JavaScript frameworks. When handling visibility of a small part of the application it is not that easy to trigger updates of an element.
Example 3: Incorrectly defined live region in JSX based frameworks
const Notification = ({ children }) => (<div role="status"><Icon aria-hidden="true" />{children}<CloseButton aria-live="off" /></div>);// Application utilizing Notification component. There is no easy way to control// the live region attribute of Notification component.// Notification is not announced ❌{results.length > 0 && (<Notification>{results.length} matching results added to table below.</Notification>)}
There are a couple of ways to work around these pitfalls. Ideal solution would be to lift the live region attributes up in the document tree to an element which is present before the updates happen. However, you can’t apply this solution when you create commonly shared components that define themselves as live regions. This issue is visible in example 3 which includes a <Notification> component with role="status". Consumers of this component cannot control its attributes.
One way to perform an update into a dynamically revealed element is to first append the element's live region into the document without content. The actual content is appended inside the element after a small delay. This will trigger a live region update and assistive technologies are able to announce it. However, this will create a visual blinking effect to sighted users.
Example 4: Live region update triggered by toggling aria-hidden
<!-- Render #1 --><div role="status"><div aria-hidden="true">Five results found.</div></div><!-- Render #2 --><div role="status"><div aria-hidden="false">Five results found.</div></div>
Theoretically we should be able to trigger live region update by changing its content's aria-hidden attribute. Initially the content would be rendered inside the live region but would be excluded from the accessibility tree by using aria-hidden="true". After a small delay the aria-hidden is toggled to "false" and contents of the live region are updated. This would control the visibility of the accessibility tree instead of the actual visual visibility in order to avoid visual blinking of the content.
This approach should work in theory but at least ChromeVox seems to ignore it. It seems like it’s tracking DOM API instead of accessibility tree. However, NVDA and VoiceOver announce these perfectly.
Example 5: A single live region defined at the end of the document
<body>...<form><input type="text" value="Unicorns" /><button>Search</button></form>...<!-- A single live region to handle announcements of whole page --><div id="live-region" role="status">Five matching results added to table below.</div></body>
The most confident solution is to use fixed live regions at the end of the document. These live regions are present on the document right after the initial page load and provide an easy way for updating the contents. Elements requiring announcements can easily query the live region based on its id attribute and update the contents.
Testing ARIA live regions
When I first encountered this issue I wanted to make sure we could catch these mistakes in unit and integration testing. We were already seeing these coming up in manual testing but that was considered too error-prone.
I've been a fan of @testing-library ecosystem for a couple of years due to their accessibility focused APIs - mostly enabled by the awesome dom-accessibility-api integration. However, it was quickly clear that testing live regions with @testing-library could easily lead to false positives. Creating confident tests required verbose and strictly structured tests.
Example 6: False positive live region test with @testing-library
// Test will pass even though loading is not announced. ❌test("Loading should be announced", () => {render(<div role="status">Loading...</div>);const statusContainer = screen.getByRole('status');expect(statusContainer).toHaveTextContent('Loading...');});
Example 6 describes how we had built our tests before. We were asserting that DOM contains an element with role="status" and that this element has the expected announcement as text. This test seemed to be asserting all the required features but was actually falsely passing. It did not check whether the text content of the live region had changed at all.
Example 7: Verbose live region test with @testing-library
// Loading is announced ✅test("Loading should be announced", () => {const { rerender } = render(<div role="status"></div>);// Status container should be presentconst liveRegion = screen.getByRole('status');const initialTextContent = liveRegion.textContent;// Update content of live regionrerender(<div role="status">Loading...</div>);// Text content should have updatedexpect(statusContainer).not.toHaveTextContent(initialTextContent);expect(statusContainer).toHaveTextContent('Loading...');});
After refactoring our tests to validate live regions correctly our tests became verbose as shown in example 7. In order to confidently test live regions with @testing-library we had to write a lot of code. In larger test cases this quickly reduced readability and maintainability. We needed a way to abstract this assertion behind a utility. There were no existing solutions in the ecosystem available so we needed to build our own.
Our goal was to provide a simple API for asserting live region announcements in tests. Only updates of live regions should be tracked in order to flag previous false positives and prevent shipping broken live regions. To keep tests simple and easy to debug we desired a synchronous API. In practice this meant that we couldn’t use the MutationObserver API due to its asynchronous nature. Instead, we should hook into DOM API methods directly.
I started to build aria-live-capture to track DOM API methods that manipulate the document. It attempts to simulate how browsers expose accessibility API events of live regions and how assistive technologies react to these events. This tool relies only on DOM API and can easily be integrated to various tooling, e.g. Jest, Cypress and Storybook. It can track various DOM interfaces and supports most of the live region attributes and elements. In addition to regular DOM it can detect changes happening inside Shadow DOM.
To finally test the live regions reliably in tests run by Jest, I built the extend-to-be-announced assertion library. It provides a simple API for asserting announcements. As the underlying aria-live-capture implementation relies purely on DOM APIs this assertion library can be used with any web framework.
Example 8: Test asserting live region with extend-to-be-announced
// Loading is announced ✅test("Loading should be announced", () => {render(<App />);const search = screen.getByRole("button", { name: "Search" });userEvent.click(search);expect("Five matching results added to table below").toBeAnnounced('polite');});
Example 8 describes how simple it is to assert live region announcements confidently. The underlying tooling abstracts tracking of DOM nodes and provides a simple API for validations.
What I learned
Building your own tooling is a great learning experience. In addition to being able to use it in multiple real-world production application tests, you'll end up diving deep to analyze the root causes.
I learned a lot about ARIA specification and how complex live regions are. There are so many different attributes to consider, e.g. aria-live, aria-atomic, aria-relevant, aria-busy, 5 different role values and one specific element type. Testing different browsers and screen readers revealed unexpected results and even led me to inspect accessibility API events using a developer version of a browser. Adding support for various DOM APIs was rewarding — until I realized how much more work is required to include support for ShadowRoot.
It's also worth noting that when designing dynamic page content changes we need to consider users with visual impairment, too. When users relying on screen magnifiers may be focusing on a small part of the application they are likely to miss changes happening outside this area. These cases can be tricky since screen magnifier users may not be using screen readers at all — aria-live announcements won't help here.
While we can build tools to help our development and testing processes, it is always extremely important to test final products with the actual assistive technologies. Developers should not blindly trust browser's accessibility trees and other related tooling. Real confidence comes from manual testing with targeted technologies — exactly as the end users use our applications.
To use the tools mentioned in this post see links below.