Accessibility Review for AI-Generated UI

The agent ran axe and gave a clean report: contrast fine, alt text present, and closed the ticket.

Then a keyboard user opened the dialog and focus fell through to the page behind the overlay. An empty required field painted a red border and said nothing to the screen reader. axe couldn't have caught either one: it checks markup, not what happens when you press Tab.

That pattern shows up constantly in AI-assisted UI. Large language models are reliable at the visible wins: contrast, basic alt text, an aria-label on an icon button when you ask; and unreliable at the behavioral ones: focus management, programmatic labels that stay tied to errors, live regions that announce status changes to screen readers, semantics, and composite-widget keyboard patterns. Deque's coverage research puts automated detection at roughly 57% of issues by volume. The other half is where agents smile and move on.

Note the denominator: Deque is measuring issue volume, not the share of WCAG success criteria that tools can touch. By success-criteria count, automation looks closer to the old 20?30% folklore. By the bugs you actually fix on a real page, automation earns its keep ? and still leaves focus order, keyboard access, and meaningful names largely to you.

In healthcare UI, that gap is not an abstract WCAG miss. It is a clinician who cannot finish a task under time pressure, or a patient who cannot book care. So every AI-generated component gets a short, targeted human review aimed at exactly what agents skip.

What the research says

The last two years of papers are blunt.

A W4A 2025 study found that accessibility-oriented prompts help, but semantic-structure barriers persist. You can tell the model to "make it accessible" and still get a div that acts like a button with the wrong role relationships underneath.

arXiv 2503.15885 compared human and LLM accessible code generation. Naive LLM output often scored more accessible than the human-written originals, especially on contrast and alt text, while ARIA remained weak. Feedback loops beat prompt tricks. In these studies, one-shot prompting didn't hold up. Iteration against a failing check did.

ASSETS 2025 reports evaluating six AI-generated sites and logging 308 accessibility errors, with just over half classed as cognitive (the abstract states 52.93%). The failures were not only missing labels. They were interfaces that were hard to understand and operate once you left the mouse behind.

A 2025 benchmark of AI-generated HTML found models frequently fall short of full WCAG compliance without follow-up prompting; Claude 3.7 Sonnet and GPT-4o needed less of it than the rest. The researchers verified outputs with keyboard and screen-reader testing. That was their evaluation method, not a finding that every model requires those checks to improve. Follow-up prompts that named concrete failures helped; human oversight still closed the gap.

Takeaway: prompting is necessary and not sufficient. Instructions raise the floor. They do not replace a human who can Tab through the dialog. If your process ends at "we added an a11y bullet to the system prompt," you've written down what you hope happens, not what you've verified.

What automation covers, and what it can't

Use automation aggressively. Just do not confuse it with a release gate on its own.

Playwright's own accessibility docs are explicit: many issues need manual testing. Focus traps, focus return, live-region timing, and "does this name make sense in context" sit outside what a rules engine can assert with confidence.

Pair axe with the behavioral checks it cannot do:

test('referral dialog is usable from the keyboard', async ({ page }) => {
  await page.getByRole('button', { name: 'New referral' }).click();
  const dialog = page.getByRole('dialog', { name: 'New referral' });

  // Automated pass ? scoped, tagged
  const results = await new AxeBuilder({ page })
    .include('[role="dialog"]')
    .withTags(['wcag2a', 'wcag2aa', 'wcag21a', 'wcag21aa'])
    .analyze();
  expect(results.violations).toEqual([]);

  // Behavioral pass ? axe will not do this for you
  await expect(dialog).toBeFocused(); // or first focusable inside
  await page.keyboard.press('Tab');
  // ?assert focus stays inside?
  await page.keyboard.press('Escape');
  await expect(page.getByRole('button', { name: 'New referral' })).toBeFocused();
});

The axe block catches the machine-checkable half. The Tab / Escape assertions catch the half that shipped broken in the opening story. Keep both. Run axe early so the agent fixes contrast and missing names before you spend the keyboard pass on structural problems. Run the keyboard assertions in the same file so a green CI job means both layers passed, not just the one that is easy to automate.

The checklist agents skip

This is the ten-minute pass. I'd paste it into the PR template. Point the agent at it as MUST / SHOULD rules, then verify by hand.

Focus order and management

DOM order should match visual order. Watch Tailwind order-*, flex-row-reverse, and grid placement that reshuffles tab order away from what the eye sees.

No positive tabindex. Dialogs move focus in, trap it while open, and return it to the trigger on close. Focus must survive a delete or re-render: lists that remove the focused row and dump the user to <body> fail in queues constantly.

Visible focus is not optional. Bare outline-none without a focus-visible:ring-* (or equivalent) replacement is a defect, not a polish choice. Agents copy outline-none from design-system resets and forget the replacement ring. Grep for it in AI diffs.

WCAG 2.2 Focus Not Obscured matters in apps with sticky headers and footers: a focused field under a sticky action bar is a focused field the user cannot see. Clinical layouts love sticky save bars. Scroll the focused control into view, or reserve space so the bar cannot cover it.

Names and labels

Real <label> elements, not placeholders doing double duty. Icon-only buttons need an accessible name (sr-only text or aria-label).

Check Label in Name (2.5.3): the visible text should be part of the accessible name so voice-control users can say what they see. An accessible name of "Submit referral request" on a button that visibly says "Send" fails this even when a screen reader user can operate it.

Tie errors with aria-describedby and aria-invalid, and keep the message element in the DOM when it is shown. Swapping the entire field for an "errored" variant often drops the description relationship. Group related controls with fieldset / legend.

Live regions and status

Mount the region empty before you update it. Agents love this pattern:

{/* Broken: often never announced */}
{saved && <div role="status">Saved</div>}

Screen readers frequently miss a live region that appears and receives text in the same paint. Prefer a persistent region:

{/* region exists first; text changes later */}
<div className="sr-only" role="status" aria-live="polite">
  {saved ? 'Draft saved' : ''}
</div>

Default to polite. Use assertive / role="alert" only for urgent interruptions. Do not remount the wrapper with a changing key. Announce autosave, async loads, validation summaries, and session-timeout warnings. The status changes clinicians glance at and assistive tech users otherwise never hear. TetraLogical's live-region write-up is still the clearest debugging guide when an announcement fails silently.

Semantics before ARIA

A <button> is not a div with an onClick. Real tables with header associations beat ARIA grids for clinical data until you have a strong reason. Headings are structure, not decoration, and agents still demote them to styled <p> tags because the mockup used a large font.

If you reach for ARIA, you are usually compensating for the wrong element. Fix the element first. Agents over-ARIA because training data is full of bad ARIA data. Your review question is simple: would a native control do this? If yes, delete the roles.

Composite widgets

Comboboxes, tabs, menus, and date pickers have keyboard contracts defined in the ARIA Authoring Practices Guide. Prefer design-system or headless primitives over a hand-rolled component the agent assembled from blog posts.

When the agent must customize, start from the APG pattern and the library primitive, then restyle. Do not start from a div and reinvent arrow-key navigation. In a design system, ban agent-authored composites unless they wrap an approved primitive ? that rule alone removes a class of regressions from AI PRs.

Zoom, reflow, motion, color

Check 200% and 400% zoom. Reflow should keep content usable without two-dimensional scrolling for vertical text blocks. Honor prefers-reduced-motion.

Never encode status with color alone. Vitals, alerts, and bed-status chips need text or an icon with a name, not just a green dot. Color-blind clinicians and grayscale VDI sessions both fail the green-dot interface.

The workflow

  1. Instructions first. Put accessibility rules in MUST / SHOULD form, scoped with path rules (applyTo / .cursor/rules), following the GitHub Accessibility team's Copilot guidance.

  2. Agent builds; you demand structure. Ask Playwright MCP for an accessibility-tree snapshot: "list every interactive element with role + name." If a control has no name, stop before you argue about styling.

  3. axe in CI. Playwright and Storybook both run it. Fail the build on violations in changed stories or routes.

  4. Ten-minute human pass. Keyboard only, then VoiceOver or NVDA on the happy path and one error path.

  5. Merge.

The human pass is short because the checklist is narrow. You are not auditing the whole product. You are verifying the behaviors models skip on the component that just changed. Ten minutes is enough when axe already ran and the accessibility tree already listed roles and names.

A Note on Healthcare UI

HHS's Section 504 says recipients of HHS funding must be WCAG 2.1 AA compliant for web and mobile. In May 2026 HHS extended the deadlines: May 11, 2027 for recipients with 15 or more employees, May 10, 2028 for smaller ones.

Don?t wait until 2027, start now. A nurse zoomed in to read small type should still be able to Tab back to where they were. A patient using a screen reader should hear that the slot they picked is unavailable, not discover it because the Continue button does nothing.

Clinical environments add constraints automation rarely models: virtual desktops, shared workstations, high zoom, and constant interruption. Build for those, and the WCAG criteria tend to follow.

Closing

Automation catches about half the issue volume. Models catch the visible half of that half when prompted well. The behavioral leftover is a short checklist for us humans.

Sources