DSHS - Copyeditor AI Agent

Built AI-assisted QA tooling that helped scale plain-language and accessibility review across 3,500+ pages, contributing to the Digital Certainty Index rising from 60.7 to 83.5 and task completion improving 40%.

A custom AI copyeditor agent, plus a manual readability check, that held 3,500+ public health pages to one standard.

Context

During the Texas Department of State Health Services' $10M CMS overhaul, every one of 3,500+ public health pages had to meet three standards at once: correct grammar, the DSHS style guide, and WCAG 2.0 AA compliance.

Patients and clinicians both read these pages, and the target was a 6th to 8th grade reading level across the board. Manual review couldn't hold that bar at that volume on a weekly sprint schedule. So I designed and built an AI copyeditor agent to handle the first line of QA, and kept a human readability check behind it.

The challenge:

  • Volume on a deadline. More than 3,500 pages moved through the migration, with review work due every weekly sprint.

  • Three standards, one page. A single blended review tends to catch the most obvious problem on a page and miss the rest. Grammar, style, and plain language each needed their own attention.

  • A target without a measure. "6th to 8th grade" was the goal, but without a consistent score on every page, meeting it came down to a judgment call page by page.

Results

  • Every page reviewed against all three standards. The agent ran grammar, style guide, and plain language QA across all 3,500+ pages, with a readability score attached to each one.

  • Readability measured, not guessed. The Flesch-Kincaid KPI and the Hemingway double-check contributed to the WCAG 2.0 AA compliance results: reading levels dropped from postgraduate to 6th to 8th grade, and task completion rose 40%.

  • Quality held at scale. The QA layer contributed to raising the Digital Certainty Index from 60.7 to 83.5 in 18 months.

My approach

  1. One agent, three dedicated QA reviews. I built the copyeditor agent to run three separate reviews on every page, each with its own instructions and one job:

    • Grammar review: mechanics, spelling, punctuation, and sentence-level errors.

    • Style guide review: consistency with the DSHS style guide on terminology, formatting, and voice.

    • Plain language review: alignment with our plain language guide, flagging anything that fell short of its rules.

  2. Readability built in as a default KPI. The agent's instructions included a Flesch-Kincaid readability calculator as a built-in measurement on every page. Each page came out of review with a grade-level score. That turned the 6th to 8th grade target from a goal into a number we could check page by page and track across the whole migration.

  3. A fourth review, done by hand. After the agent finished, I ran each page through Hemingway Editor manually to double-check the readability scores. This gave every page a second, independent read on readability before it was cleared.

    • Why the Hemingway Editor step stayed manual?

      Hemingway Editor had no integration or API the agent could connect to at the time.