Reviewing AI-Generated Technical Writing Without Breaking It

We’ve been trialling Claude and other models to draft internal technical content. One example is a guide for running a specific lab setup on a local machine. A draft can contain correct technical detail and still be difficult to read. Common problems include long sentences, repeated explanations and too much signposting.

A second AI model can improve that draft. It can also change a shell command, remove a link, alter a number or weaken an important qualification. The review step can create a technical defect while making the prose sound better.

We wanted a review process that could improve AI-generated writing without giving the editor control of the technical content. This became the prose-quality work in Agentflow 0.0.4. The same process can be added to another content workflow with a script and a CI job.

Automatic review created four problems

The first design sent each Claude-authored Markdown file to a second model for editing. This simple pipeline had four faults:

It reviewed files that were already clear.
It also reviewed internal notes and handoffs that readers would never see.
It added another model call to every writing task.
It let the editor change technical material that was outside its assignment.

Source code, research notes and public documentation need different review rules. A command inside a blog post also needs more protection than the sentence above it.

The revised process asks three questions before it accepts an edit:

Is this file eligible for AI prose review?
Does the prose contain a problem that justifies another model call?
Did the edit preserve the technical content and pass the project checks?

The complete flow is:

Claude writes reader-facing Markdown
  -> run deterministic prose checks
  -> pass: keep the original file
  -> fail: allow one editor pass into a sibling file
  -> compare protected technical material
  -> run the project validation commands
  -> accept the edit, reject it or request human review

Select files before you review them

Agentflow applies this process only when all three conditions are true:

Claude wrote the file.
The file is reader-facing Markdown.
The project selected the technical-blog or instruqt profile.

Reader-facing files include blog posts, public documentation and Instruqt assignments. Internal handoffs, research notes, reviews, chat output and source code stay outside the process.

This classification prevents accidental scope changes. A long comment in a source file does not give a prose editor permission to rewrite the file. An internal handoff does not need a publication edit.

If you build this control in another system, record the classification with the writing task. Do not infer it from the filename after the draft is complete. A field such as artifact_kind: reader-facing-markdown is easier to audit than a list of directory guesses.

Use simple checks to trigger the editor

Agentflow runs deterministic checks before it prepares an editing task:

agentflow prose check draft.md --profile technical-blog

A failing result identifies the rule and the source line:

NEEDS_EDIT technical-blog: draft.md
- sentence-too-long: line 18: sentence has 39 words; profile limit is 32

The built-in profiles check for:

sentences and paragraphs above fixed size limits;
repeated sentences; and
a small set of common filler phrases.

The checker ignores YAML frontmatter and fenced code. It returns exit status 0 for a pass, 1 when editing is required and 2 for invalid input.

These rules do not grade the article. They decide whether the workflow can spend one editing call. A passing result ends this part of the workflow and keeps the source unchanged.

Here is a typical edit:

Before: The controller is able to continue the workflow when the previous context becomes too long because the information it needs is stored separately from the conversation itself.

After: The controller can continue when a context becomes too long because its durable state is stored separately from the conversation.

The shorter version keeps the meaning. The edit is useful when a check finds the problem. There is no reason to request this rewrite when the source already reads well.

Keep the original file

An editor must never overwrite the only copy of the draft. Agentflow writes the result to a sibling file:

posts/agent-review.md
posts/agent-review.edited.md

The source remains available for a diff or a complete rollback. The controller can reject the sibling without reconstructing the original from Git history or a conversation.

Use prose prepare to create the bounded editing task:

agentflow prose prepare posts/agent-review.md \
  --profile technical-blog \
  --writer-provider claude \
  --require-skill <project-skill> \
  --check "python3 scripts/validate_post.py posts/agent-review.edited.md"

If the source passes, the command prints NO_EDIT and creates no handoff. If it fails, the command prints the preflight and launch commands for one approved editor session.

Protect technical content with exact comparisons

The preservation check extracts technical material from the source and the edited file. It compares:

fenced code blocks;
inline code;
link destinations;
file paths;
numbers; and
Markdown structure.

The edit fails when a protected value changes or disappears. The edit also fails when the prose grows instead of becoming more concise.

Run the comparison with:

agentflow prose verify \
  posts/agent-review.md \
  posts/agent-review.edited.md \
  --profile technical-blog

A successful check returns:

PASS technical-blog: posts/agent-review.edited.md

Exact comparisons catch mechanical damage. They cannot find every change in meaning. A sentence can keep the same number and still describe it incorrectly. The project validator must run after the preservation check. A person or the controller must also inspect the diff.

For a Kubernetes tutorial, the domain check might validate manifests and test shell examples. A website repository might validate frontmatter, links, routes and the generated page. The prose checker cannot replace either test.

Stop after one editing pass

The editor gets one attempt. A second automatic attempt would make it harder to identify which change fixed or damaged the draft. It would also start another retry loop around a subjective task.

The one-pass limit produces a clear result:

source passes             -> keep source
source fails, edit passes -> review sibling
source fails, edit fails  -> keep source or request human edit

The workflow does not ask the same editor to keep rewriting the file. A failed sibling remains evidence that the route did not work for that draft.

Implement the process without Agentflow

The control logic is small. This Python example shows the required decisions:

def review_prose(source, writer_model, profile, editor, domain_check):
    if not is_claude_model(writer_model) or not is_reader_facing_markdown(source):
        return {"status": "out-of-scope", "source": source}

    findings = check_prose(source, profile)
    if not findings:
        return {"status": "keep-source", "source": source}

    if editor is None:
        return {"status": "edit-required-no-route", "findings": findings}

    sibling = edit_once(source, findings, editor)

    if protected_material(source) != protected_material(sibling):
        return {"status": "reject-edit", "reason": "protected-material-changed"}

    if word_count(sibling) > word_count(source):
        return {"status": "reject-edit", "reason": "prose-expanded"}

    if not domain_check(sibling):
        return {"status": "reject-edit", "reason": "domain-check-failed"}

    return {"status": "candidate", "source": source, "edited": sibling}

You can implement check_prose with a Markdown parser and fixed thresholds. Implement protected_material as an ordered extraction of code, links, paths, numbers and headings. Compare the two ordered results exactly.

The candidate result still needs approval. The script proves that selected material survived the edit and that the project checks passed. It does not prove that the new prose has the same meaning.

Make the editor route explicit

Agentflow validates the editor provider, model, effort and optional credit cap as one route. The bundled default uses Codex Luna at medium effort. Projects can select Claude Code or GitHub Copilot when those routes are present in the model policy.

The following machine-local configuration uses GitHub Copilot:

{
  "schema": "agentflow.project-local@1",
  "version": 1,
  "prose": {
    "editor": {
      "provider": "copilot",
      "model": "claude-sonnet-4.6",
      "effort": "medium",
      "max_ai_credits": 30
    }
  },
  "skills": []
}

A project can keep the deterministic checks and disable AI editing:

{
  "prose": {
    "editor": null
  }
}

A failing draft then returns edit-required-no-route. Agentflow does not select another provider automatically.

The design was influenced by the writing problem explored in gvzdv/claudish-to-english. Agentflow does not install or import that project. Its prose workflow uses the existing Agentflow model policy, project skills and validation commands.

Agentflow 0.0.4 release

Agentflow 0.0.4 is a public preview for development and testing. The release passed 492 local tests with one documented skip. Repository validation, package inspection, clean-wheel installation, hosted CI, security checks and the release gate also passed.

Install the tagged preview in an isolated tool environment:

uv tool install "git+https://github.com/saintdle/agentflow.git@v0.0.4"
# or
pipx install "git+https://github.com/saintdle/agentflow.git@v0.0.4"

Then check the command and local dependencies:

agentflow --version
agentflow doctor

Our rule is now simple. The editor gets one attempt and writes to a sibling file. Commands, links, numbers, paths and Markdown structure must remain unchanged. The project checks must pass. If any condition fails, we keep the original.