chaff

A linter for documents. chaff checks contracts, reports, manuals and blog posts for what a machine can check: sentences too long to follow, totals that do not add up, dates out of order, references to clauses that are not there. It never rewrites the text.

For people who write or review documents in English or Japanese, and for teams that have an AI draft them and want its mistakes caught by a machine, not by another AI's opinion.

npx chaffjs article.md
npx chaffjs --genre legal/contract contract.md

Node.js 24 or later. Nothing to install first, no config file, no API key.

Get started Try it in your browser, with nothing to install Source on GitHub

Pick the kind of document (the genres)

294 rules

What it prints

The documents below were written for this page. The output under each is chaff's, run on that document when this site was built; none of it is typed by hand.

A contract that contradicts itself

A period that ends before it starts, a total that is not the sum of its items, a weekday that does not match its date, and a reference to an article that is not there. Each is decided by arithmetic or by the document's own structure, so a machine can say it every time.

# Service Agreement

## Article 1 (Term)

This Agreement runs from April 1, 2026 to March 31, 2026.

## Article 2 (Fees)

The fee consists of $5,000 for design and $6,000 for development, for a total of $12,000.

## Article 3 (Delivery)

The Contractor delivers the first draft on Monday, May 12, 2026.

## Article 4 (Termination)

Either party may end this Agreement as set out in Article 6.

$ npx chaffjs --compact --genre legal/contract contract.md

contract.md   legal/contract · English   genre from --genre

  5:26    warning The period "April 1, 2026 to March 31, 2026" ends before it starts
                  date-range-reversed
  9:83    error   The total $12,000 is not the sum of its items ($11,000)
                  total-mismatch
  13:52   error   May 12, 2026 is a Tuesday, not a Monday
                  date-weekday-mismatch
  17:51   error   "Article 6" (address 6) is not in this document
                  dangling-reference

4 findings, 112 rules not run

Text that sounds generated, and a rewrite

A draft written to sound like generated text. chaff names the stock phrases and the sections with nothing concrete in them. It does not say whether an AI wrote it.

The document

# Understanding Parallel AI Agents: A Comprehensive Guide

In today's rapidly evolving development landscape, running multiple AI agents in parallel has
emerged as a game-changing approach — one that fundamentally reshapes how teams ship software.

## The Benefits of Parallel Execution

There are three key benefits that make this approach compelling:

- **Increased velocity**: Multiple tasks can be processed simultaneously.
- **Reduced idle time**: Developers are no longer blocked waiting for a single agent.
- **Improved quality**: Different approaches can be explored in parallel.

It is important to note that these benefits are not merely incremental — they represent a
paradigm shift in the way we think about software development.

## Challenges to Consider

Of course, there are challenges. It is worth considering that resource consumption increases
significantly, and managing the state of each agent becomes more complex.

However, it should be noted that these challenges can be effectively mitigated through careful
tooling choices.

## Conclusion

In conclusion, parallel agent execution is a testament to how far AI tooling has come. As the
ecosystem continues to mature, this approach will undoubtedly become increasingly essential.
Why not give it a try?

$ npx chaffjs --compact draft.md

draft.md   blog/tech · English   genre from the default

  14:1    info    "paradigm shift" is a cliché
                  cliche
  14:1    warning "it is important to note that" emphasises without saying anything
                  empty-intensifier
  20:61   info    "more complex" does not say what it is compared with
                  comparative-without-baseline
  27:1    warning Closes with "in conclusion"
                  closing-cliche

4 findings, 70 rules not run

The same topic, rewritten by hand with numbers and decisions in it. chaff did not write this; it only checked it.

The rewrite

# Running AI agents in parallel

We run four coding agents at once, each in its own git worktree. This note is what we learned in the first month.

## What it gave us

- Four tickets move at the same time instead of one.
- Nobody waits for an agent to finish before starting the next task.
- When a fix is unclear, two agents try two approaches and we keep the better diff.

## What it cost

Each agent needs its own checkout, its own test run and its own review. On a laptop with eight cores, the fifth agent made every test run slower than the four before it.

We now cap the number of agents at four and give each one a single file to change, so two agents never edit the same function.

## Where to start

Start with two agents on two unrelated tickets, and add a third only when reviewing their diffs takes less time than writing them would.

$ npx chaffjs --compact rewrite.md

rewrite.md   blog/tech · English   genre from the default


0 findings, 70 rules not run

Your team's own words

Jargon, required sections and preferred spellings come from the team's chaff.yaml. chaff ships none of its own: what counts as jargon differs from one team to the next.

genre: business/proposal

prefer:
  e-mail: email

jargon:
  - circle back

required_sections:
  - Risks

The document

# Moving support to a shared inbox

## Proposal

Support requests now arrive by e-mail to three people. We propose one shared inbox, so a request is answered even when one of them is away.

## Cost

The shared inbox costs $40 a month. Setting it up takes two days of one engineer's time.

## Next steps

We will circle back with the support team on Friday and start in June.

$ npx chaffjs --compact proposal.md

proposal.md   business/proposal · English   genre from chaff.yaml

  1:1     error   No heading for "Risks"
                  required-sections
  5:32    warning Write "email", not "e-mail"
                  preferred-term
  13:1    info    "circle back" is a cliché
                  cliche
  13:1    warning "circle back" may only be understood inside your team (1 such word)
                  internal-jargon

4 findings, 86 rules not run

What is different about it

Same text, same result: There is no model in the lint path. With the same version and settings, the same text gives the same findings every time, so a run in CI means the same as a run on your laptop.
It checks a document against itself: Totals against their items, the start and end of a period, a weekday against its date, a number or a name written two ways, references to articles that are not there, gaps in the numbering, a term defined twice.
It says what it did not check: Every rule that did not run is listed with its reason (wrong language, not used for this genre, needs chaff test). 0 findings never has to be read as 'nothing was checked'.
Rules fit the kind of document: A contract's sentences run longer than a blog's. Pick a genre with --genre and the rules and limits fit that kind, with nothing to configure.
Japanese and English, fully: Every rule, message and screen exists in both languages. The language is read from the text.
Meaning goes to an AI only when you ask: What only reading can decide (does the conclusion say anything?) runs with npx chaffjs test. The machine narrows it to the passages that need a judge and shows what it will send before it sends it.

Why it never rewrites

A tool that tells you a text looks generated gives you nothing to act on. A tool that rewrites the text for you takes the author out of their own document, and a rewrite can drop a number or move a date without anyone noticing.

chaff reports, and the person who wrote the text decides what to change, or the AI they asked, working from a plan they can read (fix-plan). Each report ends by saying the text was not changed.

Does it send my text anywhere?

No. The check runs on your machine, or inside the page in the playground, and sends nothing.

Only npx chaffjs test calls an AI API (Anthropic or OpenAI, with your own key). It sends only the passages the machine picked out, and --dry-run shows them without calling the API. feedback writes a draft report to a file and sends nothing itself.

More than the lint

Every command returns an exit code, so each one can run in CI.

npx chaffjs compare before.md after.md: Checks that a rewrite dropped no fact and added none: numbers, dates, URLs, code, names, quotations. Read more
npx chaffjs cite source.md quotes.json: Checks that each quoted passage really is in its source. Read more
npx chaffjs ai-score article.md: A quick AI-likeness reading, low, medium or high, from signs of generated text that human documents of the genre rarely show. It is not a verdict on whether an AI wrote it. Read more
npx chaffjs grade outputs.jsonl: Grades a file of model outputs against the rules, compare and cite: a deterministic layer in an eval suite. It sends nothing. Read more
npx chaffjs fix-plan article.md: A plan for whoever rewrites the file, a person or an AI: the findings by rule, how to rewrite each, and the checks to run after. Read more
npx chaffjs . --sarif out.sarif: In CI: findings on the lines of a pull request, and baseline shelves today's findings so only new ones fail the build. Read more

How it compares

What the other tools are is quoted from their own sites. They are good at what they do, and several can be used beside chaff. The differences below are what chaff does, not claims about what the others lack.

Vale

What it is: An open-source (MIT), markup-aware linter for prose. Teams write their style guide as YAML rules.

Where chaff differs: Ships its rules, chosen by the kind of document, and checks a document's own numbers, dates and references. Reads Japanese as well as English.

textlint

What it is: An open-source, pluggable linter for natural language, written in JavaScript. Rules are installed separately from npm.

Where chaff differs: Comes with its rules built in and set per genre. A team's own rules go in chaff.yaml or a YAML rule pack.

LanguageTool

What it is: Open-source proofreading (LGPL) for more than 25 languages, finding errors a spell checker cannot. It can run as your own server.

Where chaff differs: Not a grammar or spelling checker, and reads only English and Japanese. It checks structure, consistency and arithmetic.

Grammarly

What it is: An AI writing assistant that suggests edits and paragraph rewrites, in the browser, desktop and office apps.

Where chaff differs: Never rewrites, runs on your machine or in CI, and gives the same result for the same text.

Asking an LLM

What it is: A prompt such as 'review this contract'. It reads meaning and can notice what no rule covers.

Where chaff differs: The same input does not always give the same answer, and a missed total is hard to notice. chaff's checks repeat exactly and cost nothing to run on every change. The two work together: chaff test hands an AI only the passages that need reading.

What it does not do

It does not judge meaning: Whether an argument holds, a fact is true or a contract is fair is not for a linter. chaff stops at narrowing the question (chaff test).
It misses things: It finds only what a rule covers, and only the phrasings that rule knows. 0 findings means none of the rules that ran found anything.
It is wrong sometimes: Silence one spot with , relax a rule for the team with npx chaffjs relax rule-id --why "reason", or turn it off in chaff.yaml.

How it is measured

Planted sets: self-written documents of one genre (contracts, email, reports, technical writing and more), each in a clean version and a version with mistakes planted. Every change counts which planted mistakes are reported on their line and which findings the clean versions get, and CI fails when either gets worse.

A corpus of real documents, public-domain statutes among them, is re-run on every change, so a new rule that starts flagging ordinary writing shows up before it ships.

When a finding is wrong

npx chaffjs feedback <file> --rule <rule> drafts a report with only the lines around the finding, and prints the command to open a GitHub issue with it. It sends nothing itself. Each report becomes a test and a fix.

Source and license

github.com/isamu/lab (text/)
MIT license
npm: chaffjs
Made by isamu, in the isamu/lab repository.