Bibliography
This page lists the papers and standards behind chaff’s rules. Every entry was checked against its original record (the journal, the publisher or the standards body) before it was listed. Each entry says what the work found and how chaff uses it.
How to read an entry
Entries are grouped by topic. Each one has three parts.
| Part | What it holds |
|---|---|
| Reference | Authors, year, title, where it appeared, and a link to the original record |
| Found | What the work showed, in one or two sentences |
| In chaff | The rules it supports. A work chaff does not use directly is marked [background], with the reason |
Neither a readability formula nor a detector of generated text decides whether a document is good. Both only show statistical tendencies. That is why chaff never grades a document. It points at each place to fix, one at a time. The rules themselves are listed in the Reference.
Japanese readability
- 柴崎秀子, 玉岡賀津雄 (Shibasaki and Tamaoka, 2010) doi.org
- 「国語科教科書を基にした小・中学校の文章難易学年判定式の構築」 (a formula for the school grade of a text, built from Japanese-language textbooks)
- Appeared in: 日本教育工学会論文誌 (Japan Journal of Educational Technology) 33(4), 449–458.
- Found: From 205 textbook passages, the authors built a formula that predicts which school grade a text suits. Only two measures mattered: the share of hiragana, and the number of predicates in a sentence.
- In chaff: the grounds for
max-kanji-continuousandmax-sentence-length.
- 李在鎬 (Lee Jae-ho, 2016) jhlee.sakura.ne.jp (PDF)
- 「日本語教育のための文章難易度に関する研究」 (text difficulty for teaching Japanese)
- Appeared in: 早稲田日本語教育学 (Waseda Studies in Japanese Language Education) 21, 1–16.
- Found: A formula that places a text on six levels for learners of Japanese. It uses average sentence length and the shares of Sino-Japanese words, native words, verbs and particles. The jReadability website is built on it.
- In chaff: more Sino-Japanese words made a text harder, which backs
max-kanji-continuous.
- Sato, Matsuyoshi, Kondoh (2008) aclanthology.org
- Automatic Assessment of Japanese Text Readability Based on a Textbook Corpus
- Appeared in: LREC 2008.
- Found: Using textbooks from first grade to university, the authors predicted a text’s grade from its characters alone. This is the basis of the readability tool Obi (帯). It needs no word segmentation.
- In chaff: [background] chaff gives no grade. It points at the hard places instead.
- 佐藤理史 (Sato, 2011) cir.nii.ac.jp
- 「均衡コーパスを規範とするテキスト難易度測定」 (text difficulty measured against a balanced corpus)
- Appeared in: 情報処理学会論文誌 (IPSJ Journal) 52(4), 1777–1789.
- Found: Difficulty is shown as where a text sits among real Japanese texts of all kinds, not against school grades.
- In chaff: [background] the same idea as chaff’s genres, which set different limits for different kinds of document.
- Immigration Services Agency of Japan and Agency for Cultural Affairs (2020) bunka.go.jp (PDF)
- 『在留支援のためのやさしい日本語ガイドライン』 (guidelines for plain Japanese for foreign residents)
- Found: How to write Japanese that foreign residents understand. It asks for short sentences, no double negatives, few passives, consistent です・ます endings and not too many kanji. It sets no number of characters per sentence.
- In chaff:
max-sentence-length,no-mixed-desumasuandagentless-passivecheck three of these points.nominalizationchecks its example of 調査を実施した rewritten as 調査した.
- 庵功雄 (Iori, 2016) iwanami.co.jp
- 『やさしい日本語―多文化共生社会へ』 (plain Japanese, toward a multicultural society)
- Appeared in: Iwanami Shinsho (a book).
- Found: How plain Japanese began after the 1995 Kobe earthquake, and whom it is for.
- In chaff: [background] it explains why to write short and plain, rather than giving a limit.
English readability and plain English
- Flesch (1948) doi.org
- A new readability yardstick
- Appeared in: Journal of Applied Psychology 32(3), 221–233.
- Found: A score from 0 to 100, from sentence length and syllables per word. About 70 is “easy” and about 30 is “very difficult”.
- In chaff: [background] sentence length is in every formula. chaff checks it directly with
max-sentence-length.
- Kincaid et al. (1975) eric.ed.gov
- Derivation of New Readability Formulas (Automated Readability Index, Fog Count and Flesch Reading Ease Formula) for Navy Enlisted Personnel
- Appeared in: US Navy Research Branch Report 8-75.
- Found: Using 531 Navy enlisted personnel, the formulas were recalculated to give a school grade. This is the Flesch–Kincaid grade level in wide use today.
- In chaff: [background] a grade means little until the readers are known, so chaff does not give one.
- Dale, Chall (1948) files.eric.ed.gov (PDF)
- A Formula for Predicting Readability
- Appeared in: Educational Research Bulletin 27(1), 11–20, 28. The link is the ERIC PDF that reprints it.
- Found: Readability from the share of words outside a list of 3,000 familiar words, and sentence length.
- In chaff: [background] chaff ships no word lists of its own. A team lists the words its readers will not know in
chaff.yaml.
- Redish (2000) doi.org
- Readability formulas have even more limitations than Klare discusses
- Appeared in: ACM Journal of Computer Documentation 24(3), 132–137.
- Found: A formula tells you neither why a text is hard nor how to fix it. The formulas were built on children’s schoolbooks, and nobody knows whether they hold for adult technical writing.
- In chaff: the reason chaff gives no score, and names the place and the reason instead.
- Crossley, Skalicky, Dascalu (2019) doi.org
- Moving beyond classic readability formulas: new methods and new models
- Appeared in: Journal of Research in Reading 42(3–4), 541–561.
- Found: Using readers’ judgments of comprehension, models built from many language features beat classic formulas such as Flesch–Kincaid.
- In chaff: [background] a trained model does not belong in chaff’s checks, which must give the same result for the same text.
- PLAIN (2011) wid.org (PDF)
- Federal Plain Language Guidelines
- Appeared in: guidance for US federal agencies. The PDF is gone from the government site, so the link is a copy at wid.org.
- Found: Use the active voice, call the same thing by the same word, use fewer abbreviations, and avoid double negatives. Use “must”, not “shall”, for requirements, and keep sentences and paragraphs short. It gives no word limit.
- In chaff:
agentless-passive,preferred-termandundefined-acronymcheck the voice, the terms and the abbreviations. Withshall: mustunderrequirement-modalinoptions, chaff points at “shall” and asks for “must”.
- Government Digital Service (UK) guidance.publishing.service.gov.uk
- Use clear language
- Appeared in: GOV.UK content and publishing guidance.
- Found: Split sentences over 25 words. Keep paragraphs to 5 sentences or fewer.
- In chaff: the
normallevel ofmax-sentence-lengthfor English (25 words) and ofmax-paragraph-length(5 sentences) match these numbers.nominalizationchecks its advice to use the verb (“decide”, not “make a decision”).
- ISO 24495-1:2023 cdn.standards.iteh.ai (PDF)
- Plain language — Part 1: Governing principles and guidelines
- Appeared in: ISO. The link is the sample PDF published by the reseller iTeh.
- Found: The international standard defines plain language by whether readers can find, understand and use what they need. It measures that by what readers can do, not by mechanical measures such as readability formulas.
- In chaff: like Redish, a reason chaff does not grade documents.
Rules for writing and notation
- 文化審議会 (Council for Cultural Affairs, 2022) bunka.go.jp (PDF)
- 「公用文作成の考え方」 (how to write public documents)
- Appeared in: a recommendation to the government, 7 January 2022.
- Found: Once a sentence reaches about 50 to 60 characters, check whether it has become hard to read. Documents for the public use です・ます, loanwords keep the final ー (コンピューター), and full-width and half-width characters are used consistently.
- In chaff:
max-sentence-length,no-mixed-desumasu,katakana-long-vowelandlatin-spacingcheck these points.style: koyobuninchaff.yamlsets a Japanese sentence’s limit to 60 characters, and checks mixed です・ます and a dropped final ー.
- Cabinet notice (1991) bunka.go.jp
- 「外来語の表記」 (how to write loanwords)
- Appeared in: Cabinet notice No. 2 of 1991.
- Found: English endings such as -er, -or and -ar are written with a long-vowel mark ー as a rule. It also allows the mark to be dropped where that is customary.
- In chaff:
style: bunkachoinchaff.yamlrunskatakana-long-vowelby this rule.
- JIS Z 8301:2019 webdesk.jsa.or.jp
- 「規格票の様式及び作成方法」 (rules for drafting Japanese Industrial Standards)
- Appeared in: Japanese Standards Association.
- Found: It fixes the sentence endings for a requirement, a prohibition, a recommendation and a permission, and does not use 「すべきである」. It avoids vague pointers such as 「上記の図」 (“the figure above”) and keeps an abbreviation the same throughout. The 2011 edition dropped the final ー of loanwords of three morae or more.
- In chaff:
undefined-acronymanddangling-referencecheck the abbreviations and the pointers.vague-figure-referencepoints at 「上記の図」 in a document that numbers its figures and tables, and at 「以下の箇条」 in one that numbers its clauses (10.6).style: jis-z8301-2011runskatakana-long-vowelby the 2011 edition’s rule.style: jis-z8301-2019makesrequirement-modalpoint at a closing すべきである or できる (7.3 to 7.5), and turns onvague-figure-reference(10.6).
- Kyodo News, ed. (2022) kyodo.co.jp
- 『記者ハンドブック 新聞用字用語集 第 14 版』 (the Kyodo reporters’ handbook, 14th edition)
- Found: A book of the spellings and word choices used in Japanese newspapers. Press officers and web writers use it as well as reporters.
- In chaff: chaff ships no spellings of its own. Words a team takes from this book go under
preferinchaff.yaml, andpreferred-termchecks them.
- Japan Translation Federation (JTF) jtf.jp
- 『JTF 日本語標準スタイルガイド(翻訳用)』 (the JTF style guide for Japanese translation)
- Found: Twelve basic rules, all of which a machine can check. Among them: do not mix です・ます with である, keep the final ー on katakana words, and put no space between full-width and half-width characters.
- In chaff:
no-mixed-desumasu,katakana-long-vowelandlatin-spacingcheck close to these rules. chaff does not decide whether to put the space. It checks only that one document is consistent.
Other prose linters
chaff was compared with the public lists of checks of these linters, to find kinds of mistakes it did not yet look for. No rule text or word list was copied; chaff’s lists are its own.
- textlint-ja github.com
- textlint-rule-preset-ja-technical-writing and textlint-rule-preset-japanese (rule sets for Japanese technical writing)
- Found: Checks a machine can run on Japanese text: sentence length, commas, mixed endings, ら dropped, doubled particles, invisible characters.
- In chaff: most have a chaff rule (
max-sentence-length,max-ten,no-mixed-desumasu,ra-nuki,no-doubled-joshi,invisible-character).
- proselint github.com
- A linter for English prose, with checks drawn from usage guides.
- Found: Usage mistakes a word list can catch: clichés, redundancy, uncomparable adjectives given a degree, weasel words.
- In chaff:
uncomparable-gradedfollows its check of uncomparable adjectives.
- write-good github.com
- A linter for English prose.
- Found: Passive voice, “there is” openers, a sentence-initial “So”, weasel words, wordy phrases, clichés and repeated words.
- In chaff:
agentless-passive,expletive-construction,sentence-initial-so,weasel-word,wordy-phraseanddoubled-wordcover these, andclicheits check of clichés.
- Vale Microsoft and Google styles github.com
- Rules for the Vale linter that encode the Microsoft Writing Style Guide and the Google developer documentation style guide (Google).
- Found: Checks of notation: ellipses, units, spacing, dashes, dates, ranges, contractions, the Oxford comma, heading case.
- In chaff: chaff checks that a document agrees with itself, not with one house’s choice (
contraction-consistency,oxford-comma-consistency,range-notation-consistency).ellipsis-consistencyfollows their check of ellipses the same way.
- alex github.com
- A linter for insensitive or inconsiderate wording.
- In chaff: [background] which words to avoid is the team’s call. chaff ships no such list; a team puts its own under
preferinchaff.yaml.
Scoring writing and finding errors
- Attali, Burstein (2006) ejournals.bc.edu
- Automated Essay Scoring With e-rater V.2
- Appeared in: Journal of Technology, Learning, and Assessment 4(3).
- Found: A system that scores test essays. It showed that a small set of meaningful features is enough to score them.
- In chaff: [background] chaff does not score documents.
- Ke, Ng (2019) doi.org
- Automated Essay Scoring: A Survey of the State of the Art
- Appeared in: IJCAI-19.
- Found: A review of 50 years of essay scoring research. It concludes the task is far from solved.
- In chaff: [background] one more reason chaff points at places rather than giving a score.
- Ng et al. (2014) aclanthology.org
- The CoNLL-2014 Shared Task on Grammatical Error Correction
- Appeared in: CoNLL 2014.
- Found: A shared task to correct grammar in essays by learners of English. It widened the previous year’s five error types to all types.
- In chaff: [background] chaff does not correct grammar, and never rewrites text.
- Bryant et al. (2023) aclanthology.org
- Grammatical Error Correction: A Survey of the State of the Art
- Appeared in: Computational Linguistics 49(3), 643–701.
- Found: How grammar correction moved from hand-written rules, to statistics, to machine translation.
- In chaff: [background] a guide to which errors rules can catch, the part chaff takes on.
- Mizumoto et al. (2011) aclanthology.org
- Mining Revision Log of Language Learning SNS for Automated Japanese Error Correction of Second Language Learners
- Appeared in: IJCNLP 2011.
- Found: A large corpus of errors by learners of Japanese, taken from corrections on the language-learning site Lang-8.
- In chaff: [background] research on learners’ errors, a different readership from chaff’s.
- Koyama et al. (2020) aclanthology.org
- Construction of an Evaluation Corpus for Grammatical Error Correction for Learners of Japanese as a Second Language
- Appeared in: LREC 2020.
- Found: Lang-8 is too noisy to evaluate with, so the authors built a cleaner Japanese evaluation corpus, TEC-JL.
- In chaff: [background] an example of why judging corrections needs answers a person has checked.
- 山本和英, 鄭育昌 (Yamamoto and Cheng, 2015) anlp.jp (PDF)
- 「Project Next 日本語校正タスク」 (the Project Next Japanese proofreading task)
- Appeared in: a workshop at the 21st annual meeting of the Association for Natural Language Processing (Japan).
- Found: A survey of the errors Japanese proofreading tools look for. A missing particle, the same particle twice in a row, or a mismatched adverb are easy for a machine. Errors that depend on context are hard.
- In chaff:
no-doubled-joshifinds the same particle repeated.
What generated text looks like
The works in this section study what tends to show up in generated text. chaff uses them to point at phrasing and formatting, but it never says “an AI wrote this”. Several of them also show how often human writing is mistaken for generated text.
- Gehrmann, Strobelt, Rush (2019) aclanthology.org
- GLTR: Statistical Detection and Visualization of Generated Text
- Appeared in: ACL 2019 System Demonstrations.
- Found: Coloring each word by how predictable it is raised people’s accuracy at spotting generated text from 54% to 72%.
- In chaff: [background] it needs a language model’s probabilities, which chaff’s checks do not use.
- Liang et al. (2023) arxiv.org
- GPT detectors are biased against non-native English writers
- Appeared in: Patterns.
- Found: Popular AI detectors often label writing by non-native speakers of English as generated.
- In chaff: the reason chaff judges phrasing, never the writer.
- Wu et al. (2025) aclanthology.org
- A Survey on LLM-Generated Text Detection: Necessity, Methods, and Future Directions
- Appeared in: Computational Linguistics 51(1), 275–338.
- Found: Detection methods fall into four kinds: watermarks, statistics, trained classifiers and human judgment.
- In chaff: [background] in these terms, chaff uses only surface counts.
- Reinhart et al. (2025) arxiv.org
- Do LLMs write like humans? Variation in grammatical and rhetorical styles
- Appeared in: PNAS.
- Found: Instruction-tuned models use participle clauses and nominalizations several times as often as people, and list nouns more.
- In chaff: supports
rule-of-three, andnominalization, which counts hidden verbs.
- Kobak et al. (2025) arxiv.org
- Delving into LLM-assisted writing in biomedical publications through excess vocabulary
- Appeared in: Science Advances 11(27).
- Found: Across more than 15 million biomedical abstracts, words such as “delves”, “underscores”, “showcasing” and “pivotal” jumped in 2024.
- In chaff: backs the English words
ai-telllooks for.
- Liang et al. (2024) proceedings.mlr.press
- Monitoring AI-Modified Content at Scale: A Case Study on the Impact of ChatGPT on AI Conference Peer Reviews
- Appeared in: ICML 2024.
- Found: In conference peer reviews, adjectives such as “commendable”, “meticulous” and “intricate” rose sharply.
- In chaff: supports
ai-tell, and is why it speaks only when such words pile up, never at one word.
- Zhang et al. (2024) arxiv.org
- From Lists to Emojis: How Format Bias Affects Model Alignment
- Appeared in: arXiv (not peer reviewed).
- Found: Human raters and AI judges alike favour lists, bold text, emojis and long answers, whatever the content. That is one reason models use these formats so much.
- In chaff:
bold-density,emoji-densityandrule-of-threepoint at the places where this bias shows in a text.
- Freeburg (2026) arxiv.org
- The Last Fingerprint: How Markdown Training Shapes LLM Prose
- Appeared in: arXiv (not peer reviewed).
- Found: Across 12 models, how often a model uses the em dash (—) varies widely.
- In chaff: supports
no-em-dash, and is why it counts frequency instead of flagging one dash.
- Zaitsu, Jin (2023) doi.org
- Distinguishing ChatGPT(-3.5, -4)-generated and human-written papers through Japanese stylometric analysis
- Appeared in: PLOS ONE 18(8).
- Found: In Japanese papers, counting function words and where commas fall separated human and ChatGPT text well.
- In chaff: [background] the separation uses a trained classifier, which chaff does not include.
- Zaitsu et al. (2025) doi.org
- Stylometry can reveal artificial intelligence authorship, but humans struggle: A comparison of human and seven large language models in Japanese
- Appeared in: PLOS ONE 20(10).
- Found: Japanese text by seven models could be told apart by counting. Human judges did poorly, relying on surface cues such as sentence endings, conjunctions and punctuation.
- In chaff: [background] counting beat human hunches, which is the position chaff takes.
- 林美佐, 相澤彰子 (Hayashi and Aizawa, 2026) anlp.jp (PDF)
- 「LLM による日本語生成におけるモデル固有表現パターンの分析」 (model-specific phrasing in Japanese generated by LLMs)
- Appeared in: the 32nd annual meeting of the Association for Natural Language Processing (Japan), B9-17.
- Found: Each model has set openings and closings, such as 「結論から申し上げますと」 (“to give the conclusion first”) and 「ステップバイステップで説明します」 (“I will explain step by step”).
- In chaff: the same kind of phrasing as the chat leftovers
assistant-residuelooks for.
- 岩本海風, 宮本友樹, 内海彰 (Iwamoto, Miyamoto and Utsumi, 2026) anlp.jp (PDF)
- 「大規模言語モデルで生成された文学テキストの検出と有効な識別的特徴の探求」 (detecting literary text generated by LLMs, and the features that tell it apart)
- Appeared in: the 32nd annual meeting of the Association for Natural Language Processing (Japan), P9-11.
- Found: In poems, lyrics, haiku and short stories alike, generated Japanese text varied less in sentence length.
- In chaff: supports
sentence-rhythm.
- Juzek, Ward (2025) aclanthology.org
- Why Does ChatGPT “Delve” So Much? Exploring the Sources of Lexical Overrepresentation in Large Language Models
- Appeared in: COLING 2025, pp. 6397–6411.
- Found: 21 words that suddenly rose in scientific abstracts (delve, intricate, underscore and others) are the ones ChatGPT overuses. Training data and model design did not explain it; the authors point to the stage that tunes models on human feedback as a likely part.
- In chaff: why
ai-tellpicks its words by comparing writing from before and after LLMs.
- Sun et al. (2025) arxiv.org
- Idiosyncrasies in Large Language Models
- Appeared in: ICML 2025.
- Found: which model wrote a text (ChatGPT, Claude, Grok, Gemini, DeepSeek) can be told with high accuracy from word choice alone. The habits survive a rewrite, a translation or a summary by another model.
- In chaff: [background] chaff uses no classifier. That the habits sit in word choice is why a word list is a fair instrument.
- Shaib et al. (2025) arxiv.org
- Measuring AI “Slop” in Text
- Appeared in: arXiv (preprint).
- Found: from interviews with experts, the paper splits low-quality generated text into density, relevance, factuality, repetition, templatedness, tone, verbosity and coherence. Judging a text as “slop” varies between people, but the judgement tracks these dimensions.
- In chaff: [background] templatedness and repetition are the parts chaff can count.
- Wikipedia (WikiProject AI Cleanup) en.wikipedia.org
- Wikipedia:Signs of AI writing
- Appeared in: an editors’ guide on the English Wikipedia; the version read in October 2026.
- Found: a field list, with real examples, of the shapes generated drafts leave. Among them: list items led by a bold label, emoji in headings, and links ending in “?utm_source=chatgpt.com” or citation marks such as oaicite.
- In chaff: the basis of
bold-label-list,emoji-headingandchat-citation-residue.
- fibujrsl (2026) zenn.dev
- 生成AIっぽい文章の特徴をまとめる (The features of AI-sounding writing)
- Appeared in: Zenn, 14 February 2026.
- Found: three signs of generated Japanese. They are left-over emphasis marks, lists dressed up with symbols and emoji, and words stressed again and again with brackets or symbols. The fix it gives is to drop the decoration and let the position of the conclusion and its evidence carry the stress.
- In chaff: the fix behind
bold-label-listandemoji-heading.
- Writers-hub (2026) writers-hub.co.jp
- Qiitaで生成AI記事を書く|評価される記事とAI臭さの消し方 (Writing about generative AI on Qiita)
- Appeared in: the Writers-hub LLC blog, 26 July 2026.
- Found: in generated Qiita articles, lists take up most of the text, colons are overused, and paragraphs end in abstract noun phrases. It advises turning the parts that retreated into lists back into prose.
- In chaff: the basis of
bold-label-listandcolon-lead-in.
Requirements and contradictions
- ISO/IEC/IEEE 29148:2018 standards.ieee.org
- Systems and software engineering — Life cycle processes — Requirements engineering
- Found: The international standard for what makes a good requirement. It also names words to avoid, such as loopholes and vague adverbs.
- In chaff:
requirement-smelllooks for the loopholes it names (“if possible”, “as appropriate”) and its open-ended “but not limited to” in a requirement.
- Berry, Kamsties, Krieger (2003) cs.uwaterloo.ca (PDF)
- From Contract Drafting to Software Specification: Linguistic Sources of Ambiguity
- Found: A handbook of words that let contracts and specifications be read two ways. It covers “and/or”, “all” against “each”, where “only” goes, and pronouns that point at nothing clear.
- In chaff:
requirement-smelllooks for “and/or” (及び/又は) in a requirement. Most of the others are not chaff rules yet.
- Femmer et al. (2017) doi.org
- Rapid quality assurance with Requirements Smells
- Appeared in: Journal of Systems and Software 123, 190–213.
- Found: Machine checks for warning signs in requirements. The signs are subjective words, vague adverbs, loopholes, superlatives, comparatives, negatives, vague pronouns and incomplete references. On average 59% of the findings were right, and 82% of the real problems were found.
- In chaff: supports
unqualified-superlative,excessive-hedginganddangling-reference.requirement-smelllooks for loopholes and open-ended lists (“etc.”, 等) in a requirement. It stays experimental, since only about 6 in 10 of the study’s findings were right. It also matches chaff’s view that a person confirms what a machine finds.
- Gervasi, Zowghi (2005) doi.org
- Reasoning about inconsistencies in natural language requirements
- Appeared in: ACM Transactions on Software Engineering and Methodology 14(3), 277–330.
- Found: Requirements written in plain language were turned into logic, and requirements that contradict each other were found automatically.
- In chaff: [background] turning sentences into logic is beyond what chaff does.
- de Marneffe, Rafferty, Manning (2008) aclanthology.org
- Finding Contradictions in Text
- Appeared in: ACL-08.
- Found: Some contradictions are easy to see, such as negation, opposite words and numbers that disagree. Others need knowledge of the world.
- In chaff: the basis for checking numbers and dates by machine, as
total-mismatchanddate-orderdo.
- Mavin et al. (2009) doi.org
- Easy Approach to Requirements Syntax (EARS)
- Appeared in: RE’09.
- Found: Writing every requirement in one of five fixed patterns cut common problems such as ambiguity.
- In chaff: [background] no rule checks these patterns yet.