The Fifth Error

A register of every defect found in this project, who found it, and what the pattern of discovery says about how many remain.

Opened 16 August 2026 Defects logged 21 Still open 0 Estimated undiscovered see section 3

The question this document exists to answer: how do we know we are publishing without a fifth error nobody has caught?

The answer is that we do not, and cannot. No audit proves the absence of defects. Every check tests for a mistake somebody already thought of; the dangerous error is by definition the one nobody imagined. Anyone who tells you their work is error-free is telling you how hard they looked, not what is there.

What can be done is to measure whether the search is near its end — and to publish in an order that limits what any single undiscovered error could destroy. That is what this register is for.

1. The register

Every defect found, in the order found. “Machine” means an automated check caught it; “Human” means a person reading the output caught it.

#	Defect	Consequence if unfound	Found by
1	“Woman” counted as 3 — only imraʾa matched, nisāʾ missed	A published count wrong by a factor of 12	human
2	Ayyām miscounted — four forms missed, two unrelated words admitted	Wrong number, and a word from a different root counted as this one	human
3	Arabic letters in source code silently reordered, widening a character range until it deleted the whole alphabet	Every count silently zero; no error raised. Occurred 4 times	machine
4	Counting at root level against word-level claims	Inflated some counts and wrongly destroyed two that were correct	machine
5	Lemma lookup silently merged three different words (malak / malik / mulk)	A total belonging to no word at all	machine
6	NFC reordering of shadda and kasra broke comparison of identical strings	False mismatches	machine
7	Word-index misalignment — the stored text counts the basmala, the morphology does not	Hundreds of verses with the wrong word highlighted	machine
8	Copyrighted translation quoted 645 times in a new document	Undistributable; violates a standing instruction	machine
9	Copyrighted translations live on the public site in three documents	Already published. The most serious defect found	machine
10	Sūrah 108 rendered an empty card	A visible hole in a document claiming to cover all 114	machine
11	Type-token ratio compared across unequal texts	A six-fold difference reported that was purely an artefact of length	machine
12	Zipf slope presented as an estimated exponent	A statistical over-claim a referee would catch	machine
13	Exclusivity counts inflated by hapax — half the “exclusive” roots occurred once anywhere	Headline overstating a real division	machine
14	Theme colour missing from one of three theme states	Light-mode badge on a dark card	machine
15	Stale artifact — Category 5 still served the pre-fix copyrighted version	Fix applied but not published	machine
16	Phase root counts appear not to sum to the corpus total (3,804 against 1,651)	Reader concludes the counting is broken. Numbers were right; the label invited a wrong reading	human
17	Document contradicting its own correction — §3.1 declares a measure misleading, §3.2 headlines it	Self-contradiction in a submitted paper	human
18	“Scholarly count ~1,650” asserted as a benchmark when published totals range 1,650–1,790	An apparent discrepancy that does not exist	human
19	Novelty not checked before claiming a finding — Sadeghi (2011) had published the substance	Rejection on sight	human
20	Audits raising false alarms — 4 false misalignments, then 258 false “Latin in Arabic”, then a corpus misread as 7 records instead of 36,390	An audit that cries wolf gets ignored, and the next real failure passes	machine
21	A superseded builder left runnable, and a second abandoned mid-write	Would have overwritten corrected output	machine

2. What the pattern shows

defects logged

found by machine

found by a human reading

found by both

The last figure is the important one. Two independent detectors have been searching the same work: automated audits, and a person reading the output closely. Their findings do not overlap at all. Not one defect was caught by both.

The machines caught silent corruption, arithmetic that did not reconcile, missing files, rights violations — failures with a definite shape that could be described in advance.
The human caught a number that looked wrong, a document disagreeing with itself, a claim that felt too strong, an unstated assumption. None of these had a shape until someone noticed them.

3. How many are left

Zero overlap between two detectors is not reassuring — it is the opposite. In fields that must estimate a population they cannot fully observe, the standard tool is capture–recapture: mark what the first detector finds, see how much of that the second detector re-finds, and infer the total from the overlap.

Chapman estimator N̂ = ((n₁+1)(n₂+1) / (m+1)) − 1 where n₁=15, n₂=6, m=0
N̂ = ((16 × 7) / 1) − 1 = 111

That number should not be quoted as an estimate, and it is not offered as one. The estimator assumes both detectors sample the same population with equal probability and independently. Here they plainly do not: the machines and the human were looking for different kinds of thing, which is exactly why their findings do not overlap.

But the direction is sound, and it is the honest answer to the question. When two searchers cover the same ground and share none of their findings, the reasonable inference is not that they have found everything. It is that each is blind to a whole category the other sees — and that neither has any idea how many categories remain unwatched by either.

We are not near the end of the search. We are early in it.

4. What to do instead of pretending

Since the fifth error cannot be ruled out, the response is to make it survivable rather than to claim it does not exist.

4.1 Publish in order of how narrow the claim is

An undiscovered error damages a paper in proportion to how much that paper claims. So publish the modest things first.

Order	What	The claim	Damage from an unfound error
1st	The concordance, as a dataset	“This annotation contains these roots”	Low — a correction is a version update, not a retraction
2nd	The counting-method paper	“A count needs a stated level”	Low — the argument survives a wrong worked example
3rd	Analysis of the lexical signature	“These roots distinguish these corpora”	High — an error in the measure invalidates the finding

4.2 Make it cheap for other people to find error five

Publish the code and the data, so a reader can recompute rather than trust. The value of doing this is precisely that someone may find what we did not.
State every count with its level and its method, so a wrong number can be located rather than merely suspected.
Print the evidence, not the summary — verse references for every judgement, so a reader can check the judgement instead of the conclusion.
Keep this register public and open-ended. A project that lists its own defects invites the next one to be reported rather than concealed.

4.3 Keep the human in the loop, permanently

Six of the twenty-one defects were caught by a person reading carefully, and no automated check had any chance of catching them, because none of them had been imagined when the checks were written. Automation catches what you thought of. A reader catches what you did not. Any process that removes the second is not more rigorous; it is blind in the direction it cannot see.