The goal, the measurable tests that decide whether it was met, and the three ways this fails — written before day 1, so that neither of us can move the target afterwards.
Produce a citable, downloadable dataset recording, for every one of 850 Qurʾānic content words, the meaning that the majority of twenty-one classical and modern commentaries agree on — with every dissenting reading preserved, every claim traceable to a verbatim quotation, and the rate at which annotators agree with one another published alongside it.
What this is a step toward, and is not yet. The long project is a commentary on the whole Qurʾān built from the agreement of the tradition. Seven days does not produce that. It produces the lexical foundation for it: the agreed meaning of the words that carry most of the text, established by method rather than by preference, and published so that scholars can correct it before anything is built on top.
| Artefact | Description |
|---|---|
| The dataset | One entry per lemma-sense for 850 words, in JSON and tab-separated form, on Zenodo with a DOI, licensed CC BY 4.0 |
| The evidence | Every entry carrying verbatim quotations from each commentary that glosses the word, with edition, death date and language |
| The disagreements | A separate file of every contested word — the raw material of the second paper |
| The reliability figures | Cross-window agreement, self agreement, and a hand-checked sample error rate |
| The tooling | Every script, so the whole dataset can be rebuilt from the corpus by someone else |
| Paper A draft | The resource description, for LREC or the Journal of Open Humanities Data |
| Paper B draft | Where the mufassirūn disagree, for the Journal of Digital Islamicate Research |
| The v2 roadmap | What was cut, what was deferred, and what a scholar must review before any of it is called a tafsīr |
Every one of these is measured by a script, not by opinion. All eight must hold.
| # | Test | Threshold | Measured by |
|---|---|---|---|
| 1 | Coverage. Lemmas with at least one completed entry | ≥ 700 of 850 | progress.py all |
| 2 | Gate. Entries failing validation at release | 0 | audit_entries.py all |
| 3 | Evidence. Entries whose every quotation is traceable to the corpus | 100% | clean-room checker |
| 4 | Cross-window agreement on the five held-out lemmas, day 6 | ≥ 70% | day-6 comparison |
| 5 | Self-agreement — a window reproducing its own day-2 answers | ≥ 80% | day-6 re-annotation |
| 6 | Hand-checked sample error rate, 40 random entries | ≤ 5% | Waleed, by hand |
| 7 | Deposit. DOI minted, tooling included, sources credited | yes | Zenodo |
| 8 | Honesty. Every figure above published, including the bad ones | yes | the paper itself |
Note what is NOT a success criterion: being right. Nobody can verify that from inside the project, and claiming it would require an ijāzah neither of us has. What can be verified is that the method was applied consistently, that the evidence supports what is claimed, and that annotators agreed with each other at a stated rate. A resource that reports its own error rate is usable. One that claims none is not trusted.
If a window falls behind, the response is always to cut lemmas and record the cut, never to lower the evidence standard. A dataset of 500 words with published agreement rates is a contribution. A dataset of 850 words where the last 300 were guessed is worthless, and worse, it is worthless in a way nobody can detect from outside.
Three failures, in ascending order of seriousness. The third is the only one that cannot be recovered from.
FAILURE 1 — not enough done. Fewer than 300 lemmas completed by day 6.
Detected by: progress.py all, daily.
Response: stop on day 6, publish nothing, and reschedule with a
smaller scope. The tooling and evidence survive; only the timetable
failed.
FAILURE 2 — the windows do not agree with each other. Cross-window agreement below 70%, or any window's self-agreement below 60%.
Detected by: the day-1 calibration, then confirmed on day 6.
Response: the rule is ambiguous, not the annotators. Rewrite the
instruction, re-run day 1, and restart. This is why calibration is on day
1 and not day 5 — discovering it early costs a day, discovering it
late costs the week.
FAILURE 3 — the dataset says things the evidence does not support. Any of: editions counted as agreeing that never glossed the word; quotations that cannot be found in the corpus; consensus recorded on contested entries; minority readings deleted.
Detected by: audit_entries.py, which already refuses
all ten of these; and by the hand-checked sample.
Response: do not publish. A resource that misrepresents the
tradition is not a smaller contribution than intended — it is a
negative one, and for a text like this one, it would be worse than doing
nothing at all.
Report what the commentators said. Never decide who is right.
Every entry may claim: “N of M editions that gloss this word here render it as X; the others say Y, recorded in full.”
No entry may claim: “This word means X.”
The first is a measurement any reader can check against the sources. The second is an act of scholarly authority. Keeping strictly to the first is what makes this work publishable by someone without formal training — and it is also, simply, what is true of what the method can establish.
| Day | Work | Must be true by nightfall |
|---|---|---|
| 0 | Scope locked, shards cut, cache built, windows opened | All 850 lemmas have evidence; nine shards verified; gate tested |
| 1 | All nine windows do the same five calibration lemmas | Cross-window agreement measured and ≥ 70%. If not, fix the rule and repeat day 1 |
| 2 | Production begins, own shards | Every window has completed ≥ 10 lemmas and passed its gate |
| 3 | Production | ≥ 30% of total lemmas done; flag rate stable |
| 4 | Production | ≥ 55% done. Decision point: if below 40%, cut scope now and record it |
| 5 | Production | ≥ 80% done, or the cut is already made and written into SCOPE |
| 6 | Re-annotation, held-out set, merge | Both agreement figures computed; dataset merged; clean-room recount agrees |
| 7 | Validate, sample-check, deposit, draft | DOI live; error rate published; both papers drafted |
Three things, in order of certainty.