Cohesion Coding Rubric
The published method behind our analysis of submissions to the Royal Commission on Antisemitism and Social Cohesion. It describes structural features of an argument. It does not rank them, and it does not score anyone as right or wrong.
Where this has got to — 29 August 2026
This page was published before a single document was coded, and that was deliberate. An instrument released after its findings can always be suspected of having been shaped to produce them. Releasing it first is the only way to make that suspicion answerable rather than permanent.
A first pass over the corpus has now been made, and it is published below. It is a preliminary scan of a limited dataset, not a result: 82 submissions of the more than 20,000 received — about 0.4 per cent, chosen by the Commission rather than sampled, and the coding is not yet complete. One coder has now coded the six-document reliability subset; the second coding is under way. Nothing is published until both are finished and per-item agreement has been computed — and eight of the twenty-six items are excluded from that agreement test, for reasons set out in the method below. On its face that published set spans the opposing positions and includes submissions sharply critical of the Commission’s own framing — a point in its favour, and one we record here rather than leave to inference.
Next: settling the decision rule for item 2.4 before either coder opens a document, then independent double-coding of a subset and the reliability test. Items that two coders score differently will be revised or dropped.
Analysis of the New South Wales buyback
A separate piece of work, approached with the same discipline as this page: summarise what was actually said, attribute it, and be explicit about what has not been verified.
A public webinar on 3 September 2026 produced one claim that bears directly on the constitutional question raised in our companion paper — whether the Commonwealth attached conditions to the money it is putting into the scheme. The summary sets out what was said, what it would mean if it held up, and why, as it stands, it is worth nothing as evidence.
What this measures, and on what
The unit of analysis is one submission document. Every item is answered from the text of that document alone, with a verbatim quotation and a page reference recorded against every score.
The rubric asks four questions of each submission. What does it propose people be answerable to, and to whom? How does it represent what other groups believe? What does it make permissible, and what restrains that? And what kind of reason does it run on?
An item with no supporting quotation is recorded as absent, never as zero, and never inferred. Where a coder is uncertain, the entry is “unclear”. Guessing is the failure this instrument exists to prevent.
What this rubric does not do
This is the load-bearing constraint. An instrument that scored submissions as better or worse would become a device for ranking people its author disagrees with, and would rightly be read that way whatever the stated intent.
No quality, persuasiveness or merit score. A submission that codes as monolithic and unrestrained is not thereby wrong; one that codes as evidenced and restrained is not thereby right.
No record of whether we agree with any position taken in any submission.
No finding on whether a claim is true. Item 2.2 asks only whether evidence was offered — never whether that evidence was good.
No conclusion about an author’s motives. Every code describes a feature of a text.
Any argument about what a pattern means for social cohesion is a separate layer, kept out of the scoring and defended on its own.
Four dimensions, twenty-six items
Accountability structure
What the submission proposes people be answerable to, and to whom. Not a question about the author — a question about what the proposal builds.
- 1.1A named, identifiable role or office carrying a duty?Present · Absent · Unclear
- 1.2A register, licence or standing record of who holds that duty?Present · Absent · Unclear
- 1.3Where does accountability run?Local · Central · Both · Neither
- 1.4Does the duty persist beyond a single transaction or event?Present · Absent · Unclear
- 1.5A stated mechanism for sanctioning or removing someone who fails the duty?Present · Absent · Unclear
- 1.6Does the structure bring members of different communities into repeated contact?The cross-cutting item. Conflict falls where affiliations cut across one another and rises where they align, so this item carries most of the dimension’s weight.Present · Absent · Unclear
Meta-perception
How the submission represents what other groups think, believe or intend — and whether it shows any sign of having checked.
- 2.1Does it attribute beliefs, motives or intentions to a group other than its own?Present · Absent
- 2.2For each attribution: evidenced, or asserted? Recorded as two counts.Evidenced means the submission quotes, cites or links to something that group actually published or said. This item does not assess whether the evidence is any good.
- 2.3Does it distinguish a violent or extreme minority from the general population of that group?Present · Absent · Unclear
- 2.4Does it anywhere concede that another group has genuine grounds for fear?Not agreement — acknowledgement that the fear has a basis. Expected to be the single most diagnostic sentence in any document in this corpus.Present · Absent · Unclear
- 2.5Does it concede good faith to any party it disagrees with?Present · Absent · Unclear
- 2.6Is the other group treated as internally varied, or as a single actor?Varied · Monolithic · Mixed · Unclear
The paired test
This runs across the corpus rather than on a single document, and it is why the test set was chosen for mutual reference. Where submission A attributes a belief to group B, and B has its own submission in the corpus, the attribution is set beside what B actually wrote.
| Code | Meaning |
|---|---|
| Accurate | B’s own submission states substantially what A says B believes. |
| Partial | B states something adjacent, but narrower, broader or qualified in ways A omits. |
| Contradicted | B’s own submission states the opposite, or expressly disclaims the position. |
| Unaddressed | B’s submission does not deal with the point; no comparison is possible. |
Only contradicted and partial findings support any claim about a perception gap, and only for the pair concerned. Nothing here licenses a claim about what a community believes — only about what one document said concerning another document’s authors.
Justification structure
What the submission makes permissible, and what it puts in the way. The dimension on which any proposal that grants a power — including Hidden Shield’s own — is most exposed.
- 3.1Does it identify a class of people, as distinct from a course of conduct, as the problem?Present · Absent · Unclear
- 3.2Does it propose any measure restricting the liberty, speech, funding, movement or status of an identified group?Present · Absent · Unclear
- 3.3If 3.2 is present, does it propose a corresponding restraint, safeguard, review or appeal?A named mechanism is required. An adjective is not a restraint.Present · Absent · Not applicable
- 3.4Is any value framed as non-negotiable — outside trade-off or balancing?Sacred framing is not a fault. It predicts which disagreements can be bargained and which cannot.Present · Absent · Unclear
- 3.5Is dehumanising register used of any group?Expected to be rare in this corpus. It must still be codeable, and quoted in full where found.Present · Absent
- 3.6Is the duty the submission asserts owed universally, or only to its own community?Universal · Own community · Both · Unclear
Warrant type
What kind of reason the argument runs on. Not exclusive — every type present is coded, then the dominant one marked. Two parties with opposite conclusions and a shared warrant can argue; two with the same conclusion and different warrants often cannot, and rarely see why.
- 4.1Legal-rights — statute, common law, international instruments, constitutional argument.
- 4.2Empirical-evidentiary — data, studies, incident counts, research findings.
- 4.3Moral-deontological — duty and obligation, right and wrong as such.
- 4.4Theological — scripture, tradition, covenant, religious teaching.
- 4.5Experiential-testimonial — lived experience, personal or community account.
- 4.6Institutional-procedural — how bodies should operate, governance, process, oversight.
- 4.4aWhere a theological warrant is present: which tradition, and what duty is grounded in it?Recorded in the submission’s own words, untranslated. This is the input to the overlap analysis — the same obligation surfacing across traditions in different vocabularies is the finding, and paraphrase destroys it.
Four values, and no composite score
Nothing else is derived. No index is built by summing across dimensions — a single total would smuggle a merit ranking back in through the arithmetic.
| Measure | Definition | Reads as |
|---|---|---|
| Cross-cutting | 1.6 present and (1.1 or 1.2 present) | The proposal creates a named duty that puts different communities in repeated contact. |
| Attribution ratio | evidenced ÷ (evidenced + asserted) | How much of what this document says about others it troubled to source. |
| Fear conceded | 2.4 present | Binary. Reported as a raw count across the corpus, never averaged. |
| Permission without restraint | 3.2 present and 3.3 absent | A power is sought and nothing is offered to bound it. |
Decision rules for the contested items
Eleven items carry almost all the risk of two coders reaching different answers. Each has an explicit rule and a worked boundary case. Where a document is still not settled by the rule, the entry is “unclear” and the item’s wording goes on the revision list.
Where accountability runs
Code by where the duty-holder answers when something goes wrong, not by where the paperwork is kept. A national register operated locally is still central if a complaint travels to the national body.
Cross-cutting contact
Present requires that the mechanism necessarily brings different communities together as a feature of how it operates.
A body whose membership is drawn from more than one community; a duty discharged jointly.A body that undertakes to consult widely. An author’s hope that a measure will improve relations.Evidenced or asserted
Evidenced requires a locatable source attributable to the group itself — its publication, statement, testimony or filing.
A quotation from the group; a study measuring that group’s attitudes; a media report attributing a statement to it, flagged as second-hand.“Their real aim is…”, “everybody knows that…” with nothing behind it.Distinguishing the extreme from the general
Requires an explicit distinction in the text. The absence of a sweeping claim is not a distinction — this is the commonest way a coder credits a document with something it never said.
“A small number within…”, “elements of…”, “this is not representative of…”Silence. Careful phrasing. A document that simply never generalises.Conceding genuine grounds for fear
The most consequential item in the instrument, and the one most likely to be coded generously. Present requires both: a fear held by a group other than the author’s own is named, and the text indicates that fear has some basis in fact or experience.
“Muslim Australians have reason to fear for their safety after the events of…” — a fear named, and grounded.“We condemn attacks on mosques.” That condemns conduct; it concedes no grounds for fear. Nor does noting that a group claims to be afraid, nor general regret about division, nor recording incidents while disputing that they justify the fear.Varied or monolithic
Varied requires explicit acknowledgement of disagreement or diversity within the other group. Where a document does both in different passages, code mixed and record both quotations — the contradiction is itself data.
A class of people, or a course of conduct
The hardest line in the instrument. A class is identified where the problem is located in who people are or which group they belong to. Conduct is identified where the problem is located in what is done, whoever does it.
Class: a religion, ethnicity, nationality or community named as the problem.Conduct: “organisations that fund…”, “those who chant…” — defined by what is done or professed, not by ascribed identity.A restrictive measure
Includes criminal prohibition, deregistration, defunding, loss of charitable status, visa or immigration consequence, protest restriction, employment consequence.
Calls to enforce existing law; education programmes; monitoring or reporting that carries no consequence.A corresponding restraint
Requires a named mechanism. An adjective is not a restraint, and this is precisely where a sympathetic coder will be tempted.
Judicial review, a right of appeal, a sunset clause, independent oversight, a defined evidentiary threshold, a stated review period.“Proportionate”, “carefully applied”, “with appropriate safeguards”, “subject to due process” — where no mechanism is specified.A non-negotiable value
Requires framing that places the value outside balancing altogether. Strong advocacy is not sacred framing, and treating it as such would code almost every document in the corpus present.
“Under no circumstances”, “never acceptable in any context”, “not a matter for negotiation”.“Vitally important”, “must be a priority”, “we urge in the strongest terms”.Whose duty, and which warrant dominates
For 3.6, read the duty asserted, not the harm described. A submission may describe harm to its own community at length while asserting a duty owed to everyone; that is universal.
For the dominant warrant, the test is load-bearing rather than word count: the warrant which, if removed, would collapse the argument. A submission opening with three pages of testimony and resting its recommendation on the Racial Discrimination Act is dominantly legal-rights.
How a document is coded
- Read the whole document once without coding. Coding on first read produces scores anchored to whichever passage came first.
- Second pass: code every item, recording a verbatim quotation and page reference for each. No quotation, no score.
- Enter “unclear” wherever the text will not settle the question. The rate of unclear entries is itself a diagnostic — an item that is routinely unclear is badly worded and gets revised.
- Two coders work independently and do not confer until both are finished. Discussing a document mid-code destroys the only check this instrument has.
- Compare, and compute percentage agreement for each item across the reliability subset.
- Any item scoring below 80 per cent agreement is revised or dropped before the rubric is used on the rest of the corpus.
- Freeze the rubric. Write down the expected findings. Then code the rest. A pattern predicted and found is evidence. A pattern produced by adjusting the instrument until it appeared is not.
This rubric will change. Here is when, and why.
“Subject to change” on its own is a hedge, and it invites the obvious suspicion — that the instrument will be adjusted when the results are unwelcome. So the conditions for change are set out here in advance, before any document has been coded.
Before the rubric is frozen
Changes at this stage are expected, and are driven by one thing only: the reliability test. An item that two coders scored differently, an item that proved ambiguous in practice, or a category the documents plainly contain that the rubric failed to anticipate. Every change is logged with the reason for it.
After the rubric is frozen
Once coding of the full corpus begins, any change to the instrument requires the original result to be published alongside the revised one. A revision that quietly replaces a finding is not a revision, it is a retraction without notice.
In every case
Every version remains available at its own address. Nothing is silently replaced, and the change log records what altered and why. If this page ever shows a rubric whose history has been tidied away, the work on it should be disregarded.
Change log
v1.1 — 27 August 2026. Added decision rules and boundary cases for the eleven items carrying the highest divergence risk. Item wording unchanged.
v1.0 — 27 August 2026. First draft. Four dimensions, twenty-six items, four derived measures. Nothing coded against it.
Two standing caveats
The published set is not a sample
Fifty-five submissions are published of more than twenty thousand received — roughly 0.3 per cent. The published set is selected by the Royal Commission, released in batches themed around hearing blocks, and requires the author’s consent.
No finding from this corpus can be stated as what submitters think. It can only be stated as what the organised advocacy landscape the Commission chose to surface contains. Any claim on this site that goes further than that is an error, and we would like it pointed out.
The author has an interest
The Hidden Shield submission is one of the documents to which this rubric is applied, and its author built the rubric. That is a real conflict, and it is not cured by declaring it.
It is partly answered by three things: the instrument was published before any findings; the raw coding sheet will be published so that anyone may re-score the documents; and where Hidden Shield codes worst will be reported in the visible part of the results, not an appendix. On the reading of the rubric’s own author, items 3.2 and 3.3 are where that is expected to bite — Hidden Shield seeks a permission, and rests its restraint on an oath.
09 — Working papers
Preliminary analysis of a limited dataset
Working papers are published here as they are produced, before coding has been done, so that the reasoning can be followed and corrected from the beginning rather than presented as settled.
The first is a transcript of a working session held on 28 August 2026. It reports a machine scan of the Royal Commission’s published submissions, a reading of the titles authors gave their own documents, and a test of the Hidden Shield submission against evidence in the corpus that supports it alongside evidence that tells against it.
It is preliminary, and the dataset is limited. The Commission has published 55 submissions of the more than 20,000 it received — roughly 0.3 per cent — and that set was selected by the Commission rather than sampled. No claim about what submitters in general think can rest on it. Nothing in the transcript has yet been coded against the rubric above, and several of the patterns it describes are expected not to survive coding. It is offered as work in progress, not as a finding.
Developing a Social Cohesion Rubric — working transcript, 28 August 2026 (PDF)
How to disagree with this
Anyone may take the same documents, apply the same rubric, and reach different scores — and if you do, we would like to see them.
Disagreement is more useful to this work than agreement. An item that two people outside this project score differently is an item that needs rewriting, and we would rather learn that from you than defend it later. Correspondence is welcome from researchers, from the organisations whose submissions appear in the corpus, and from anyone who thinks a code has been applied wrongly.
Download the blank coding sheet The published submissions
Submissions are published by the Royal Commission on Antisemitism and Social Cohesion and are reproduced nowhere on this site. All links go to the Commission’s own pages.