How a coder reads a submission
The Cohesion Coding Rubric asks twenty-six questions of a submission. This page shows the instrument as a whole, and then shows the most error-prone of those questions being answered on two real pairs of documents — including the mistake a coder makes if they read quickly.
The model
One submission goes in. Twenty-six items are answered against it, each with a verbatim quotation and a page reference. Eight of those items feed four derived measures. Then nothing converges: there is no total, and the absence of the total is the design.
Cells outlined in steel are the eight items that feed a derived measure. Cells tinted gold are the eight excluded from the agreement test. A dot marks a keystone item. The gold dashed wire is the one input a measure needs absent rather than present. Nothing on this diagram is coloured by how any document scored.
How a coder picks the sentence
Item 2.1 asks whether a submission attributes beliefs, motives or intentions to another group. Where that group has its own submission in the corpus, the paired test sets the two side by side and codes the pair ACCURATE, PARTIAL, CONTRADICTED or UNADDRESSED. Only CONTRADICTED and PARTIAL support a perception-gap claim, and only for that pair.
- Look for a claim about what another group thinks, not an opinion about them. “They are dangerous” is an opinion. “They assert X”, “their real aim is Y”, “they believe Z” is an attribution. The test is whether it could in principle be checked against something that group said.
- Quote it exactly, with its page. Verbatim, never a summary. A summary is already a coding decision, made before the rule was applied.
- Decide evidenced or asserted. Evidenced requires a locatable source attributable to the group itself. This is the step coders get wrong most often, and the first example is why.
- Find what the other group’s own submission says about the same thing. Not the nearest sentence, and not the most quotable one. The one addressing the same object as the attribution.
- Code the pair, and write the coder note. Where the call was close, record the reading you rejected and why. Those notes are what the next version of the rubric is built from.
Example one — an attribution the other party expressly denies
CONTRADICTED
B addresses the same object — whether its activity is antisemitic — and denies it. That is a real contradiction between two documents in front of the same Commission, and the rubric records it without deciding who is right.
The trap, and it is the whole lesson
Fifteen pages earlier, on p. 6, Free Palestine Melbourne writes that “the protests for a free Palestine have encompassed a broad and diverse group of people”, and lists Jewish organisations among the attendees. It sits beside the topic, it reads like a rebuttal, and it is the sentence a hurrying coder takes. It is about a different object. A’s claim is about content generated online; that sentence is about who turns up to rallies. Coding the two as a contradiction would put a mismatched pair in the published sheet and inflate the perception gap with something that was never there. Select on the object of the claim, never on proximity or on how well the sentence argues the point.
Now code the attribution itself — evidenced, or asserted?
A’s sentence carries footnote 117, a citation to a report of the Israeli Ministry of Diaspora Affairs. A citation makes the claim checkable. It does not make it evidenced. The rule reads: a locatable source attributable to the group itself. A third party’s ranking of Free Palestine Melbourne is not Free Palestine Melbourne saying anything. It codes as asserted, footnote and all.
Example two — an attribution that is neither confirmed nor denied
PARTIAL — AND THE RULE DOES NOT SETTLE IT
The same officer is the source of both. B’s submission speaks of “many”, not a vast majority, and about how Zionism was experienced rather than how many people identify with it. Nothing here confirms the quantified claim, and nothing denies it. A coder could defend PARTIAL, and could defend UNADDRESSED. The rubric does not yet say which — so the coder writes the note:
Paired test, A → ZFA. B’s own submission is by the same officer but makes no quantified claim; it says “many Australian Jews” and speaks to lived experience, not rates of identification. Coded PARTIAL on the basis that it addresses the underlying territory. UNADDRESSED is equally available and I could not separate them on the rule as written. The paired test needs a test for whether B has addressed the same proposition or merely the same subject.
That note is worth more than the code. It is what the next revision of the paired test will be built from. And in the same passage A makes a second attribution — that it is “cynical to conflate criticism of the government of Israel with antisemitism” has “been asserted by the Jewish Council of Australia” — which is a second named group with its own submission, and a second pair to code. Attributions come in clusters. Count each one.
Why these documents, and how to have the coding corrected
These four submissions were chosen for the shape of the coding problem in them, not for who wrote them or what they argue. Specifically, they were chosen because they sit outside the six-document set being used to test whether two coders applying this rubric independently reach the same answers. Working an example on those six would contaminate that test. Every quotation above has been checked against the Commission’s published PDF at the page given.
Being used as a worked example is not a finding about any of these organisations, and no organisation named here has been ranked, scored or assessed. What is shown is a method being applied in public, on documents anyone can download and check.
If you are one of the organisations quoted here and you believe the coding is wrong, say so and it will be corrected on this page, with the correction recorded rather than quietly made. That applies to the selection of the sentence, the verdict on the pair, and the evidenced-or-asserted call alike. The whole point of publishing the instrument before the findings is that the coding can be argued with while it still matters.
What the paired test does not do
It checks the corpus, not the world. It asks what B’s submission says, never whether A is right. Free Palestine Melbourne denying the characterisation does not make the characterisation false, and the Zionist Federation’s chief executive may well have said exactly what he is quoted as saying somewhere outside his submission. Neither question is in front of the coder.
It does not score either party. A CONTRADICTED pair is a finding about two documents, held to that pair. It is not evidence of a general perception gap, and it is not a mark against anybody.
It is not yet reliability-tested. Two coders are coding the same six documents independently. Until that is done and per-item agreement is computed, every code on this page — including the two above — is one coder’s reading, published so that it can be checked.
The rubric in full Download the blank coding sheet All analysis and papers