Thesis. A theory whose central quantity cannot be operationalized is not a theory; it is a mood. ODTOE survives this objection by providing five distinct recipes for estimating B(O, C) — each appropriate to a different empirical situation — and by being explicit about which recipe to use when. This post catalogues them.
Recipe 1: Paired-comparison elicitation
Use when: you have a human observer and you want a fast, defensible scalar estimate of their B on a specific class of claims.
Procedure: present pairs of claims (one in the observer's strong domain, one in a weak domain) and ask which they would defend more strongly under counter-evidence. Iterate until preferences are stable. The implied B-ranking is the elicited value. See Measuring B-parameter for the formal protocol and the standard error analysis.
Trades off: low cost, low precision. Good for screening, not for publication-grade measurement.
Recipe 2: Regression on known outcomes
Use when: the observer has produced many decisions and you can score the outcomes.
Procedure: regress outcome quality against the four components (F, E, σ, Λ) measured at decision time. The fitted exponents are the implied weights w1..w4; the predicted B for any new context is then computable.
Trades off: requires data and ground truth. Cleanest for forecasters, traders, and engineers; harder for one-off decisions.
Recipe 3: Perturbation response
Use when: you want to test whether a claimed B is real.
Procedure: perturb the context — inject noise (raise σ), redact some data (lower Λ), or present a contradiction (probe E). Measure how much the observer's stated position shifts. High-B observers shift less; low-B observers shift more. The slope of position-shift vs. perturbation is an indirect measure of B.
Trades off: invasive but rigorous. This is the technique closest to a physics experiment.
Recipe 4: Inter-observer convergence
Use when: you have several independent observers on the same configuration.
Procedure: compute pairwise agreement (e.g., Cohen's κ or analog), weighted by independent estimates of each observer's individual F. The convergence rate as you add observers — and the asymptote — give an estimate of the joint B of the panel. This is the technique used in the coherence measurability paper.
Trades off: requires multiple observers and an independent F-estimate. Best for collective claims (science, journalism, jurisprudence).
Recipe 5: Lambda-grounded decomposition
Use when: you have detailed context data and want a component-wise B.
Procedure: decompose Λ explicitly into its sub-components (source diversity, source recency, source independence, source verifiability). The lambda empirical reinforcement paper gives the canonical decomposition. Then estimate F and σ relative to the decomposed Λ. E is estimated from the observer's emotional alignment with the corpus. This is the most thorough recipe and the one used in formal corpus work.
Trades off: heavy machinery. Use when you need component-level diagnostics, not just a scalar.
When to use which
- Single human observer, fast read: Recipe 1.
- Track record exists: Recipe 2.
- Adversarial / rigorous setting: Recipe 3.
- Several observers: Recipe 4.
- Full diagnostic / publication-grade: Recipe 5.
What you should not do
Do not collapse B to a single Bayesian credence. Do not report B without specifying the recipe. Do not aggregate B across very different contexts C — B is context-relative, and a high B in one context says nothing about another. The belief paper has the full list of common pitfalls.
Cite this post
Pankratov, A. (2026). What Does It Mean to 'Measure' Coherence? Operational Recipes for B(O,C). ODTOE Blog. https://odtoe.org/blog/what-does-measuring-coherence-mean-operational-recipes