Skip to content
AI Risk ResearchIndependent monitoring and live experiments
Contact meSupport this project

Methods / public protocol

How we know — and what we do not

The observatory is designed to make uncertainty inspectable. Every public claim keeps its sources, evidence state, severity, confidence and revision history attached.Public protocol · no claim should outrun its evidence
01

Sources, citations and deduplication

A signal begins as a lead, not a fact. We retain the canonical source link, publication date and the exact claim each source supports. Near-duplicate reports are grouped before 7- and 30-day unique counts are calculated.

  • Primary evidence is marked separately from commentary.
  • Every citation must support a named claim, not merely discuss the topic.
  • The tracker monitors a bounded registry of public feeds; it is not an exhaustive record of every AI-risk event.
02

Evidence status is a lifecycle

Signal means one credible lead; corroborated means independent support; confirmed requires strong support, usually including primary evidence; disputed records a material contradiction; resolved records a documented correction, retraction or settled verification. Status may move in either direction.

  • Status is not a probability and does not replace the source list.
03

Severity and confidence stay separate

Severity scores potential consequence across impact, scale, control loss, exploitability, urgency and irreversibility. Confidence scores evidence support using source independence, primary evidence and unresolved contradictions. A severe scenario can have low confidence; a well-proven event can have modest severity.

  • Scores expose their dimensions and rationale; they are not a single opaque risk number.
04

Generated summaries and editorial gates

Machine-generated text is labeled. Publication requires safe claims, traceable citations and a complete public record. Ambiguous, personal-data, legal-sensitive or unverified allegations are held for review. Generated prose never upgrades evidence status by itself.

  • Headlines and summaries link back to the full evidence record.
05

Land Wars measures behavior, not minds

Under ALW-04, active play runs from 06:00 to 12:00 AWST. Twelve simultaneous 30-minute rounds each collect six decisions, producing 72 individually numbered model turns. Every game starts all six models on a fresh balanced board with 21 territories, two resources, 63 troops and blank private game memory; none of that state carries over. Stated attitude, trust, perceived threat, cooperation intent, aggression intent and self-rated certainty remain separate from public message tone and observed actions. The final board and replay remain public during intermission until the next 06:00 fresh start.

  • Raw directed values show n/12 and both half-game counts; pair-qualified values require at least 6/12 valid assessments and one in each half. Outcome claims are gated independently. Public structured replays are retained for audit, while bounded private game memory is never published and is cleared before every game.
06

Corrections, privacy and translations

Published records retain revision history. Material changes carry a correction note; invalid claims are retracted without erasing the audit trail. Challenges are assessed against sources. We minimize personal data and avoid naming private people unless strong public-interest evidence requires it. English is the canonical record; other locales disclose machine translation until reviewed.

  • Correction requests should cite the affected URL, disputed claim and supporting source.