The thing that separates a serious AI security publication from an aggregator is not what it covers — it’s what it refuses to publish, and why. The decision rule is the source ladder. This post walks through ours.
The ladder, in one paragraph
We rank sources in five tiers. Tier 1 is primary (vendor advisory, court filing, regulator action, peer-reviewed paper). Tier 2 is established secondary (Reuters, Bloomberg, KrebsOnSecurity, 404 Media, The Markup) where a reporter has independently verified. Tier 3 is specialty secondary (AI-focused publications, named researcher disclosures). Tier 4 is user reports (individual accounts, triangulated). Tier 5 is speculation and aggregator content. We publish only when we have at least one Tier 1–2 source. The single exception is the provisional entry: a narrow Tier 3-only case, or a Tier 4 pattern triangulated across independent accounts, can run while clearly marked verification: partial.
Tier 1 — Primary
These are the events themselves, not reports about them.
- Vendor advisories: a security disclosure published by the model provider on its own domain, signed by its security team. Anthropic’s
anthropic.com/news, OpenAI’sopenai.com/index/disclosures, Google’s VRP advisories. - Court filings: PACER documents, EU court records, regulator complaint dockets. Filings are not findings — a complaint alleges, it does not prove — but the document itself is primary.
- Regulator actions: FTC consent orders, EU AI Office decisions, UK ICO enforcement notices, NIST guidance. These have legal weight.
- Peer-reviewed disclosures: a paper at USENIX Security, S&P, NDSS, AAAI Safe AI workshop, or AI Village proceedings, where the affected vendor has been notified.
If we have one Tier 1 source, we publish. That’s it.
Tier 2 — Established secondary
Reporting from publications with a track record of verification. We treat Reuters’ AI security desk, Bloomberg, the Wired security beat, KrebsOnSecurity, 404 Media, and The Markup as Tier 2 when the reporter has named primary sources and includes a date of verification. The newsroom’s editorial standards stand in for our own where we can’t independently reach the source.
The trap here is reading an AI-summarized version of a Reuters story on an aggregator and treating it as Tier 2. The summary is Tier 5. We cite Reuters directly or we don’t cite at all.
Tier 3 — Specialty secondary
This is where most AI-specific incidents live, and where editorial judgment does the most work.
- AI-specific publications with named editors and a corrections page (rare, but they exist).
- Named researcher disclosures on platforms with stable identity (institutional pages, lab-affiliated blogs, signed PGP posts).
- Vendor blog posts that are not technically advisories — a write-up that doesn’t go through the security team’s process but is published under a real engineer’s name.
A Tier 3 source on its own is enough only when combined with a Tier 4 corroboration or a partial Tier 1 (e.g., the vendor confirmed in passing on a podcast). When we publish from Tier 3 alone we flag the entry verification: partial.
Tier 4 — User reports
Individual users describing things. A single Twitter thread is not enough. A pattern across multiple independent accounts — different geographies, different timestamps, different framings of the same outcome — can be enough to publish a provisional entry while we work toward Tier 2.
Provisional entries are clearly labeled verification: partial. They don’t get a permanent slot until they level up to Tier 2.
Tier 5 — Don’t
AI-summarized news posts. Anonymous Reddit claims. Unverified screenshots. SEO-farmed listicles. The Slack screenshot with no headers. We do not link to Tier 5 sources even to debunk them — citing them is a signal-boost.
Two worked patterns
These are illustrative patterns rather than accounts of specific entries. They show how the ladder resolves two common shapes of claim.
Tier 4 plus a Tier 1 corroboration clears the bar. A named security engineer at a model provider posts a thread describing a deserialization bug in a fine-tuning ingest path, naming the affected commit and the patch. The thread alone is Tier 4. If that same patch is independently visible in the vendor’s public release notes, the release note is a Tier 1 corroboration and the pair meets the threshold. The entry then cites the release note, not the thread.
Ten reposts of one unreached claim do not. A viral claim circulates that an open-source model shipped weights containing embedded credentials. Ten secondary outlets carry it, none of them reached the original poster, and the repository is taken down before any reporter examines it. Ten Tier 5 restatements of a single unreached Tier 4 claim do not add up to a Tier 2 source, so the claim stays unpublished until someone reaches primary evidence.
What “verification” actually means
It is not the same as “true.” It is the documented process by which we satisfied ourselves the claim is supported by the strongest available evidence. Verification is reproducible: a reader looking at our cited sources should be able to recover the same conclusion without trusting us.
When we get verification wrong, we correct the entry in place, date the correction, and state what changed and why.
Cross-references for readers
For the broader methodology this ladder slots into, see our incident logging methodology. For which public catalog to check a claim against before running it, see the AI incident database comparison. For the same tiering applied under time pressure inside an organization rather than in a newsroom, see the incident-response playbook for AI systems. For the editorial standard applied in the weekly cycle, see how AI Sec Weekly works. For mitigation-side coverage when an incident has defensive implications, the defensive cluster’s guardml.io covers guardrails, content filters and model defenses.
Why this matters
The AI security beat is fast. Models change in days. Tools ship in weeks. Aggregators race to publish. A reader trying to understand whether a thing actually happened — or whether they should patch on Monday — needs a publication that does the verification work and shows it. The ladder is the work.
