How to Check an AI Claim

A claim is about a system, not a vibe

News headlines, vendor tweets, and chat answers often talk about “AI” as if it were one creature with one skill. It is not.

AI is a category of machine-based systems. A claim is true or false about a particular system, on a particular date, doing a particular task.

Before you believe it, ask four questions:

1. Which model?
2. Which product?
3. Which date?
4. Which task?

If those four cannot be named, you do not yet have a checkable claim. You have a vibe.

Work a headline

Take a sentence like: “New AI is smarter than doctors.”

That sentence does not name a model, a product, a date, or a medical task. “Smarter” is not a measurement. You cannot check it until it becomes something like: “On this date, this vendor published that this model, in this product configuration, scored this result on this named test.” Then you can open the source.

Model, product, and agent are not the same thing

A model is a component: it takes inputs and produces outputs. A product wraps a model in an interface, tools, retrieval, memory, safety filters, and account rules. An agent is a system that can take actions over multiple steps — search, call tools, write files, send messages — under some set of permissions.

Tools, retrieval, memory, and permissions change what is true.

A statement such as “the model cannot see current news” may be true of weights used by themselves and false of a product that can search the web. A statement such as “it cannot spend money” may be true of a chat box and false of an agent with a payment tool. When you check a claim, name the whole system, not only the brand.

First-party, secondary, and rumor

Rank the source before you rank the excitement.

First-party means the developer published it: a system card, model card, technical report, product documentation, or a numbered evaluation the lab put its name on. OpenAI’s GPT-4 System Card is one example of that kind of document. [openai-gpt4-system-card] It describes evaluations and limits OpenAI chose to publish. It is not an independent audit.

Secondary means a journalist, blog, or social post summarizing someone else’s work. Secondary sources can be useful pointers. They are not the fact.

Rumor means an unsourced number or a leak with no document you can open. Unpublished parameter counts, secret token limits, and private training-set sizes are unverified until the developer publishes them. Do not treat them as facts. GPT-4’s parameter count is a famous case: widespread numbers circulate; OpenAI did not publish one in the system card or technical report.

Use first-party sources for what a lab published. Use independent evaluations for comparisons. Use legal text for what the law says. A tweet is none of those.

A benchmark measures a test, not “intelligence”

A benchmark is a named test with a dataset, a scoring rule, and a date. A high score means the system did well on that test, under those conditions.

It does not mean the system is intelligent in general. It does not mean the product you can buy today still matches the score. Leaderboards can be gamed, overfit, or outdated: people tune to the test, tests leak into training data, and new product versions replace the one that was measured.

Independent evaluations exist so you are not stuck trusting only the vendor’s slide. Stanford CRFM’s Holistic Evaluation of Language Models (HELM) is one such project: many scenarios and many metrics, not one “IQ for AI.” [liang2022helm] Read what was tested, on which version, with which tools allowed.

Fluency is not evidence

A fluent answer can be complete, incomplete, outdated, or wrong. Polished language is a style of output, not a proof.

For consequential facts — health, money, law, safety, someone’s job — check the underlying source. Ask the system for a citation if that helps you audit, then open the citation yourself. A generated footnote is not automatically real.

Do not paste secrets to “test” a claim

You do not need to drop a password, medical record, API key, or another person’s private data into a tool to see whether a claim is true.

If you want to test a product, use information you are allowed to share. Data handling depends on the provider, the product, the account type, the contract, and the settings. There is no universal rule that “nothing is stored” or that “everything is used for training.”

A short recap

• Name the system: model, product, date, task.
• Separate the model from the product from the agent.
• Prefer first-party documents for vendor facts; independent evals for comparisons; law from legal text.
• Treat unpublished numbers as unverified.
• Read a benchmark as a test result, not as intelligence.
• Check consequential facts. Fluency is not evidence.
• Keep secrets out of the test.

NIST’s AI Risk Management Framework is one public vocabulary for this kind of care: map what the system is, measure what you can, manage the risk, and govern the process. [nist-ai-rmf-2023] It is a voluntary framework, not a score and not a law.

The OECD’s updated definition of an AI system is a machine-based system that infers from inputs how to generate outputs such as predictions, content, recommendations, or decisions. [oecd-ai-2024] Start there. Then ask what this particular system actually is.