Instrument

AGI hype check

Hold a claim up to the light. Tick what the claim actually provides, and any red flags you notice. The hype check scores the evidence, not the technology.

Evidence the claim provides The claim says what it means by "AGI" or the capability involved. It names specific tasks or benchmarks the system was tested on. Results are compared with a baseline: humans, earlier systems, or simple methods. Someone other than the developer has evaluated or reproduced the result. The evaluation method is published (paper, system card, or technical report). Limitations or failure cases are described, not only successes. People outside the company can actually use or test the system. Steps to avoid test-data leakage (contamination) are described.

Red flags you notice The main evidence is a polished demo or video. It relies on words like "sparks", "approaching", or "basically AGI" without measures. It generalises from one impressive example to general intelligence. The source benefits financially from the claim and offers no outside check.

How the score works

Each evidence item present adds 10 points, up to 80. Each red flag subtracts 10, with a floor of zero. Bands: 60 and above, well evidenced; 40 to 50, partly evidenced; 20 to 30, mostly shine; below 20, all bling.

Limitations: the weights are a simple editorial judgement, not a validated instrument. A claim can score well and still be wrong, or score poorly and later prove right. Use it to decide which questions to ask, guided by how to read benchmarks and demo versus deployment.

A magnifying glass resting on a page of printed text — Photo: Spencer Liao / Unsplash