About truth-bot
What this is
truth-bot is an automated political fact-checker that decomposes speeches and public statements into atomic, individually verifiable claims, then runs each claim through multiple large language models simultaneously. Each model performs live web searches and returns a structured verdict. The results are aggregated into a consensus verdict with an explicit strength score.
How verdicts are produced
Each claim is sent to all configured LLM providers in parallel. Every provider is instructed to search Tier 1 government sources first (BLS, BEA, CBP, etc.) before citing secondary sources. The providers return a structured JSON verdict including a label, confidence level, explanation, source URLs, and a self-reported caveats field flagging source-quality gaps.
Consensus is computed by majority vote. Three or more models returning the same label = "Strong consensus." Two models agreeing = "Weak consensus." No majority = "Models split."
Headline pill: 5-bucket coarse-axis projection
Each model returns one of six labels — True, Mostly True, Exaggerated, Misleading, False, Unverifiable. Those per-model labels are preserved in full on the per-claim model strip for audit. The headline pill on each claim collapses the panel into a five-bucket "Truthy scale" (True · Truthy · Unverifiable · Falsey · False) so directionally aligned panels don't look split just because two models split Mostly True from Exaggerated.
Two projections are published side-by-side and you can flip between them with the Lens chip in the status bar:
| 6-bucket label | Lenient (default) | Strict |
|---|---|---|
| True | True | True |
| Mostly True | Truthy | Truthy |
| Exaggerated | Truthy | Falsey |
| Misleading | Falsey | Falsey |
| False | False | False |
| Unverifiable | Unverifiable | Unverifiable |
Split-projection guardrail: if the panel still has no plurality after projecting (e.g. 2-2 Truthy/Falsey), the headline shows "Models split" rather than tie-breaking — the projection is not allowed to manufacture agreement that isn't there.
Per-model strip is unaffected. Individual model verdicts always render on the original 6-bucket axis so you can see the editorial nuance each model assigned, even when the headline rolls up to Truthy or Falsey.
Models
- Anthropic claude-opus-4-7 — primary verifier
- OpenAI gpt-5.4 — verifier (batch mode)
- Google gemini-2.5-pro — pending API key
- xAI grok-4 — pending API key
Source tier hierarchy
| Tier | Sources | Trust weight |
|---|---|---|
| Government | .gov, .mil — BLS, BEA, CBO, Census, CDC, etc. | Highest |
| Wire | AP, Reuters | High |
| Established | NYT, WaPo, BBC, NPR, CBS, NBC, ABC | Medium-High |
| Academic | Peer-reviewed journals, university presses | Medium-High |
| Fact-check | PolitiFact, FactCheck.org, Snopes, FullFact | Medium |
| Other | Blogs, opinion sites, social media, unverified sources | Low |
Known limitations
- Small corpus: Each claim is verified independently with no cross-claim context. Recurring rhetoric may be rated inconsistently across speeches.
- Hallucinated citations: Language models can fabricate plausible-looking URLs. All cited sources should be independently verified before drawing conclusions.
- Training-data bias: Model verdicts may reflect the political slant of their training data. The multi-model consensus is designed to partially mitigate this, but systematic bias in all four providers would not be caught.
- Temporal grounding: Web search retrieves recent results but may miss updated statistics or corrections published after the pipeline run. The generation timestamp on each report indicates when verification occurred.
Full verdict prompt (hash: 7c462496)
Verbatim prompt sent to each model for verdict synthesis.
You are an expert fact-checker. Given a claim and a set of evidence snippets, use your web search tool to research the claim and return a verdict.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
STEP 1 — MANDATORY PRIMARY-SOURCE SEARCH
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Before citing any aggregator, news outlet, or commentary site, you MUST attempt to retrieve relevant Tier 1 government primary sources if the claim touches any of the following domains:
• Jobs, unemployment, wages, inflation, labor force → search site:bls.gov
• GDP, trade, personal income → search site:bea.gov
• Federal budget, deficit, national debt → search site:cbo.gov OR site:treasury.gov
• Social Security, Medicare → search site:ssa.gov OR site:cms.gov
• Health data, disease statistics → search site:cdc.gov OR site:hhs.gov
• Census data, demographics, population → search site:census.gov
• Energy, oil, gas, electricity production → search site:eia.gov
• Education data, graduation rates, test scores → search site:nces.ed.gov
• Crime statistics → search site:bjs.ojp.gov OR site:fbi.gov
• Immigration, border data → search site:dhs.gov OR site:cbp.gov
• Weather, climate, ocean, satellites → search site:noaa.gov OR site:climate.gov
• Geological hazards, maps, minerals → search site:usgs.gov
• Consumer protection, antitrust, fraud → search site:ftc.gov
• Communications, spectrum, broadband → search site:fcc.gov
• Transportation safety, aviation, highways → search site:ntsb.gov OR site:fhwa.dot.gov
• Public transit statistics → search site:transit.dot.gov OR site:bts.gov
• Agriculture, crops, food safety → search site:usda.gov OR site:fda.gov
• Environmental regulations, emissions → search site:epa.gov
• Federal workforce, pay scales → search site:opm.gov
• Workplace safety → search site:osha.gov
• Housing programs, fair housing → search site:hud.gov
• Securities markets, corporate filings → search site:sec.gov
• Veterans benefits, health, education → search site:va.gov
• International affairs, treaties, aid → search site:state.gov OR site:usaid.gov
• Defense budgets, military personnel (public) → search site:defense.gov OR site:dod.mil
• Small business statistics → search site:sba.gov
• Patents, trademarks, IP → search site:uspto.gov
• Federal courts, opinions (public records) → search site:courtlistener.com OR site:uscourts.gov
• IRS tax statistics, forms guidance → search site:irs.gov
• Federal procurement, contracts → search site:usaspending.gov OR site:sam.gov
• NASA missions, Earth science → search site:nasa.gov
• National security public datasets → search site:odni.gov OR site:dhs.gov
• Nuclear regulation → search site:nrc.gov
• Mine safety → search site:msha.gov
• Railroad safety → search site:dot.gov OR site:frs.dot.gov
• Pipeline safety → search site:phmsa.dot.gov
• Maritime administration → search site:maritime.dot.gov
• Federal Reserve economic data → search site:federalreserve.gov OR site:stlouisfed.org
• FDIC bank data → search site:fdic.gov
• NCUA credit union data → search site:ncua.gov
• Treasury interest rates, debt → search site:fiscaldata.treasury.gov
• USAspending / federal awards → search site:usaspending.gov
• Grants.gov program data → search site:grants.gov
• Regulations.gov rulemaking → search site:regulations.gov
• Federal Register → search site:federalregister.gov
• Congress bills, votes (public) → search site:congress.gov
• GAO reports → search site:gao.gov
• OMB budget guidance → search site:whitehouse.gov/omb OR site:omb.gov
• CFTC derivatives markets → search site:cftc.gov
• FinCEN financial crimes (public) → search site:fincen.gov
• BIS export controls (public) → search site:bis.doc.gov
• ITA trade enforcement → search site:trade.gov
• NOAA fisheries → search site:fisheries.noaa.gov
• USFS forestry → search site:fs.usda.gov
• NPS visitation, parks → search site:nps.gov
• BLM land use → search site:blm.gov
• FEMA disaster programs → search site:fema.gov
• ATF firearms statistics (public) → search site:atf.gov
• DEA drug scheduling (public) → search site:dea.gov
• SAMHSA behavioral health → search site:samhsa.gov
• NIH research grants directory → search site:reporter.nih.gov
• NSF science statistics → search site:nsf.gov
• DOE energy programs → search site:energy.gov
• NIST standards → search site:nist.gov
• USITC trade investigations → search site:usitc.gov
• Ex-Im Bank financing (public) → search site:exim.gov
• OPIC / DFC development finance (public) → search site:dfc.gov
• Peace Corps programs → search site:peacecorps.gov
• Smithsonian collections (public) → search site:si.edu
• Library of Congress collections → search site:loc.gov
• National Archives records → search site:archives.gov
• Census economic indicators → search site:census.gov/economic-indicators
• BEA regional accounts → search site:apps.bea.gov
• BLS CPI/PPI methodology → search site:bls.gov/cpi
• CMS open data → search site:data.cms.gov
• HRSA health workforce → search site:hrsa.gov
• AHRQ quality measures → search site:ahrq.gov
• FDA drug approvals database → search site:accessdata.fda.gov
• USDA nutrition database → search site:fdc.nal.usda.gov
• EPA enforcement/compliance → search site:echo.epa.gov
• DOT crash statistics (NHTSA) → search site:nhtsa.gov OR site:crashstats.nhtsa.dot.gov
• FAA operations data → search site:faa.gov/data
• TSA throughput (public) → search site:tsa.gov
• CBP travel statistics → search site:cbp.gov/newsroom/stats
• ICE detention statistics (public) → search site:ice.gov
• USCIS immigration statistics → search site:uscis.gov/tools/reports-and-studies
• State visa bulletin → search site:travel.state.gov
• USTR trade agreements → search site:ustr.gov
• U.S. Mint coin production → search site:usmint.gov
• Bureau of Engraving currency → search site:bep.gov
• USPS service performance → search site:usps.com/household/service-performance
• FCC broadband maps → search site:broadbandmap.fcc.gov
• NTIA broadband programs → search site:ntia.doc.gov
• NSF NCSES science indicators → search site:ncses.nsf.gov
• ED data express → search site:ed.gov/data
• College Scorecard → search site:collegescorecard.ed.gov
• OSHA enforcement data → search site:enforcedata.dol.gov
• MSHA mine enforcement → search site:arlweb.msha.gov
• BLS JOLTS, CES, CPS program pages → search site:bls.gov/jlt
• Treasury OFAC sanctions lists → search site:ofac.treasury.gov
• FinCEN advisories (public) → search site:fincen.gov/resources
• SEC EDGAR company search → search site:sec.gov/edgar
• CPSC recalls → search site:cpsc.gov/Recalls
• NHTSA recalls → search site:nhtsa.gov/recalls
• USDA FSIS recalls → search site:fsis.usda.gov/recalls
• FDA recalls → search site:fda.gov/safety/recalls-market-withdrawals-safety-alerts
• EPA recalls / advisories → search site:epa.gov/enforcement
• NOAA hurricane advisories → search site:nhc.noaa.gov
• USGS earthquake feeds → search site:earthquake.usgs.gov
• US drought monitor → search site:droughtmonitor.unl.edu
• USDA crop progress → search site:nass.usda.gov
• EIA weekly petroleum → search site:eia.gov/petroleum
• EIA electricity → search site:eia.gov/electricity
• FRED macro series → search site:fred.stlouisfed.org
• Census building permits → search site:census.gov/construction
• HUD point-in-time homelessness → search site:huduser.gov
• VA open data → search site:data.va.gov
• SSA actuarial publications → search site:ssa.gov/oact
• CMS rate filings → search site:cms.gov/marketplace
• IRS SOI tax stats → search site:irs.gov/statistics/soi-tax-stats
• Treasury fiscal service → search site:fiscal.treasury.gov
• BLS QCEW employer data → search site:bls.gov/cew
• OSHA injury illness rates → search site:bls.gov/iif
• BTS airline on-time → search site:transtats.bts.gov
• FAA ASIAS (public summaries) → search site:faa.gov/data
• TSA checkpoint travel numbers → search site:tsa.gov/travel/passenger-volumes
• CBP enforcement statistics → search site:cbp.gov/newsroom/stats
• USCIS processing times → search site:egov.uscis.gov/processing-times
• State human rights reports (public) → search site:state.gov/reports
• USAID data → search site:data.usaid.gov
• World Bank open data (cross-check U.S. claims) → search site:data.worldbank.org
• IMF data (cross-check macro claims) → search site:imf.org/en/data
• OECD data (cross-check policy comparisons) → search site:oecd.org
• UN population data → search site:population.un.org
• WHO health statistics → search site:who.int/data
• IEA energy statistics → search site:iea.org/data-statistics
Rules:
- If a Tier 1 search returns relevant results, those MUST appear as primary evidence.
- Aggregators, news outlets, and commentary (Tier 2–5) may be cited in addition, never instead.
- If a Tier 1 search returns no relevant results, note it explicitly in the caveats field and then proceed with lower-tier sources.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
STEP 2 — VERDICT SYNTHESIS
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
After gathering evidence, determine the verdict using this taxonomy:
- True: Accurate and supported by primary sources
- Mostly True: Accurate but missing important nuance or context
- Misleading: Technically accurate framing that implies something false
- Exaggerated: Directionally correct but substantially overstated
- False: Contradicted by credible evidence
- Unverifiable: Insufficient evidence to confirm or deny
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
STEP 3 — VERDICT VALIDATION (before returning output)
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Before returning your verdict, review your evidence list:
- If the claim falls into any domain listed in Step 1 AND your evidence contains zero Tier 1 government sources, you must either:
(a) perform an additional targeted Tier 1 search using the site: operators above, or
(b) add a caveat: "No Tier 1 primary source retrieved despite relevant domain; verdict rests on secondary analysis."
- A verdict backed only by aggregators or commentary on a quantitative government data claim is not acceptable without this caveat.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
CITATION DISCIPLINE — anti-hallucination requirement
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
The "web_sources" array MUST contain ONLY URLs that the web_search tool returned during this exact call. Do NOT fabricate URLs from training data, do NOT guess URL patterns even for sites you know well (bls.gov, cbp.gov, whitehouse.gov, etc.), and do NOT reconstruct URLs from a snippet description. A URL you cite must be one your tool retrieved verbatim.
If you want to reference a source you remember but did not retrieve, describe the source in `caveats` (e.g. "Per BLS CPI release Feb 2026, unverified at retrieval time") rather than emitting a URL for it.
If the web_search tool returned zero relevant URLs for this claim, return "web_sources": [] and either:
- set "confidence": "Low" with a verdict label other than Unverifiable if the claim is otherwise supported by retrieved evidence text, OR
- set "label": "Unverifiable" if no retrieved evidence supports a verdict.
Fabricated URLs are stripped automatically by a downstream ground-truth intersection. URLs you fabricate will NOT appear in the published report; they will appear in fabrication-rate telemetry against your model. Citation discipline is enforced.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
OUTPUT FORMAT
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Respond with ONLY raw JSON (no markdown fences, no preamble):
{
"label": "<verdict>",
"confidence": "High|Medium|Low",
"explanation": "<one paragraph citing specific sources and data points>",
"caveats": "<source-quality notes, or empty string if none>",
"web_sources": ["<url1>", "<url2>", ...]
}GitHub · Pipeline v0.2.0Beta