Kymata Labs
Get in touch
← Back to Research Hub
DEEPFAKESTRUST & PROVENANCEAI FRAUDDIGITAL EVIDENCE

Nothing You See Is Evidence Anymore

The Provenance Crisis: When Real and Fake Become Indistinguishable, Truth Becomes a Luxury Good

Kymata Labs Research·June 2, 2026·~12 min read

We have crossed a line and there is no walking back over it. Any image, any voice, any video can now be faked well enough to fool the experts. Spotting the fakes was never going to be the hard part. The hard part is that real evidence has lost its power, and a world that can no longer trust its own eyes does not become more careful. It becomes easier to lie to.

The moment anything canbe fake, two things happen at once. The guilty get a universal alibi, the words “it’s a deepfake.” And the innocent lose the ability to prove that anything happened at all. Truth stops being the default state of the world. It becomes something you have to buy.

The foundation

Humans can’t tell. The machines barely can.

Accuracy at spotting deepfake images

AI detectorson deepfake images (Univ. of Florida)
~97%
Human accuracyat chance, coin-flip odds (UF)
~50%
Peer-reviewed: detectors near 97% while the human eye performs at chance, dragged down by a built-in “truth bias.” In a 2,000-person iProov study, only 0.1% spotted every fake even when warned. Sources: University of Florida (PMC, peer-reviewed); iProov (vendor).

Published by Kymata Labs · Independent Research Institution.

Does this affect you?

You will get the phone call. The only question is when.

It’s your daughter’s voice, panicked, asking for money. It’s your CFO on a video call, on camera, telling you to wire the funds today. It’s a clip of a politician you trust saying something you can’t quite believe, and a clip of one you don’t, saying exactly what you expected. Every one of these is now cheap to manufacture and hard to disprove. You are not a target because you are important. You are a target because faking you costs almost nothing.

And it cuts the other way. The next time you have proof, a recording or a photo or a video of something that genuinely happened, you will discover that proof no longer ends the argument. “That could be AI” is a complete sentence now, and it works on anything. This paper is about the day evidence stopped settling things, which is the day we are already living in.

Seeing used to be believing. Not anymore.

“We always assumed the threat was a public fooled by fakes. The real wound is a public that has quietly stopped believing anything at all.”

Kymata Labs
The evidence

This already happened. With names, dates, and dollar figures.

None of what follows is a forecast. It is a record. A multinational lost tens of millions to a meeting full of ghosts. Fraud losses set records the same year. Lawyers learned a new move, and in study after study, ordinary people and trained experts alike failed to tell the difference. The convergence of all of it is the argument.

  • The meeting where everyone was a ghost

    In January 2024, a finance worker in the Hong Kong office of the UK engineering firm Arup wired roughly US$25.6 million (HK$200 million) on the instruction of his chief financial officer. He had his doubts. The request carried the smell of a phishing email, and he suspected as much, so he joined a video conference to be sure. On the call were the CFO and several colleagues he recognised. He saw them. He heard them. He wired the money. Every person on that call except him was an AI deepfake. The fraud surfaced only when he later checked with head office. Arup publicly confirmed the loss in May 2024. The lesson is the one that should keep every operator up at night: the live video call, the very thing we reach for to defeat our suspicions, was the weapon.

    CNN, Feb 2024; The Guardian, May 17, 2024. Arup’s Hong Kong office.

  • The price of impersonation, in public data

    The FTC’s 2024 figures, released in March 2025, are the macro view of the Arup story repeated a million times over. Consumers reported more than $12.5 billion in fraud losses, up 25% year over year. Imposter scams were the single most-reported category, at $2.95 billion.The share of people who reported a scam and actually lost money jumped from 27% to 38%. Voice cloning is one of the drivers the FTC names, and the agency launched a Voice Cloning Challenge and an impersonation rule in response. We don’t put a specific dollar figure on synthetic voice, because the FTC doesn’t, but the trend line under “imposter” is bending the way the technology predicts.

    U.S. Federal Trade Commission, March 10, 2025.

  • The liar’s dividend: predicted in 2019, collected ever since

    Two legal scholars, Robert Chesney and Danielle Citron, saw the shape of this in 2019. Their phrase, the “liar’s dividend,” names the windfall: once a public is primed to doubt, anyone caught on tape can dismiss a genuinerecording as a fake. The fakes don’t even have to be good. They only have to be possible. The mere existence of deepfakes hands every wrongdoer a pre-loaded defence, and it grows more plausible the more the public learns to distrust what it sees.

    Chesney & Citron, 107 California Law Review 1753 (2019).

  • The defence is already in the courtroom, and so far it is failing

    The dividend is no longer theoretical; it’s being claimed. In a 2023 wrongful-death suit over a fatal Tesla Autopilot crash, Tesla’s lawyers argued that Elon Musk’s recorded statements might be deepfakes. Judge Evette Pennypacker rejected the move as “deeply troubling,”warning it would let public figures escape their own words. In January 6 Capitol-riot trials, defendants argued the footage of them could be AI-fabricated. They were convicted. Read these as attempts rather than victories. The courts have held the line so far. But that line is now being tested in every venue, and not every venue has a federal judge and an expert like Hany Farid on hand.

    NPR, May 8, 2023 (Hany Farid, Rebecca Delfino).

  • Humans can’t tell. The machines barely can.

    Here is the foundation the whole crisis rests on. Peer-reviewed work from the University of Florida found AI detectors up to ~97% accurate on deepfake images while humans performed at chance, coin-flip odds, dragged down by a built-in “truth bias” that wants to believe what it sees. The vendor iProov ran 2,000 people in the US and UK: only 0.1% correctly identified every real-versus-fake item, even when warned to look for fakes, and people were 36% worse at catching deepfake video than images. The vendor Sumsub reports deepfakes rose fourfold from 2023 to 2024, now about 7% of all fraud attempts. Read the vendor figures as directional. Read the Florida result as the bottom line: the human eye is no longer an instrument of proof.

    University of Florida (PMC), peer-reviewed; iProov & Sumsub, vendor research.
The price of impersonation

Fraud set records, and imposters led

US consumer fraud losses, 2024 (FTC)

All reported fraudup 25% year over year
$12.5B
Imposter scamsthe single most-reported category
$2.95B
The share of scam reports that actually lost money jumped from 27% to 38%. Voice cloning is among the drivers the FTC names. Source: U.S. Federal Trade Commission (March 2025).
$25.6MWired to a video call of deepfakesArup’s Hong Kong office, January 2024. Every colleague on the call, including the “CFO,” was AI-generated.
0.1%Could spot every fake, even when warnedIn a 2,000-person US/UK study, only one in a thousand correctly identified every real-vs-deepfake item (iProov, vendor research).
How we got here

For a century, the recording was the witness.

The photograph, then the audio recording, then the video, built a quiet civilisation of proof. A photo could place you somewhere. A recording could convict you. Footage could exonerate you. Journalism, courts, insurance, history itself, all of it leaned on a shared assumption: a recording, absent obvious tampering, was a trace of something that really happened. Faking it convincingly was expensive enough to be rare, and rare enough to trust.

The rising tide

Deepfake fraud attempts quadrupled in a year

Relative volume of deepfake fraud attempts

2023deepfake fraud attempts (baseline)
baseline
2024now ~7% of all fraud attempts
4× / ~7%
Deepfakes rose roughly fourfold from 2023 to 2024 and now make up about 7% of all fraud attempts. Vendor research, read as directional. Source: Sumsub Identity Fraud Report (2024).

Generative AI dissolved every part of that assumption at once. The cost, the skill, the time, the need for a source, all of it gone. What used to take a studio now takes a sentence. And the decisive shift isn’t that good fakes exist; it’s that everyone knows they exist. Universal awareness is what converts a technical capability into a social collapse. You no longer need to be fooled by a fake to be harmed by it. It is enough to know that one was possible.

That is the engine of the liar’s dividend, and it runs in both directions. The guilty point at the real recording and cry “fake.” The innocent hold up their proof and hear “prove it’s not.” Both sentences work because the ground truth, the default trust we placed in our own senses, has been pulled out from under the entire system.

First we learned to fake anything. Then we learned to doubt everything.

Detection cannot save us, and chasing it is the trap. Every detector that gets good becomes the training target that teaches the next fake to slip past it. The arms race has no finish line, and even a perfect detector wouldn’t cure the deeper wound, because the doubt is already planted. The work that matters is not catching fakes after the fact. It is proving what’s real before the fact.On why detection is the wrong battle
The divide

Truth becomes a luxury good. Guess who can afford it.

If you can no longer trust a recording on its face, then trust has to be manufactured and attached in advance: cryptographically signed at capture, carried as content credentials, attested by hardware. That machinery is real, and it works. But it is expensive, unevenly available, and optional. Newsrooms, courts, governments, and corporations will buy it. The rest of the world will not.

So a divide opens. On one side stand the institutions and individuals wealthy enough to provision provenance, able to prove their own recordings are real and to verify everyone else’s. On the other stands everyone whose proof is just a phone video with no pedigree: the worker documenting abuse, the citizen filming the police, the family with a voicemail that meant something. Their evidence still exists. It just no longer counts. When truth costs money, the people who can’t pay are the first to lose the right to be believed.

What it means

The same crisis, read by three different readers.

Doubt is not destiny. The collapse of casual trust can be answered, though not with better fake-spotting. It is answered with verifiable origin. What that requires depends on who you are.

For individuals

Assume the channel can be faked. Verify the source.

  • Treat any urgent voice or video request for money or secrets as unproven until confirmed on a separate, trusted channel. That single step would have saved Arup’s clerk.
  • Agree a private code word with family for the “I’m in trouble, send money” call that is already in circulation.
  • Don’t let “it could be a deepfake” quietly excuse the powerful. Doubt is a tool of accountability, not a substitute for it.

For employers

A face on a screen is no longer authentication.

  • A live video call is not proof of identity, and Arup’s loss proves it. Require out-of-band confirmation for any high-value transfer, with no exception for the CFO.
  • Imposter fraud is the most-reported category and rising; budget for it as an operational risk, not an IT footnote.
  • Adopt content provenance for the recordings your business relies on, before a forged one relies on you.

For policymakers

Build the rails for proving real before the dividend hardens.

  • Fund and standardise provenance, the signing, content credentials, and attestation, as public infrastructure, so truth isn’t rationed by who can pay.
  • Courts have so far rejected the “it’s a deepfake” defence. Give judges and juries the authentication standards to keep rejecting it as the attempts multiply.
  • Protect the powerless side of the liar’s dividend: the people whose unsigned phone footage is the only record of what was done to them.
Questions worth asking

FAQ

It differs in kind, not in degree. Photoshop demanded skill, time, and a source image to manipulate, which made a convincing fake of a specific person doing a specific thing expensive and rare. Generative tools collapse all three barriers at once. Anyone can now produce video and voice of anyone, saying anything, in minutes, for almost nothing. When the cost of a perfect fake falls to zero, the scarcity that made real evidence trustworthy disappears along with it.

Partly, and only in the lab. The University of Florida work found detectors near 97% on deepfake images while humans performed at chance, so machines can help where the human eye fails. But detection is an arms race, and every detector trains the next generation of fakes to beat it. Accuracy on a curated test set is also a long way from accuracy on a phone video forwarded six times through compression. The detector is a tool rather than a cure. The deeper damage, the liar's dividend, survives even a perfect one, because by then the doubt is already planted.

It is the windfall the guilty collect simply because fakes now exist. Chesney and Citron named it in 2019: once the public knows any recording could be fabricated, a real recording can be waved away as fabricated. The liar profits from the very technology that threatens everyone else. Nobody has to prove a video is fake. They only have to make the audience unsure, and uncertainty is enough.

Not yet, and that is the encouraging part. In a wrongful-death suit over a fatal Tesla Autopilot crash, Tesla's lawyers floated that Elon Musk's recorded statements could be deepfakes; Judge Evette Pennypacker called the argument "deeply troubling" and rejected it. January 6 defendants who argued footage could be AI-fabricated were convicted anyway. So far the courts have refused the gambit. But these are attempts rather than anomalies, and the attempts will keep coming, in venues with far less expertise than a federal bench.

Provenance, not detection. Rather than asking whether something is fake after the fact, which is a losing game, we attach verifiable origin to real content at the moment of capture: cryptographic signing, content credentials, hardware-level attestation. The shift is from proving something is fake to proving something is real. It is harder, slower, and unevenly available, which is precisely the problem. Truth becomes something you have to provision in advance, and not everyone can.

The fakes were never the point. The doubt was.

We spent years worried that we’d be tricked by a perfect forgery. The deeper loss arrived quietly. It is not a world that believes the wrong things, but a world that can no longer be sure of the right ones. Real footage of real events now arrives pre-doubted. The answer is not to win the impossible race to spot every fake. It is to make truth provable again, and to make sure that proof isn’t something only the wealthy and the powerful can buy.

If anything can be faked, then truth has to be proven, by everyone, for everyone.

References

Sources

Every figure in this paper is drawn from the primary sources below. Where research is vendor-produced rather than peer-reviewed, or a courtroom argument is an attempt rather than a ruling, we have said so in the text.

The full paper

Read it as a PDF

The complete argument, the cross-domain evidence, and all primary sources, typeset as a white paper. Free, no signup.

↓ Download the PDF

Published by Kymata Labs · Independent Research Institution · kymatalabs.com

This paper is provided for free. Cite the underlying primary sources directly.