In January 2024, a finance employee at a multinational firm in Hong Kong authorized a wire transfer of $25 million. He had attended a video call before doing it. His CFO was on the call. His colleagues were present. The conversation was contextually accurate, and the instructions followed normal procedure.

Every person on that call was AI-generated.

Not a recording. Not a static image with a voice-over. A live, interactive, AI-generated video call: responsive to questions, fluent in context, visually convincing. From the employee's seat, it was indistinguishable from a real meeting. He had no reason to doubt what he was seeing, because everything he had ever been taught about verifying a request was satisfied.

Most security write-ups of this incident end the same way, with a call for vigilance. Train the employees. Sharpen the awareness program. Watch for the glitch in the video.

I want to be precise about why that is the wrong lesson.

Vigilance is not the answer to a system specifically engineered to pass the vigilance test.

The trust layer is no longer reliable

Underneath every business process is an invisible layer of infrastructure that nobody designed and everybody depends on: the set of signals that tell you the person speaking to you is real, the document in front of you is authentic, and the instruction you just received is legitimate. I call it the trust layer.

For thirty years, we treated those signals as reliable because forging them was expensive. A convincing fake voice took a studio. A convincing fake face on live video was science fiction. So we built our approval chains on recognition: I know that voice. I can see his face. The CFO was on the call.

That economic barrier is gone. The trust layer is not eroding, and it is not at risk. It is no longer reliable. The technology doing the attacking is improving faster than the technology doing the detecting, and there is no reason to expect that gap to close.

Find the approvals that rest on recognition

Here is the exercise I run with executive teams, and you can run it this week. Take your highest-value approval workflows and ask one question about each: what is the final control, the last thing that actually stops a fraudulent instruction?

In an uncomfortable number of cases, the honest answer is that an employee recognized the requester. The wire approval above a threshold, authorized on a video call. The vendor banking-detail change, confirmed by a phone call to a familiar voice. The emergency access grant, approved because the CISO asked for it in person, on screen. The payroll redirect, the M&A instruction, the certificate issuance.

Recognition is an authentication step nobody wrote down. It appears in no control matrix, so no audit ever tests it. And it is now the weakest link in every chain it appears in.

Rebuilding with the opposite assumption

Every authentication protocol, every communication channel, every approval chain built on the assumption that you can verify the identity of the person on the other side needs to be rebuilt with the opposite assumption: the person may not be a person.

The controls are not exotic. What is new is the discipline of applying them to conversations, not just logins.

  1. Out-of-band verification for money movement. Any instruction that moves funds or changes banking details gets confirmed on a channel the requester did not choose: a callback to a number from the system of record, never from the meeting invite or the email signature.
  2. Dual control on independent channels. Two approvers is not dual control if both sit on the same compromised call. Independence of channel matters more than the number of signatures.
  3. Challenge information for verbal instructions. Shared secrets for high-value requests are crude, and they work, precisely because they never transit the channel being spoofed.
  4. Treat live video as unverified input. Presence on a screen is no longer proof of anything. Write that sentence into your payment policy, verbatim, so nobody has to argue it under pressure at 4:55pm on a Friday.
  5. Rate-limit trust escalation. The Hong Kong call worked because a single interaction authorized a $25M movement. Structural limits on amounts, cooling periods, and escalation tiers mean one deceived human cannot be a complete failure mode.

Notice what is absent from that list: deepfake-detection tooling. Buy it if you like, but do not build your control structure on winning an arms race the defense is currently losing. Build it so that losing the detection race does not matter.

The next five years

We do not have regulatory frameworks for synthetic identity in business process, and most organizations do not have the technical infrastructure for verified communication. Both will come. In the meantime, the practical work is unglamorous: inventory the approvals that rest on recognition, add out-of-band verification, and rehearse the refusal. Make sure an employee who declines a CFO's video instruction is praised in front of the company, not punished.

The next five years of security will not be primarily about protecting data. They will be about verifying reality. The organizations that internalize that early will treat it the way they once treated the move to zero trust: not a product purchase, but a redesign of what they are willing to believe.