Cybersecurity

Protecting business from AI phishing and deepfakes: how to recognise fake voices, videos and emails created by artificial intelligence and prevent financial fraud

Published: 4 min read
A face that is half real and half digital mesh, with a sound wave — a deepfake
A deepfake combines a synthesised face and a cloned voice to impersonate an executive.

In early 2024, a finance employee at the Hong Kong office of the global engineering firm Arup joined a video conference with the “chief financial officer” and several “colleagues”. All of them turned out to be deepfakes generated by artificial intelligence. Following the call, the employee made 15 transfers totalling about 25 million US dollars. The case has become a symbol of a new era of fraud: you can no longer trust even what you see and hear in real time.

Generative AI has made social engineering attacks cheap, scalable and convincing. A few minutes of audio from a webinar or interview is enough to clone a voice, and language models write flawless phishing emails in any language. Let’s look at how to recognise a fake and, more importantly, how to build processes that stop a single call from wiping out a company’s budget.

The three faces of AI fraud

1. Voice deepfakes (vishing)

An attacker calls an accountant using the voice of the owner or CEO and asks them to urgently pay an invoice or change a supplier’s bank details. As early as 2019, a UK energy company lost around 220,000 euros after a call in which fraudsters imitated the voice of the parent company’s chief executive. Since then, the technology has become far more accessible.

2. Video deepfakes

Fake video calls are used to “confirm” decisions, run fake job interviews and even pass online identity verification. Attackers often blame a poor connection to explain visual artefacts.

3. AI phishing and business email compromise (BEC)

Language models analyse public information about a company (LinkedIn, news, press releases) and produce personalised emails that read like genuine internal correspondence. The classic advice to “look for grammar mistakes” no longer works.

How to recognise a fake

  • An unusual channel or urgency. “I can’t talk, just pay it by the end of the day” or “this is a confidential deal, don’t tell anyone”: pressure through time and secrecy is the main sign of fraud.
  • Changed payment details. Any request to change a supplier’s account should automatically trigger an additional check.
  • Video artefacts. Unnatural blinking, blurred edges around hair and ears, lips out of sync with audio, odd lighting, a face that “drifts” when the head turns.
  • Voice artefacts. A flat intonation with no pauses or breathing, metallic undertones, delays before answering an unexpected question.
  • Email domain and headers. Look-alike domains (company-ua.com instead of company.ua), or a reply-to address that differs from the “From” field.
A phishing email with a hook and a smartphone making a callback to verify the request
A callback to a known number breaks most fraud scenarios.

Processes that stop fraudsters

The technical signs of a fake become less noticeable every year, so the main defence is not a “sharp eye” but processes that do not depend on how convincing the call was.

  1. Callback verification. Any financial request received by phone, messenger or video call is confirmed by calling back on a number from the internal directory, never one taken from the email or message.
  2. The four-eyes principle. Payments above a set limit and changes to bank details require approval from two independent employees.
  3. Code words. Executives and the finance team should agree on secret phrases to confirm non-standard instructions.
  4. Technical email protection. Configure SPF, DKIM and DMARC with a reject policy, use attachment and link filtering, and tag external emails.
  5. Regular simulations. Training phishing campaigns and exercises with deepfake scenarios build a “stop and verify” reflex.
  6. Minimise public data. Limit the amount of publicly available voice and video recordings of executives, and do not publish details of internal payment approval processes.

What to do if an attack has already happened

Time is critical. Contact your bank immediately and ask it to block or recall the transfer: the chances of recovering funds are highest in the first few hours. Preserve all evidence, including call recordings, emails with full headers and phone numbers. Report the incident to the cyber police and, if necessary, to CERT-UA. Then run a post-incident review to find out which link in the process failed.

Key takeaway

A deepfake attacks trust, not technology. So the best defence is a culture in which verifying a request is seen not as distrust of a manager, but as a standard professional procedure.