The Phishing Email That Knows Your Boss's Writing Style — AI Did That

The email landed in a finance manager's inbox on a Tuesday morning. It came from what appeared to be her CFO's address, referenced a specific acquisition deal she'd been briefed on the previous week, used the exact sign-off phrase her CFO always used, and asked her to initiate a wire transfer before end of business. The tone was right. The context was right. The urgency was calibrated — pushy, but not weirdly so.

It was a complete fabrication. The CFO had sent no such email. The acquisition reference had been scraped from a LinkedIn post and a press release. The writing style had been cloned from publicly available communications. The whole thing had been generated, personalized, and deployed at scale by a threat actor using tools that, until about 18 months ago, simply didn't exist in this form.

Welcome to AI-assisted phishing. It's not a future threat. It's the current one.

From Spray-and-Pray to Surgical Strikes

For most of its history, phishing was a volume game. Attackers sent millions of generic emails — fake bank alerts, IRS notices, shipping confirmations — and counted on a small percentage of recipients being distracted, trusting, or just unlucky enough to click. The grammar was often broken. The logos were slightly off. Security awareness training taught people to look for exactly those tells.

Generative AI broke that model entirely.

Large language models can produce fluent, contextually appropriate, culturally calibrated text at zero marginal cost. They don't get tired. They don't make the kind of errors that trained eyes used to catch. And when you feed them data — scraped LinkedIn profiles, public filings, social media posts, leaked email archives, company websites — they can produce communications that don't just sound human. They sound like a specific human.

Researchers at several security firms, including documented work published by IBM X-Force and Abnormal Security, have demonstrated that LLM-generated spear-phishing messages achieve significantly higher click rates than human-written equivalents in controlled testing environments. The gap isn't marginal. In some tests, AI-generated messages outperformed human-crafted ones by 30% or more.

The Data Harvesting Pipeline

Understanding why these attacks are so effective means understanding the data pipeline that feeds them.

Before an AI-assisted phishing campaign launches, attackers run reconnaissance. This isn't new — spear-phishing has always required research. What's new is the automation and scale. Open-source intelligence tools now crawl LinkedIn, Twitter/X, GitHub, corporate blogs, press releases, regulatory filings, and court documents to build detailed profiles of targets and their organizations.

That profile feeds into the generation layer. The LLM isn't just writing a convincing email — it's writing a convincing email that references the target's actual job title, their actual recent projects, their actual organizational structure, and the actual communication norms of their workplace. It can mimic the sentence length patterns of a specific executive. It can replicate the informal shorthand a team uses internally if any of that has ever appeared in public-facing communications.

Some of the more sophisticated campaigns observed by threat intelligence teams have incorporated real-time context — referencing news events from that morning, mentioning a conference the target was publicly attending, or acknowledging a product launch that happened the previous day. That kind of timeliness used to require a human operator with time to burn. Now it's automated.

Defeating the Filters

Traditional email security tools look for signatures: known malicious domains, suspicious attachment types, URL patterns that match threat intelligence feeds, and statistical anomalies in message content. These approaches work reasonably well against known, static threats.

AI-generated phishing is dynamic by design.

Because the content is generated fresh for each campaign — sometimes for each individual target — there are no signatures to match. The sending infrastructure is often clean, using compromised legitimate accounts or newly registered domains with no prior reputation. The URLs in the messages frequently point to legitimate cloud services like Google Docs, OneDrive, or Dropbox before redirecting, which means link-reputation checks come back clean.

Some researchers have documented adversarial prompt engineering being used specifically to evade AI-based email classifiers. Attackers are probing commercial email security products, identifying the patterns they flag, and instructing their generation models to avoid those patterns. It's a feedback loop — the defenders' AI becomes a training signal for the attackers' AI.

Voice and Video Enter the Chat

Text is only part of the story now. The same generative capabilities that produce convincing written phishing are being extended into voice and video.

Vishing — voice phishing — has gone through its own AI-assisted evolution. Voice cloning tools, several of which are commercially available and require only a few seconds of sample audio, can produce convincing audio impersonations of executives. There have been documented cases in the US of finance employees being called by what they believed was their CEO's voice, authorizing transactions that turned out to be fraud. The FBI has issued explicit warnings about AI-generated voice fraud targeting businesses.

Deepfake video is still more resource-intensive, but the cost curve is dropping fast. Security teams that haven't thought about what a video verification workflow looks like for high-value transaction approvals are behind the curve.

What Defenders Are Working With

The defender side of this equation isn't standing still, but it's honest to say the advantage currently sits with the attackers.

Behavioral analytics — looking at communication patterns rather than content — is one of the more promising approaches. If an email claiming to be from your CFO is being sent from an IP in a geography your CFO has never emailed from, at a time they never email, that's a signal worth flagging regardless of how convincing the text is. Tools from vendors like Abnormal Security, Darktrace, and Microsoft Defender for Office 365 are investing heavily in this space.

Out-of-band verification remains the most reliable control for high-stakes requests. If someone emails asking for a wire transfer, a vendor change, or sensitive data access, call them back on a known number. Not the number in the email. This is basic, it's been basic for years, and it still gets skipped constantly under time pressure.

Security awareness training needs to evolve beyond "look for bad grammar." That heuristic is dead. Training now needs to focus on process adherence — the idea that certain categories of request require certain verification steps regardless of how legitimate the communication looks. The email looking right is no longer sufficient grounds for acting on it.

The Uncomfortable Truth

The uncomfortable reality is that AI-powered social engineering attacks are going to get better faster than most organizations can adapt their defenses. The tooling is cheap, accessible, and improving continuously. The attack surface — human judgment under time pressure — doesn't change.

The organizations that will weather this period best aren't necessarily the ones with the most sophisticated technical controls. They're the ones that have built verification cultures, where employees feel empowered to slow down and confirm before acting, even when the email looks completely legitimate.

Because in 2024, looking completely legitimate is the easiest thing an attacker can do.