Blog/Complete guides

The Complete Guide to AI-Powered Cyberattacks

What criminals and state actors verifiably do with AI today, what is still marketing or speculation, and which of your defenses genuinely need to change.

CyberWatch AI2 November 2026 · 15 min read
A person sitting at a desk in a dim office in front of two computer monitors showing text

In February 2024, Microsoft and OpenAI published a joint account of what state-affiliated groups were actually doing with large language models. The list was deflating. Translating documents. Researching satellite communications protocols. Asking for help with a script. Drafting plausible emails. Microsoft's own summary noted that it had not identified significant attacks using the models it monitors.

A year later, Google's threat intelligence team published a similar review of how actors were using Gemini and reached a similar conclusion: mostly productivity, mostly research and code troubleshooting, with jailbreak attempts that recycled prompts from the public internet and mostly failed. Then in August 2025, Anthropic published a different kind of case: an actor who had used an agentic coding tool across most of an intrusion, from reconnaissance through credential harvesting to writing the extortion notes.

Those three reports, read together, describe the real situation better than any vendor keynote. AI is in attackers' hands, it is being used on live operations, and what it changes is cost and scale rather than the fundamental shape of an attack. This guide separates what is documented from what is projected, because the two require completely different responses. The first needs changes to your controls. The second needs you to stop reading and go fix your multi-factor authentication.

How to read any claim about AI and cybercrime

Three questions filter most of the noise.

Who observed it, and in what? Threat reporting that comes from the people operating the platform (Microsoft, Google, Anthropic, OpenAI) or from incident responders working real cases is evidence. A demonstration in a lab, a capability described in a research paper, and a product page describing what an attacker "could" do are not the same thing. All three are useful, and only the first tells you what is happening now.

Is the AI doing the attack, or writing about the attack? There is an enormous difference between a model that drafts a convincing message and a model that compromises a network. Most reported misuse is the first. Some recent reporting describes the second, and that is the part worth tracking closely.

Would the attack have worked anyway? A phishing email with perfect grammar still needs a victim without phishing-resistant authentication. A cloned voice still needs a payment process that approves transfers on a phone call. AI removed a signal that was already weak. It did not create the weakness it exploits.

The UK National Cyber Security Centre's assessment of the near-term threat makes the same point in official language: AI will almost certainly increase the volume and impact of attacks, the uplift is uneven across the attack chain, and it is concentrated in social engineering and reconnaissance. Those are precisely the stages where the work used to be manual and the quality used to be limited by the attacker's language skills and local knowledge.

What attackers verifiably do with AI today

Social engineering at scale, in any language

This is the clearest and largest change. Writing a convincing message in a language you do not speak, in the register of a specific industry, referencing the right job titles and the right internal systems, used to take real effort or a collaborator. Now it takes a prompt and some scraped context.

The practical effects on your inbox:

  • Language is no longer a filter. Organizations in markets that were previously protected by a less common language now see fluent, idiomatic approaches.
  • Personalization is cheap. The tailored approach that used to be reserved for a chief financial officer can now be produced for every member of a finance team. Our article on AI-written phishing covers what that looks like at the message level.
  • Volume and variation together. Signature-based email filtering and user-reported-template matching both work better against a thousand copies of one message than a thousand variants, and variants are now free.
  • Conversation, not a single email. Models make a sustained back-and-forth cheap, which suits approaches that build rapport over several days before asking for anything. The FBI's Internet Crime Complaint Center documented exactly this pattern in a campaign impersonating senior US officials: text and AI-generated voice messages used to establish a relationship first, then a move to another messaging platform, then the malicious link.

Europol's early assessment of large language models for law enforcement flagged the same priority areas: fraud and social engineering, disinformation, and assistance with code. Three years on, that ordering has held up.

Voice cloning and live impersonation

Cloning a recognizable voice now needs a short sample, and the sample is usually already public in a conference recording, a podcast, a voicemail greeting or a video posted by the organization's own marketing team. Video is harder in real time but no longer out of reach for a motivated operator.

The use that matters to organizations is not entertainment deepfakes. It is a call that carries authority into a process that was only ever protected by recognition: a payment release, a bank detail change, a password reset at the help desk, an approval outside normal hours. NSA, FBI and CISA published joint guidance on deepfake threats to organizations for this reason, and their recommendations are about verification and preparation rather than about teaching staff to spot artifacts. We cover the organizational version in detail in deepfake and voice-cloning fraud against organizations.

Reconnaissance and target selection

Reconnaissance is tedious: reading filings, mapping org charts, working out who approves what, correlating a LinkedIn change with a likely gap in coverage, summarizing a technology stack from job adverts. Models are good at exactly this kind of reading and summarizing, and the Microsoft and Google reports both list open-source research as a top observed use.

What this means practically is that the gap between a generic campaign and a targeted one has narrowed. Information you considered public but obscure, such as finance team structure, holiday cover arrangements, supplier names and the format of your internal ticket references, is now cheap to assemble into a usable pretext.

Code assistance and operational tooling

Attackers write software. They maintain loaders, panels, scrapers, credential stuffing tools and data exfiltration scripts, and most of that code is unremarkable. Model assistance helps there the same way it helps any developer: faster debugging, translation between languages, refactoring, understanding someone else's code.

Google's review described exactly this: scripting and development help, explaining unfamiliar technologies, and assistance with payload development, without a novel AI-specific technique appearing. Anthropic's reporting goes further and describes an actor developing and selling ransomware variants with model assistance, which is a meaningful data point about where the floor of technical skill now sits.

What has not changed is the detection surface. Code that encrypts files at scale, deletes shadow copies and tampers with security tooling behaves the same whether a person or a model wrote it. The CISA advisory on RansomHub is a useful reality check on what current ransomware intrusions actually look like: phishing, exploitation of known vulnerabilities, and password spraying against credentials from old dumps. Nothing in that list is an AI problem.

Fraud infrastructure and fake identity

A category that gets less attention than phishing and probably deserves more. Generative tools make the supporting cast of a fraud cheap: convincing company websites, product photography, invoices, identity documents of varying quality, employee headshots, reviews, and entire professional histories for personas used in recruitment and investment approaches.

Anthropic's report describes North Korean operators using a model to obtain and then perform fraudulent remote employment at Western companies, which is a fraud against your hiring process rather than your firewall. If your onboarding grants access based on documents and a video interview, that is now part of your attack surface.

Extortion, negotiation and pressure

Once an attacker has data, the remaining work is pressure: identifying who in the stolen files would be most damaged by publication, drafting notes to individuals, writing to regulators and journalists, and running negotiation in fluent business English. All of it is text generation, and all of it was previously a bottleneck for groups operating in a second language.

What is still speculative

Being specific about this matters, because budget spent on imaginary threats is budget not spent on the ones in the advisories.

Fully autonomous end-to-end attacks

Agentic tooling has moved quickly, and the Anthropic case shows a single operator using a coding agent across many stages of an intrusion. That is assisted, not autonomous: a human chose the targets, made the decisions, and handled the failures. A system that independently selects victims, adapts through a defended environment, and monetizes the result without supervision is not what current threat reporting describes. Treat announcements of one with the same skepticism you would apply to any unverified capability claim.

Mass discovery of novel zero-days

AI-assisted vulnerability discovery is real and improving, mostly in fuzzing, crash triage and memory-safety bugs in open-source code, and it is dual use: the same work is published by defenders and vendors. What has not appeared in threat reporting is an adversary running a discover-to-weaponize pipeline at a scale that changes your patching strategy. Your exposure today still comes overwhelmingly from known vulnerabilities in internet-facing systems that have had a patch available for months.

Undetectable polymorphic malware

The idea that a model rewrites malware on each execution to defeat detection is old, predates modern AI, and runs into the same wall it always did. Modern endpoint detection looks at behavior, process relationships, and the sequence of actions taken, not at file hashes alone. Changing the source does not change the need to gain persistence, escalate privileges, reach data and move it. Polymorphism raises the cost of signature-based scanning and leaves behavioral detection largely intact.

"AI malware" as a meaningful category

Be suspicious of the term. It usually means one of three ordinary things: code written with model assistance, malware that calls out to a model as a service, or a tool with an AI brand on the sales page. None of them is a new class of threat requiring a new class of product. Ask what the thing does on an endpoint, and the conversation becomes normal again.

The other half: attacks on AI systems

Everything above treats AI as the attacker's tool. The faster-growing risk for most organizations is the reverse, because you are probably deploying AI yourself. Once a model reads untrusted content or holds a credential, it becomes part of your attack surface.

NIST's adversarial machine learning taxonomy sorts this into evasion, poisoning, privacy and abuse attacks, and the OWASP Top 10 for LLM Applications puts prompt injection first because it is the one with no clean fix. If you are deploying anything that reads documents, emails or web pages on a user's behalf, start with prompt injection and LLM application security. If that system can also take actions through tools and APIs, the permission and auditability problems in autonomous AI agents with tool access matter more than anything in the threat reporting above.

What actually changes in your controls

Most of the useful response to AI-assisted attacks is not AI. It is the removal of controls that depended on a human noticing something.

Retire recognition as a control

Any control whose description contains "staff will notice" needs revisiting. Spelling, grammar, generic greetings and awkward phrasing are gone as signals. Voice familiarity is gone. Video presence is going. What survives is structural: does this request match a known process, did it arrive through the channel that process uses, and has it been confirmed through a second channel you chose rather than one the sender offered. Our guide to social engineering covers the underlying patterns, which have not changed at all.

Phishing-resistant authentication, everywhere it fits

The single highest-value change. Passkeys and hardware security keys do not care how good the fake login page was, because the credential is bound to the real domain and cannot be replayed elsewhere. Where you cannot deploy them, number matching and app-based approval are still a large improvement on SMS codes. Credentials in old breach dumps, sprayed against accounts without a second factor, remain one of the most common ways ransomware operators get in.

Processes that do not depend on who is on the call

  • Dual authorization for payments above a threshold and for any change to supplier bank details, with the second approver contacting the supplier on a number from your own records.
  • Out-of-band callback for credential resets and MFA re-enrollment, using the directory number, never a number supplied in the request.
  • A stated rule that nobody is punished for pausing an urgent request to verify it. Urgency is the mechanism; removing its social cost removes much of its power.

Shift detection left of the message

If content quality no longer distinguishes a real email from a fake one, lean on the things attackers still struggle to fake: domain age and registration, authentication results for the sending domain, first-time-sender context, display name mismatches against your directory, and anomalies in the reply-to and routing. Enforce DMARC on your own domains so your brand is harder to use against your suppliers. The detailed version is in our complete guide to phishing.

Train for the request, not the artifact

Awareness training built around spotting typos is now actively misleading. Useful training teaches people the shape of a request that should always trigger verification: a payment change, a credential or code, an urgent instruction from a senior person outside normal channels, a new supplier detail, a request to move the conversation to a personal number or another app. Simulations should reflect the current quality of attacks, which means well-written and contextual, not obviously fake.

Reduce what a single compromised account can reach

Better social engineering means more accounts will be compromised, so the cost of one compromise is the number that matters. Least privilege, short session lifetimes for sensitive applications, mailbox forwarding restrictions, limits on application consent, and logging that lets you reconstruct what an account did are what turn an incident into a contained event.

Your own AI use is part of the picture

Two of the AI risks most likely to cost you something are not attacks at all. The first is data placed into tools nobody reviewed. When a model is blocked at the network edge, the work does not stop; it moves to a personal phone and a personal account, where you have no log, no data processing agreement and no way to answer a customer asking where their information went. A single sanctioned tool with terms you have read, configured so that inputs are not used for training, is a better security outcome than a ban that half the organization routes around.

The second is AI output accepted without review. Generated code pulled into a product, a generated policy document that contradicts your actual controls, a generated supplier assessment nobody checked: these create real exposure without any adversary involved. The fix is ordinary. Treat model output as a draft from an unverified source, require the same review you would require from a new contractor, and record who approved it.

Write this into a short policy rather than a long one. Which tools are approved, what categories of data must never be pasted into any of them (customer personal data, credentials, unreleased financials, anything under a confidentiality obligation), what must be reviewed before use, and who to ask when the answer is unclear. A page that people read beats a chapter nobody opens.

What AI changes for defenders

The symmetry argument, that attackers and defenders both get better so nothing changes, is too neat. The gains land in different places.

Defenders gain where the bottleneck is volume: summarizing an alert's context before a human looks at it, drafting and tuning detection logic, translating a query between tools, triaging a pile of reported emails, explaining an unfamiliar binary's behavior, reviewing code for the mistake you keep making. Those are real reductions in time-to-decision for small teams, which is where the shortage is worst.

Defenders do not gain much where the bottleneck is authority or process. A model cannot decide to take a production system offline, cannot make your supplier confirm a bank change, and cannot create the log that was never enabled. Teams that bought an AI analyst while still lacking centralized logging and tested backups bought the wrong thing in the wrong order.

The practical advice is unglamorous: use AI to compress the work your team already does and understands, keep a human accountable for every consequential decision, and do not let an AI purchase displace the control it is supposed to help you operate.

A realistic threat model for a normal organization

If you are a business of ten to a thousand people rather than a government target, the AI-related risk that is likely to reach you, in rough order of probability:

  1. A well-written, contextual phishing or invoice fraud attempt that no longer has the tells your staff were taught to look for.
  2. A voice call or voice note impersonating a colleague, executive or supplier, aimed at a payment, a credential or an approval.
  3. Account takeover using credentials from somewhere else, in which AI played no part at all and never needed to.
  4. A fraudulent job applicant or supplier with a generated history, aimed at access rather than at a single payment.
  5. A problem in an AI feature you deployed: an assistant that can be steered by content it reads, or an agent with a token broader than its job.
  6. Data leaking into an unsanctioned tool because the sanctioned one was blocked.

Notice that items one to three are addressed by controls that existed before any of this, and that item five only exists because of a choice you made. That is the honest summary of AI's effect on most threat models: it raised the quality of the inputs and added one new surface of your own making.

Not sure about a message or a link? Paste the message or link into CyberWatch AI Scan for a free check.

How to track this without being sold to

A short reading list beats a product demonstration. The platform providers publish threat reporting on misuse of their own services, which is the closest thing to primary evidence. National agencies publish assessments that are deliberately careful about confidence levels, and the NCSC near-term threat assessment is a good model for how to phrase uncertainty. For risks in systems you build, the OWASP list for LLM applications and the NIST adversarial machine learning taxonomy are the reference points practitioners actually use.

When a new claim appears, apply the three questions from the start of this guide, then ask one more: what control would have stopped this regardless of the AI? If there is a clear answer and you do not have that control, you have found your next piece of work. If there is no clear answer, you are probably looking at a demonstration rather than a threat.

The short version

AI has made attackers more productive at the human-facing parts of their work: writing, researching, impersonating and pressuring. It has not yet rewritten the technical core of intrusions, which still runs on stolen credentials, unpatched internet-facing systems and someone clicking something. The defenses that matter most are the ones that never relied on a person detecting a fake: authentication that cannot be replayed, money and access processes with a second channel built in, and enough logging to answer what happened. Everything else in the AI security conversation is either a problem you are choosing to adopt or a problem you do not have yet.

Frequently asked questions

Are AI-powered cyberattacks real or just marketing?

Both. Published threat reporting from Microsoft, Google and Anthropic documents real criminal and state use of large language models, mostly for research, translation, phishing text, scripting help and fraudulent personas. The fully autonomous AI attack tool sold in vendor presentations is not what shows up in those reports.

What do attackers actually use AI for today?

Writing and translating social engineering messages, researching targets, generating fake personas and documents, cloning voices, debugging and refactoring their own tooling, and automating parts of an intrusion such as reconnaissance and extortion note writing. These are efficiency gains on existing techniques.

Can AI write malware that antivirus cannot detect?

Models can produce working malicious code and have been used to help build ransomware and loaders. That does not make the output undetectable. Behavioral detection and endpoint controls key on what code does rather than on how it was written, and recycled AI-generated code is often easier to detect, not harder.

Does AI make phishing impossible to spot?

It removes the easiest signals. Bad grammar, odd phrasing and generic greetings were never reliable, and they are now close to useless. Context, the request being made, and the channel it arrives through are what still work, which is why verification processes matter more than individual judgment.

Should we block AI tools inside the organization?

Blocking rarely holds and pushes use onto personal accounts where you have no visibility. A sanctioned tool with data handling you have reviewed, a short usage policy and logging gives you more control than a ban.

What is the difference between attacking with AI and attacking AI?

Attacking with AI means using a model as a tool to carry out a conventional attack more cheaply. Attacking AI means exploiting an AI system itself, through prompt injection, training data poisoning or model extraction. The second category is the newer risk if you deploy AI features.

Does AI help defenders as much as attackers?

In different places. Defenders gain on triage, log summarization, detection engineering and code review, where volume is the bottleneck. Attackers gain on content generation and research, where quality and language used to be the bottleneck. Neither side gets a decisive advantage from the current generation of tools.

What single control helps most against AI-assisted attacks?

Phishing-resistant multi-factor authentication, followed by out-of-band verification for money movement and credential resets. Both work regardless of how convincing the message or the voice was, because they do not depend on anyone recognizing a fake.

Sources

  1. The near-term impact of AI on the cyber threat, UK National Cyber Security Centre
  2. Adversarial Misuse of Generative AI, Google Threat Intelligence Group
  3. Staying ahead of threat actors in the age of AI, Microsoft Security Blog
  4. Detecting and countering misuse of AI: August 2025, Anthropic
  5. ChatGPT - the impact of Large Language Models on Law Enforcement, Europol
  6. Senior US Officials Impersonated in Malicious Messaging Campaign, FBI Internet Crime Complaint Center
  7. NSA, FBI, and CISA Release Cybersecurity Information Sheet on Deepfake Threats, CISA
  8. OWASP Top 10 for LLM Applications 2025, OWASP Gen AI Security Project
  9. AI 100-2 E2025, Adversarial Machine Learning: A Taxonomy and Terminology of Attacks and Mitigations, NIST
  10. #StopRansomware: RansomHub Ransomware, CISA
Share this article
LinkedIn WhatsApp X
For organizations

Your people get these messages at work too.

CyberWatch AI sends your staff realistic practice attacks, trains the gaps it finds, and shows management exactly where the organization stands.