1.4 AI-Based Cybersecurity Attacks

AI tools have made cyberattacks faster, cheaper, and way more convincing than they used to be. The same tech that lets you generate a song in someone's voice or write an essay in seconds also lets attackers clone your boss's voice, write flawless phishing emails, or scan your social media to build a profile of you. Understanding how adversaries weaponize AI, and how regular people can defend against it, is a huge piece of modern cybersecurity.

How Adversaries Use AI to Power Cyberattacks

AI doesn't create totally new categories of attacks. Instead, it supercharges old ones. Phishing, impersonation, reconnaissance, and malware writing have all existed for decades. What's changed is the scale, quality, and speed at which attackers can pull them off.

more resources to help you study

practice questions

Voice and Video Deepfakes for Impersonation

Adversaries can take voice and image samples of a real person (often pulled from social media, podcasts, or YouTube) and feed them into AI tools to build a digital avatar. With just a few seconds of audio, a tool can clone someone's voice well enough to fool friends, coworkers, or even family.

Here's how this plays out in real attacks:

• An attacker calls an employee using a cloned version of the CEO's voice and demands an urgent wire transfer. There have been real incidents where companies lost millions this way.
• A scammer calls a grandparent pretending to be their grandchild ("I'm in trouble, please send money"), using a voice clone built from TikTok clips.
• On a video call, a deepfake avatar of a manager asks an employee to share login credentials or sensitive files.

This is becoming an even bigger deal as more organizations adopt voice-based authentication ("your voice is your password"). If an attacker can clone your voice, they can potentially get into your accounts.

AI-Generated Phishing Messages

For years, one of the easiest ways to spot a phishing email was bad grammar. Many phishing messages were written by attackers who weren't native speakers of the target's language, so awkward phrasing was a giveaway.

That tell is basically gone now. Adversaries use large language models (LLMs), which are AI systems trained on huge amounts of text, to write phishing messages that sound completely natural in any language. An attacker in one country can send a perfectly written email in English, Mandarin, Spanish, or German without speaking a word of it.

A modern AI-generated phishing email might:

• Match the tone and writing style of your actual coworker
• Reference real projects pulled from a company's public website
• Be personalized to you based on your LinkedIn profile
• Include zero grammar mistakes

The takeaway: you can no longer rely on "weird wording" as your main phishing filter.

Prompt-Based Extraction from LLMs

LLMs are trained on massive datasets, and they also remember (sometimes) what users type into them. Adversaries can craft clever prompts designed to pull sensitive information back out.

Two main sources of leaked info:

1. Training data. If an LLM was trained on data that included private emails, leaked documents, or sensitive code, an attacker might trick the model into repeating it.
2. User input. If employees paste confidential data into a chatbot (like internal source code or customer records), and that tool uses inputs for further training, an adversary could potentially extract that data later through targeted prompts.

This is why a lot of companies now ban employees from pasting work info into public AI tools.

Poisoning LLM Training Data

LLMs learn from the open internet. That includes websites, forums, articles, and basically anything public. Adversaries can exploit this by publishing or modifying websites to contain false information, hoping that data ends up in future training sets.

If enough fake content shows up across enough sites, the model may start repeating those lies as if they were facts. This is called data poisoning. The scary part is that the false output sounds confident and authoritative, so users may trust it. Imagine an attacker spreading fake "official" instructions for resetting a bank password, and an AI assistant later recommending those steps to real users.

AI-Powered Reconnaissance

Before an attack, adversaries gather information about their target. This phase is called reconnaissance. AI tools make it way faster.

An attacker can use AI to:

• Scrape your LinkedIn, Instagram, X, and TikTok in seconds
• Pull names of your family members, your school, your job, your pets
• Identify your coworkers and your company's tech stack
• Cross-reference public records and data breach dumps

All of this gets combined into a detailed profile that's used to craft a personalized attack. The more an attacker knows about you, the more convincing their phishing message or impersonation attempt becomes. That's why oversharing online is a real security risk.

AI-Enhanced Coding Tools for Malware

AI coding assistants help developers write, debug, and review code. Adversaries use the same tools for the opposite purpose:

• Writing new malware quickly, even if the attacker isn't a strong coder
• Modifying existing legitimate code to add malicious behavior
• Scanning massive codebases to find vulnerabilities (bugs that can be exploited)

A vulnerability hunt that used to take a skilled hacker weeks can now happen in hours. This lowers the barrier to entry for cybercrime, meaning more people can launch sophisticated attacks than ever before.

How to Protect Against AI-Augmented Attacks

The defenses below aren't fancy or expensive. Most are habits you can start using today.

Establish Shared Secrets with People You Trust

A shared secret is a word or phrase that only you and one other person know. If you ever get a strange call from "your mom" asking you to wire money, you can ask for the shared secret. A voice clone built from public videos won't know the answer.

This is one of the simplest and most effective defenses against voice deepfake scams, especially for families. Pick a random word together (something not on social media) and agree to use it in high-stakes situations.

For example: your family decides the word is "pineapple." If anyone calls in a panic claiming to need money, the real family member can confirm by saying "pineapple." An impersonator can't.

Enable Multifactor Authentication (MFA)

Multifactor authentication (MFA) requires more than one type of proof to log in. The typical categories are:

• Something you know (password, PIN)
• Something you have (phone, security key)
• Something you are (fingerprint, face, voice)

If a system only uses voice authentication and an attacker clones your voice, they're in. But if logging in also requires a code from your phone or a hardware key, the voice clone alone isn't enough. MFA blocks a huge percentage of account takeovers, including ones that involve AI impersonation.

Don't Feed Sensitive Data to AI Tools

Treat any AI chatbot or assistant like a public forum. Some tools use user input for ongoing training, which means whatever you paste in could theoretically be extracted later by someone else.

Things you should never paste into a public AI tool:

• Passwords or API keys
• Customer or patient data
• Internal company documents or source code
• Social Security numbers or financial info
• Private messages or personal details about others

If your school or workplace offers an enterprise version of an AI tool with privacy guarantees, that's safer. But default consumer chatbots? Assume anything you type might be stored.

Verify AI Output with Trusted Sources

LLMs sound confident even when they're wrong. They can hallucinate facts, repeat poisoned training data, or just be outdated. Always double check important information using reputable, stable, non-AI-based sources like:

• Official government or organization websites
• Established news outlets
• Peer-reviewed publications
• Documentation from the actual software or service you're using

If an AI tool tells you the steps to reset your bank password, go to your bank's actual website to confirm. If it cites a study, look up the study yourself. Treat AI output as a starting point, not a final answer.

Putting It All Together

AI hasn't invented brand new attacks. It's made the existing playbook (phishing, impersonation, recon, malware) cheaper, faster, and more convincing. Voice clones bypass the "I'd recognize their voice anywhere" instinct. LLMs write phishing that reads like a native speaker. AI scrapers build a profile of you in minutes. And AI coding tools help even beginner attackers write working malware.

The defenses are mostly about adding friction and verification. Shared secrets stop voice clone scams. MFA stops single-factor breaches. Keeping sensitive data out of public AI tools stops leaks. And verifying AI output stops you from acting on false or poisoned information. None of these are perfect alone, but layered together, they make you a much harder target.