Prompt injection is an AI-based attack where an adversary crafts a carefully worded input to trick a large language model (LLM) into ignoring its safety rules, revealing hidden data, or producing harmful output. It maps to AP Cybersecurity Topic 1.4.
Prompt injection is when an attacker feeds a large language model (LLM) a sneaky input designed to make it do something it wasn't supposed to do. Think of it like social engineering, except the target is an AI instead of a person. Instead of tricking an employee into handing over a password, you trick the model into ignoring its built-in instructions.
This falls under AI-based cybersecurity attacks in Topic 1.4. EK 1.4.A.2 covers how adversaries use generative AI and LLMs as attack tools, and prompt injection is the technique that turns the model itself into a weapon. A classic example: an attacker types a message asking the LLM to reveal the training data it was built on, or to bypass the filters that normally block harmful answers. The model's job is to be helpful, and the attacker exploits that helpfulness by phrasing the request in a way that slips past its guardrails.
Prompt injection lives in Unit 1: Introduction to Security, under Topic 1.4 AI-Based Cybersecurity Attacks. It supports learning objective AP Cybersecurity 1.4.A, which asks you to explain how adversaries use AI-powered tools to augment cyberattacks. It also ties into AP Cybersecurity 1.4.B, because part of defending against AI threats (EK 1.4.B.3) is knowing what NOT to feed these tools in the first place. The whole point of this corner of the course is that AI is now a two-way street: defenders use it, but so do attackers, and the model itself can become the vulnerability.
Keep studying AP Cybersecurity Unit 1
Visual cheatsheet
view galleryLarge Language Models (LLMs) (Unit 1)
Prompt injection only makes sense once you understand what an LLM is. The LLM is the system being attacked, and prompt injection is the input that exploits how it follows instructions. No LLM, no prompt to inject.
Generative AI Attacks (Unit 1)
Prompt injection is one specific flavor of the broader generative AI attack category in EK 1.4.A.2. Where generative AI attacks describe the whole toolkit, prompt injection is the precise technique of manipulating the model through crafted text.
Training Data Poisoning (Unit 1)
Both attack the AI itself, but at different stages. Poisoning corrupts the model BEFORE it's trained by feeding it bad data; prompt injection manipulates the model AFTER it's deployed by feeding it a bad request.
Social Engineering and Phishing (Unit 1)
Prompt injection is basically social engineering aimed at a machine. The same psychology of crafting a convincing, manipulative message applies, except you're persuading an algorithm instead of a human to drop its defenses.
Expect prompt injection in multiple-choice questions that describe an attack scenario and ask you to name the technique. One practice stem describes an attacker crafting a carefully designed input to an LLM to make it reveal the training data it was built on, and the correct answer is prompt injection. Your job is to read the scenario, spot that the attacker is manipulating the model through its input, and pick the right term over close lookalikes like training data poisoning. No released FRQ has used this term verbatim, but it fits the AI-attack reasoning in Topic 1.4, so be ready to explain in your own words how an adversary uses an LLM offensively.
Timing is the giveaway. Training data poisoning corrupts the model during the training phase by feeding it false or malicious data, like flooding the web with fake claims so the model learns wrong facts. Prompt injection happens after deployment, attacking a finished model through the input you type. One poisons the recipe; the other tricks the chef after the dish is already made.
Prompt injection is an attack that uses a carefully crafted input to make an LLM ignore its rules, leak hidden data, or produce harmful output.
It maps to Topic 1.4 and supports learning objective AP Cybersecurity 1.4.A on how adversaries use AI-powered tools to augment attacks.
Think of prompt injection as social engineering against a machine: you persuade the model to drop its guardrails instead of persuading a person.
Don't confuse it with training data poisoning, which corrupts the model BEFORE training, while prompt injection manipulates a model AFTER it's deployed.
A key defense (EK 1.4.B.3) is never entering personal or sensitive data into AI tools, since that data can be exposed through injection attacks.
Prompt injection is an AI-based attack where an adversary feeds a large language model a specially worded input to trick it into ignoring its safety rules, revealing training data, or producing harmful content. It's covered in Unit 1, Topic 1.4.
No. Prompt injection attacks a finished, deployed model through the input you type, while training data poisoning corrupts the model earlier by feeding it bad data during training. Both target the AI, but at different stages.
Phishing targets a human with a deceptive message to steal credentials, while prompt injection targets an LLM with a deceptive input to override its behavior. Prompt injection is essentially social engineering aimed at a machine instead of a person.
Yes. It can appear in multiple-choice questions that describe an attacker crafting an input to manipulate an LLM, and you'd be asked to identify the technique. It falls under learning objective AP Cybersecurity 1.4.A.
A core defense from EK 1.4.B.3 is never entering personal or sensitive data into AI-powered tools, since anything you feed an LLM could potentially be extracted by an attacker through a clever injection.
Connect this key term to the AP exam workflow: review the course, practice questions, and check related study tools.