This guide addresses the decision-making you need when a scenario includes AI tools on either side of a security situation. It is designed to be used after you have the core risk workflow down, adding a layer of critical evaluation for AI output and AI-assisted processes.
- AI-assisted risk identification: Using AI tools to surface vulnerabilities or anomalies, while recognizing that AI output requires human verification before acting on it.
- AI-assisted mitigation: Applying AI recommendations for controls or responses, with awareness that AI may miss context-specific constraints.
- AI-assisted detection: Leveraging AI for anomaly detection or threat classification, while accounting for false positives and the limits of training data.
- AI collaboration traps: The guide specifically flags treating AI output as automatically correct as the most common reasoning error in AI-involved scenarios.
Given a scenario where an AI tool flags a potential intrusion, can you explain what additional steps a security analyst should take before responding, and why AI output alone is not sufficient justification for action?
| Scenario element | What AI can assist with | What still requires human judgment |
|---|
| Risk identification | Surfacing patterns and anomalies at scale | Determining whether a flagged item is actually a risk in this context |
| Mitigation selection | Generating candidate controls based on known vulnerabilities | Evaluating whether a control fits the organization's constraints |
| Attack detection | Classifying traffic or behavior against trained models | Investigating alerts and ruling out false positives |
| Documentation | Drafting risk statements from structured inputs | Verifying accuracy and adding scenario-specific nuance |