3.5 Detecting Network Attacks

Networks are noisy places. Every login, every webpage request, every packet bouncing between devices gets recorded somewhere, and adversaries hide their attacks inside all that normal chatter. Detecting attacks means knowing what tools to use, how to tune them, and what patterns to look for in mountains of log data. This topic walks through the automated systems that do the heavy lifting, how AI fits in, how to pick the right detection method, and what specific attacks look like when you spot them in the logs.

Automated Tools for Detecting Network Attacks

Modern networks produce way too much data for humans to review by hand. That's where automated detection tools come in. They pull data from switches, routers, servers, firewalls, and user computers, then analyze it for signs of trouble. Most of that data lives in log files, which are time-stamped records of everything happening on a device or network.

There are three main tools you need to know.

Network Intrusion Detection System (NIDS)

A network intrusion detection system (NIDS) is an automated tool that watches network traffic and figures out if something malicious is happening. When it spots an attack, it generates an alert so the security team can investigate. The key word here is detection: a NIDS notices the problem but doesn't actually stop it.

Network Intrusion Prevention System (NIPS)

A network intrusion prevention system (NIPS) does everything a NIDS does, plus it can take action. If a NIPS sees an attack, it can:

• Close ports
• Block specific IP or MAC addresses
• Reject specific protocols

So NIDS = alarm system, NIPS = alarm system that can also lock the door. The tradeoff is that a NIPS can accidentally block legitimate traffic if it misidentifies something.

Security Information and Event Management (SIEM)

A security information and event management (SIEM) system is the big-picture tool. It pulls in data from a bunch of different sources at once: firewalls, NIDS, NIPS, device logs, application logs, and more. By looking at all of these together, a SIEM can spot patterns that no single tool would catch on its own.

When a SIEM raises an alert, security analysts investigate to decide if it's a real threat. They then follow standard operating procedures (SOPs) to either resolve the issue or escalate it to someone with more authority.

Using AI to Enhance Threat Detection

Here's the scale problem: a medium-sized organization generates millions, sometimes tens of millions, of log entries per day. Even a huge team of analysts can't read through all that. AI fills the gap.

Threat detection teams build AI algorithms that learn to classify traffic patterns as either malicious or normal. Instead of giving a yes-or-no answer, these models use probabilistic calculations. They report a percentage representing how likely something is to be malicious. A connection might score 12% suspicious (probably fine) or 94% suspicious (probably an attack).

Setting Thresholds

Organizations decide their own threshold for when a probability score triggers an alert. This is a balancing act:

• Threshold too high (say, 95%): only the most obvious attacks get flagged, so real attacks may slip through undetected.
• Threshold too low (say, 40%): the system flags tons of normal activity, and the security team gets buried in false alerts.

Finding the right threshold is one of the most important tuning decisions a security team makes.

Choosing a Network Detection Method

When picking how a detection system actually identifies threats, there are two main approaches, plus a combination of both. Which one you choose depends on your network's traffic, your data sensitivity, and your budget.

Signature-Based Detection

Signature-based detection compares network data against a database of known indicators of compromise (IoCs), called signatures. Think of it like antivirus software: if the traffic matches a known bad pattern, it gets flagged.

Key features:

• Fast and efficient, especially on networks with high traffic volume
• The signature database must be regularly updated with the latest known attack patterns
• It cannot detect brand-new attacks (no signature exists yet)

Anomaly-Based Detection

Anomaly-based detection compares current traffic to a baseline of normal recorded activity. If the current traffic falls outside the expected tolerance range, the system raises an alert.

Important details:

• Baselines must be recorded on uncompromised systems. If you record your baseline while you're already infected, "normal" includes the malware.
• It works best on networks with consistent traffic patterns, because anomalies only stand out when there's a steady pattern to deviate from.
• It can catch novel attacks that signature-based detection would miss
• It runs slower than signature-based detection

Hybrid Detection

Hybrid detection combines signature-based and anomaly-based methods. Networks that handle sensitive or critical data (hospitals, banks, power grids) usually go this route. The downsides: hybrid is the most expensive option, and it generates the most alerts because both methods are running at once.

Picking the Right Method

The CED gives you four criteria to weigh:

If novel attacks are expected but hybrid is too expensive, anomaly-based is the way to go since signature-based literally can't catch something it has no signature for.

Evaluating the Impact of a Detection Method

Once you've picked a method, you need to evaluate how well it actually works. There are four big factors.

Speed of Detection

Faster detection means faster response, which means less damage. Signature-based detection is the fastest, especially when traffic volume is high, because it's just doing pattern matching against a database.

Cost

Budget matters. Here's the ranking from cheapest to most expensive:

1. Signature-based (cheapest)
2. Anomaly-based (needs more powerful hardware to handle the analysis)
3. Hybrid (most expensive, since you're running both)

False Positive Rate

A false positive happens when the system raises an alert about activity that turns out to be totally normal.

• Signature-based has almost no false positives. If it matches a known bad signature, it's probably actually bad.
• Anomaly-based and hybrid have higher false positive rates because any deviation from baseline triggers an alert, even harmless ones.

High false positive rates cause two big problems:

1. Time and resources get wasted investigating fake alerts.
2. Alert fatigue sets in. This is when responders get so used to false alarms that they start assuming every alert is fake before even checking. That's how real attacks get ignored.

False Negative Rate

A false negative happens when an attack slips through undetected. This is arguably worse than a false positive, because you have no idea anything is wrong.

• Signature-based systems are easier for adversaries to bypass. Tweak the attack slightly so it doesn't match a known signature, and you're invisible.
• Anomaly-based and hybrid systems are harder to bypass, since even a modified attack will likely create unusual traffic patterns.

False negatives can lead to data loss, system damage, service disruption, or worse.

Identifying Attacks in Log Files

The last big skill is recognizing specific attacks based on what they look like in network data. Each attack leaves its own fingerprint.

Evil-Twin Attacks

An attacker sets up a fake Wi-Fi access point with an SSID (service set identifier) that mimics a legitimate one, like "Starbucks_WiFi_Free" next to the real "Starbucks_WiFi." To detect:

• Regularly scan for SSIDs that look suspiciously similar to your legitimate ones
• Use signal triangulation to physically locate the rogue access point and shut it down

Jamming Attacks

A jamming attack floods the wireless spectrum with noise so nothing can connect. Signs to look for:

• No wireless devices in a particular area can connect, even though the network should be working
• Scanning shows high electromagnetic (EM) noise in the wireless frequency range

ARP Poisoning

In an ARP poisoning attack, the attacker sends fake ARP messages to associate their MAC address with another device's IP. Detection methods:

• Monitor network traffic for unusual ARP messages, especially duplicate MAC address ARP packets
• Check the ARP table on the default gateway for entries that don't match expectations

MAC Flooding

An attacker overwhelms a switch by sending tons of Ethernet frames with different MAC addresses, filling up the switch's MAC table and forcing it to broadcast traffic to everyone. To detect:

• Monitor for a sudden surge of Ethernet frames with many different MAC addresses
• Check the MAC address table on the switch for an abnormal number of entries

DNS Poisoning

DNS poisoning is one of the hardest attacks to detect because everything looks normal from a traffic standpoint. The clearest sign:

• Your website experiences a sudden, unexplained drop in traffic. Users are being redirected somewhere else without knowing it. If that happens, examine your DNS records.

Smurf Attacks

A smurf attack sends ICMP echo requests (pings) to a network's broadcast address using a spoofed source IP, causing every device to flood the victim with replies. The signature is obvious once you know to look:

• A sudden spike in ICMP requests aimed at the broadcast address

General Network-Based IoCs

Beyond specific attacks, there are general network-based IoCs you'll find in packet capture files. These show up in source and destination IP addresses, ports, and protocols. Watch for:

• Connections to known malicious IP addresses
• Unauthorized network scans (lots of connection attempts to different ports)
• Unusual spikes or slowdowns in network traffic
• Mismatched port-application traffic (for example, HTTP traffic on a port that shouldn't be running a web service)

When you're analyzing a packet capture or log file, those four categories are your starting checklist. Most attacks leave at least one of those breadcrumbs behind.