Topic 3.5 · Unit 3: Securing Networks · 35 minute session

3.5 Detecting Network Attacks

Networks are monitored by automated tools: a network intrusion detection system (NIDS) alerts, a network intrusion prevention system (NIPS) can also block, and a security information and event management (SIEM) system correlates many sources. Artificial intelligence (AI) models score how likely activity is malicious. Signature-based detection is fast with few false positives but misses new attacks; anomaly-based detection catches new attacks on consistent networks; hybrid does both at higher cost.

Big idea No single detection method wins: choose by traffic volume, consistency, sensitivity and the chance of new attacks.

Course and exam description (CED) codes covered: 3.5.A.13.5.A.23.5.A.33.5.A.43.5.B.13.5.B.23.5.B.33.5.B.43.5.C.13.5.C.23.5.C.33.5.C.43.5.D.13.5.D.23.5.D.33.5.D.43.5.E.13.5.E.23.5.E.33.5.E.43.5.E.53.5.E.63.5.E.7

Last reviewed against the CED:

By the end of this session you can

  • 3.5.A Identify types of automated security tools used to detect network attacks.
  • 3.5.B Explain how organizations can leverage artificial intelligence (AI) to enhance threat detection and response.
  • 3.5.C Determine a network detection method.
  • 3.5.D Evaluate the impact of a network detection method.
  • 3.5.E Apply detection techniques to identify indicators of network attacks by analyzing log files.

Already know this? Jump to the checkpoint.

1. Automated detection tools (3.5.A)

Detection tools analyze data collected from switches, routers, servers, firewalls and user computers, usually stored in log files 3.5.A.1.

Tool What it does
Network intrusion detection system (NIDS) 3.5.A.2 Analyzes data to decide whether malicious activity is happening on the network, and generates an alert
Network intrusion prevention system (NIPS) 3.5.A.3 Detects like an intrusion detection system (IDS), and can also mitigate or halt the attack by closing ports, blocking specific internet protocol (IP) or media access control (MAC) addresses, or rejecting specific protocols
Security information and event management (SIEM) 3.5.A.4 Collects and analyzes data from many sources (firewalls, NIDS and NIPS, device logs, application logs) to find patterns of an attack, then raises an alert. Analysts investigate whether it is a true threat and follow standard operating procedures to resolve or escalate it

2. AI in threat detection (3.5.B)

  • Computers log every user action, and firewalls, IDSs, intrusion prevention systems (IPSs) and other sensors log all traffic. A medium-sized organization logs millions, even tens of millions, of data points a day: far too many for any human team 3.5.B.1.
  • Threat detection teams build AI algorithms that analyze this data and classify patterns as malicious or normal 3.5.B.2.
  • The models are probabilistic: they report a percentage for how likely something is malicious 3.5.B.3.
  • Each organization sets its own alert threshold 3.5.B.4:
Threshold Result
Too high Real attacks may go undetected
Too low The team is overwhelmed with false alerts

3. Choosing a detection method (3.5.C)

Network trait Best fit Why
High traffic volume 3.5.C.1 Signature-based Compares data to a database of known indicators of compromise (IoCs), called signatures. Runs faster than anomaly-based. The database must be kept updated.
Consistent traffic patterns 3.5.C.2 Anomaly-based Compares data to a baseline recorded on uncompromised systems, and alerts when data types or volumes fall outside a tolerance range. Needs consistent patterns to work.
Sensitive or critical data or services 3.5.C.3 Hybrid Combines both. More expensive than either alone and generates more alerts.
Novel attacks likely 3.5.C.4 Anomaly-based (if hybrid costs too much) Signature-based detection cannot detect a new attack.

4. Evaluating a detection method (3.5.D)

Factor Signature-based Anomaly-based Hybrid
Speed 3.5.D.1 Faster, especially on high-traffic networks Slower
Cost 3.5.D.2 Cheaper hardware Needs more expensive hardware Most expensive
False positives 3.5.D.3 Almost none Higher Higher
False negatives 3.5.D.4 Easier to bypass Harder to bypass Harder to bypass
  • Faster detection means faster response 3.5.D.1. Tools and running costs must fit the budget 3.5.D.2.
  • A high false positive rate wastes time and resources on harmless activity and causes alert fatigue, when responders assume alerts are false before investigating 3.5.D.3.
  • A false negative is when an adversary gets past the detection system, which can lead to loss, harm, disruption or destruction of data and systems 3.5.D.4.

5. Indicators of each network attack (3.5.E)

Attack How to detect it
Evil twin 3.5.E.1 Regularly scan for service set identifiers (SSIDs) that look suspicious or similar to legitimate local ones. Use signal triangulation to locate and disable the access point.
Jamming 3.5.E.2 No wireless device in an area can connect, and scanning finds electromagnetic (EM) noise in the wireless range.
ARP poisoning 3.5.E.3 Unusual address resolution protocol (ARP) messages, especially ARP packets with duplicate MAC addresses, and a check of the ARP table on the default gateway.
MAC flooding 3.5.E.4 An unexpected surge of Ethernet frames with different MAC addresses, and a check of the switch’s MAC address table.
DNS poisoning 3.5.E.5 Hard to detect. An abrupt, otherwise unexplained drop in website traffic is a reason to examine domain name system (DNS) records.
Smurf attack 3.5.E.6 A sudden increase in internet control message protocol (ICMP) requests sent to the network’s broadcast address.

Network-based IoCs are found by analyzing traffic, often in packet capture files, looking at source and destination IP addresses, ports and protocols 3.5.E.7:

  • connections to known malicious IP addresses
  • unauthorized network scans
  • unusual spikes or slowdowns in traffic
  • mismatched port and application traffic, such as web traffic on a port meant for something else

The 3.1 simulators already let you play defender for ARP poisoning, MAC flooding and DNS poisoning. Below, hunt for IoCs in a firewall log.

Lock it in: memory hooks

One picture per idea. If you remember the hook, you can rebuild the details.

  • NIDS vs NIPSD is for doorbell, P is for padlock: detection rings, prevention locks.
  • Signature vs anomalySignature checks a most-wanted list; anomaly notices when the street looks different from normal.
  • Choosing a methodBusy network, go signature. Steady network, go anomaly. Critical network, go hybrid (and pay for it).
  • False resultsFalse positive: the alarm cries wolf. False negative: the wolf walks in silently.

Common misconceptions

  • "Signature-based detection catches everything because it knows every attack."

    Not quite. It only knows attacks already in its database. It cannot detect a new attack, and it is easier to bypass than anomaly-based or hybrid detection.

  • "A lower AI alert threshold is always safer."

    Not quite. Too low and the team drowns in false alerts and starts ignoring them. Too high and real attacks go undetected.

  • "A NIDS stops attacks."

    Not quite. A NIDS detects and alerts. A NIPS can also act, by closing ports, blocking addresses or rejecting protocols.

Simulation · [3.5.E.6, 3.5.E.7]

Log hunt: find the indicators of compromise in a firewall log

The school has listed 203.0.113.0/24 as known malicious. The web server is 10.0.1.10. Ports: 443 is Hypertext Transfer Protocol Secure (HTTPS) and 22 is secure shell (SSH); HTTP is unencrypted web traffic and TCP is the transmission control protocol. Tap every line that is an indicator of compromise.

edge-fw · traffic log 10:00 to 10:05swipe sideways for more
TIMESOURCEDESTPORTPROTONOTE
  1. Fine. A student laptop loading the school website over HTTPS on port 443. Normal.

  2. Red flag. A connection to a known malicious IP address. That laptop may be infected and calling out.

  3. Red flag. One device probing hundreds of ports across a whole subnet: an unauthorized network scan.

  4. Fine. Remote login from the admin subnet on port 22, the right port for secure shell. Expected.

  5. Red flag. Mismatched port and application: web traffic on port 22, which belongs to secure shell.

  6. Red flag. A sudden flood of internet control message protocol (ICMP) requests sent to the broadcast address 10.0.2.255: the sign of a smurf attack.

  7. Fine. Normal again.

Exam traps

  • A NIDS alerts; a NIPS can also block. In Unit 2 terms, an IPS is corrective.
  • Signature-based detection cannot detect a new attack, however fast it is.
  • An alert threshold set too high misses real attacks; set too low, it causes alert fatigue.
  • DNS poisoning is hard to detect: look for a sudden, unexplained drop in website traffic.

By the numbers

Say each list out loud, then tap to check.

3 tools Say them, then tap
3 methods Say them, then tap
  • signature-based
  • anomaly-based
  • hybrid
4 criteria for choosing Say them, then tap
  • traffic volume
  • traffic consistency
  • sensitivity or criticality
  • likelihood of novel attacks
4 factors for evaluating Say them, then tap
  • speed
  • cost
  • false positive rate
  • false negative rate
4 network IoCs Say them, then tap
  • known malicious IP addresses
  • unauthorized scans
  • unusual spikes or slowdowns
  • mismatched port and application

In 30 seconds

  1. Tools analyze log data from switches, routers, servers, firewalls and computers. NIDS alerts; NIPS can block; SIEM correlates many sources and analysts follow standard procedures.
  2. Networks log millions of events a day. AI classifies patterns as malicious or normal and reports a probability; the organization sets the alert threshold.
  3. Signature: fast, efficient on high-traffic networks, almost no false positives, needs updates, misses new attacks. Anomaly: needs a clean baseline and consistent traffic, catches new attacks, costs more, more false positives. Hybrid: both, most expensive, most alerts.
  4. Evil twin: scan for look-alike SSIDs, triangulate. Jamming: nobody can connect, EM noise. ARP poisoning: duplicate MAC ARP packets, gateway ARP table. MAC flooding: surge of frames with different MACs, switch table. DNS poisoning: sudden unexplained drop in site traffic. Smurf: spike of ICMP requests to the broadcast address.
  5. Network IoCs: known malicious IP addresses, unauthorized scans, unusual spikes or slowdowns, mismatched port and application.

Cover this list, say the points out loud, then take the checkpoint without scrolling back up.

Checkpoint · 8 questions

Checkpoint: Detecting Network Attacks

Pick an answer to mark it. Every answer is explained, and you can try any question again. Your saved score counts first tries only, and stays in this browser.

  1. 1 What can a network intrusion prevention system (NIPS) do that a NIDS cannot? 3.5.A.2, 3.5.A.3
    Show answer

    Answer: A. Both analyze data and detect attacks. A NIPS can also mitigate or halt an attack by closing ports, blocking specific IP or MAC addresses, or rejecting protocols.

  2. 2 What makes a SIEM system different from a single sensor? 3.5.A.4
    Show answer

    Answer: D. A security information and event management (SIEM) system correlates data from firewalls, NIDS and NIPS, device logs and application logs, then alerts analysts.

  3. 3 An AI detector scored real attack traffic at 72% likely malicious, but no alert fired because the threshold is 95%. What does this show about the threshold? 3.5.B.3, 3.5.B.4
    Show answer

    Answer: C. If the threshold is set too high, real attacks may go undetected. Too low, and the team is overwhelmed with false alerts.

  4. 4 A busy video streaming company wants fast detection with very few false positives. Which method fits best? 3.5.C.1, 3.5.D.1, 3.5.D.3
    Show answer

    Answer: B. Signature-based detection is more efficient on high-traffic networks, runs faster, and has almost no false positives.

  5. 5 A research lab expects adversaries to try brand-new attacks, but cannot afford hybrid detection. What should it choose? 3.5.C.4
    Show answer

    Answer: C. Signature-based detection cannot detect a new attack. Anomaly-based detection is preferred when hybrid detection costs too much.

  6. 6 What is alert fatigue? 3.5.D.3
    Show answer

    Answer: C. Alert fatigue happens when responders get used to false positives and take alerts less seriously.

  7. 7 A log shows a sudden spike of ICMP requests sent to 10.0.2.255, the network's broadcast address. Which attack is most likely? 3.5.E.6
    Show answer

    Answer: A. Smurf attacks can be detected by a sudden increase in ICMP requests sent to the network's broadcast address.

  8. 8 A gateway's ARP table lists the same MAC address for two different IP addresses. Which attack does this indicate? 3.5.E.3
    Show answer

    Answer: C. Duplicate MAC address ARP entries, found by checking the default gateway's ARP table, indicate ARP poisoning.

Key terms: flip the cards

Read the term, say the definition out loud, then flip to check. Struggling a little to remember is what makes it stick.

  • Network intrusion detection system (NIDS)tap to flip

    An automated tool that analyzes network data to determine if malicious activity is happening and generates an alert.

    Example and CED codes
  • Network intrusion prevention system (NIPS)tap to flip

    An automated tool that detects malicious network activity like an IDS and can also halt it by closing ports, blocking addresses or rejecting protocols.

    Example and CED codes
  • Security information and event management (SIEM)tap to flip

    A system that collects and analyzes data from many sources to detect attack patterns and alert analysts, who resolve or escalate the alert.

    Example and CED codes
  • Signature-based detectiontap to flip

    Detection that compares data to a database of known indicators of compromise; fast, almost no false positives, but cannot detect new attacks.

    Example and CED codes
  • Anomaly-based detectiontap to flip

    Detection that compares data to a baseline of normal activity and alerts on data outside a tolerance range; catches new attacks but costs more and has more false positives.

    Example and CED codes
  • Hybrid detectiontap to flip

    Detection that combines signature-based and anomaly-based methods; the most expensive option and it generates more alerts.

    Example and CED codes
  • Indicator of compromise (IoC)tap to flip

    Evidence in data or traffic that a system may be under attack, such as connections to known malicious addresses or mismatched port and application traffic.

    Example and CED codes
  • False positivetap to flip

    An alert for activity that is actually harmless; high rates waste time and cause alert fatigue.

    Example and CED codes
  • False negativetap to flip

    When an adversary bypasses a detection system, so a real attack raises no alert.

    Example and CED codes
  • Alert fatiguetap to flip

    When responders get used to false positives and take alerts less seriously, assuming they are false before investigating.

    Example and CED codes
  • Triangulationtap to flip

    Locating a signal's source by measuring its strength from several positions.

    Example and CED codes

Questions students ask

What is the difference between signature-based and anomaly-based detection?

Signature-based detection compares data to a database of known indicators of compromise; it is fast with almost no false positives but cannot detect new attacks. Anomaly-based detection compares data to a recorded baseline of normal activity and alerts on anything outside a tolerance range, so it can catch new attacks but has more false positives and costs more.

What is alert fatigue?

When responders get used to false positives and start taking alerts less seriously, assuming they are false before investigating.

How do you detect a smurf attack?

Watch network traffic for a sudden increase in ICMP requests sent to the network's broadcast address.

↑ Top

Enlarged view of the selected figure