An alert could be a suspicious sign-in, a malicious application is running, unfamiliar connections are being used, or a device is possibly under a compromised user. But not all alerts are a true security incident. Some false positive, some more investigations are needed before determining their severity.

Knowing what to do after an alert comes in is key to anyone's security operations learning.

We will discuss the common sequence of events in the life cycle of a security alert: from detection, investigation, response, to closure.

1. Alert Generation – Warning of Suspicious Activity

Typically, this is triggered by a security tool that detects something that matches a detection rule or behavioral analytic.

Common sources include:

• Endpoint Detection and Response (EDR) platforms

• Security Information and even Event Management (SIEM) systems

• Firewalls and network security appliances are examples of network security appliances.

• Identity and access management platforms

• Email security gateways

• Cloud security monitoring tools

For instance, a SIEM (security information event management) platform could trigger an alert for a user if they attempt to sign in several times, yet finally sign in.

Detection rule is used to discover a pattern that might be considered suspicious, but doesn't necessarily mean it is suspected that an account has been compromised.

Key takeaway: An alert is not a security incident; it is an indicator that something is afoot and you need to investigate it further.

2. Alert Triage: What is important?

Alert triage is next.

A SOC analyst checks all information to decide if the alert is relevant, suspicious and needs further investigation.

Typical triage activities include:

• The description of the alert and the logic behind the detection

• Finding impacted users, endpoints, IPs, and apps

• This allows you to check the event timeline.

• Reviewing severity and confidence.

• Correlating related alerts

• Seeking familiar non-threatening causes

Think of first detection event that is repeated when someone fails to log in.

Before taking any action, an analyst should look into whether the activity was from a legitimate user who entered a wrong password, an automated application with outdated credentials, or a potential brute force attack.

Context is critical. A large amount of unsuccessful logins does not prove malicious intent.

3. Investigation: Building the Bigger Picture

When the alert is still suspicious, the analyst enters into a more in-depth investigation.

It frequently requires gathering extra telemetry from several different sources, and assembling a timeline of events.

The analyst may analyze:

• The reputation of the source IP address and its reputation

• The typical way that a user logs into the system. The usual way the user logs in to the system.

• Report results and failure information. Report results and reasons for failure.

• Geometry and hardware errors.

• MFA activity

• The use of sensitive resources will be available in subsequent access.

• Endpoint or identity alerts that are related.

This information can be correlated by security platforms like Microsoft Sentinel, Defender XDR, and others.

The goal is to determine what has occurred, what assets or identities may have been compromised and if the activity is malicious.

Evidence is the basis of a good investigation, not assumptions.

4. Incident Classification and Escalation

When sufficient evidence has been gained, the analyst decides on the next action.

Possible outcomes include:

False positive: The activity is not malicious and it may require tuning if it produces unwanted alerts frequently.

Benign true positive: The detection was correct and the activity is a legitimate one.

Suspicious activity: More evidence/investigation needed.

Confirmed Security Incident: Evidence exists of malicious activity or a security incident that needs incident response.

Depending on the SOC's operating procedures, the analyst can escalate the case to a senior analyst, incident responder, threat hunter or security engineering team.

This is where good record keeping is crucial. The escalation should include what was identified, what was investigated, evidence found, and what is recommended.

5. Incident Response: Containing the Threat

Once it has been determined that malicious activity is occurring, the response team reacts to the nature, scale and authorized protocol of the incident.

Response actions vary based on the situation but might include:

• Isolating an infected device

• Shutting off or limiting a hacked account

• Revocation of sessions or credentials – active.

• Blocking malicious indicators

• Removing persistence mechanisms

• Eradicating malware

• Recovering affected systems

These actions should be influenced by the normal authorization and change-control procedures. Business impact must also be considered as part of containment, particularly as related to the critical production systems.

For instance, an employee workstation that is infected with a threat could be isolated easily, but a production server could be a critical business service.

The aim is to minimize damage and to retain the evidence needed for investigation.

6. Documentation and Closure

Alerts need not be closed for an investigation to be complete.

An incident record should include:

• A summary of the incident(s) in a few words.

• The affected assets & identities.

• Appropriate time references and documentation

• A process to follow and findings from the investigation.

• Responses to action and their likely consequences. Activities and their likely consequences.

• Escalations and approvals

• Final disposition and/or disposition reason (closure reason)

The documentation helps with compliance, improvement of detection, investigation in the future and incident reviews.

It also enables another analyst to be able to comprehend the case without having to do a fresh investigation.

7. Detection Improvement – Learning from each incident

The last step is "continual improvement".

SOC teams should look to see if the detection was successful, if enough context was provided in the alert, and if analysts had to perform repetitive manual tasks.

Potential improvements include:

• Refining detection logic

• Minimizing false positives

• Adding contextual enrichment

• Correlating related alerts

• Repeated steps in investigations: automation.

• To streamline ticket routing and escalation

• Creating new detections based on attacker actions they see.

Automation can support analysts to gather evidence and enrich alerts in a faster manner. Automated response actions, however, should be appropriately safeguarded, particularly when they can impact business operations.

The goal is not just to extinguish warnings quicker. For enhancement of detection quality, consistency of investigation and effectiveness of response.

Final Thoughts

The SOC process is a series of steps that involve detection, triage, investigation, classification, response, documentation and continuous improvement before the significance of a SOC alert can be determined.

They all play a part in an effective security operations process.

It's better for aspiring SOC analysts to understand how to investigate alerts in a systematic way than to memorize what the alerts are. Once you get some experience, the quality of detection, contextual enrichment, and workflow automation can enhance the entire process.

In future articles, I’ll cover real-world SOC engineering scenarios such as Microsoft Sentinel, KQL, threat detection, incident investigation and security automation.

Which part of the alert lifecycle is the most difficult for you: triage, investigation, tuning detection, or response automation?