Not all Logs are Created Equal
Posted on March 12, 2026 • 7 minutes • 1285 words
To have a well-oiled incident response team, it is crucial to have strong detection rules. Although security teams know what good detection rules look like, many other teams that help provide logging to enable detection do not know which information is useful to an incident response team or a detection team. This gap of what is useful and what is not can cause friction between the cyber teams (incident response and detection) and the teams that help them. This blog post helps bridge this gap.
Understanding the Different Teams
The Incident Response (IR) team, Security Operations team, or Security Operations Center (SOC) is responsible for responding to potential security incidents. The alerts the SOC or IR receives are generated by the detection team, which writes detection rules.
A Detection Team is responsible for creating rules that catch potentially malicious behavior or abnormal activity. It creates rules by analyzing logs over time to identify anomalous patterns, baselining normal behavior, and developing logic based on threat-actor activity and intelligence. A good detection engineer must know their log sources well, be knowledgeable about the ever-changing threat landscape, and write rules that are robust enough to withstand shifts in attacker behavior, using both behavior-based and heuristic-based rules. However, not all organizations can afford detection and incident response teams. Depending on an organization’s maturity level, there might be a single team that handles both incident response and detection engineering.
The logging team is not a specific organization or team, but rather a team that a detection engineering team relies on to enable logging. For example, it could be the enterprise Salesforce team or the team responsible for a custom application within the organization.
Difference between a Useful Log and a Not-So-Useful Log
A common example of a log that is enabled but not sufficient is audit logs. An audit log records system activity, the logged-in entity, the timestamp, and the outcome. Audit logs for most SaaS (Software as a Service) applications are available for free to help organizations with compliance. The example below shows an audit log:
{
"timestamp": "2025-03-15T09:04:32.441Z",
"event_type": "user_login",
"username": "jsmith",
"outcome": "success"
}
In plain English
On March 15th, 2025, at 9:04 am UTC, jsmith logged in successfully.
Even though the log above might look useful, for an enterprise application with thousands of users, these logs provide very little context. For example, knowing that the user logged in at 9:04 am UTC does not give the IR team any actionable items. It does not tell them whether this is an unusual sign-in, which device the user logged on to, or what the user did after logging in. Without this context, an audit log like this cannot distinguish between legitimate and malicious users. In addition, this log is very noisy because it creates an entry every time a user logs in. This can hinder performance and unnecessarily increase the SIEM (Security Information and Event Management) budget. Noisy audit logs burn the budget for nonessential logs, preventing the ingestion of more useful logs that could improve detection or other parts of the security organization.
Example of a Useful Log
{
"timestamp": "2025-03-15T09:04:32.441Z",
"application": {
"name": "HR Portal",
"version": "4.2.1",
"environment": "production"
},
"event_type": "user_login",
"outcome": "success",
"user": {
"username": "jsmith"
},
"request": {
"ip_address": "192.168.1.105",
"network": "corporate",
"user_agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36"
},
"authentication": {
"password_verified": true,
"mfa_sent": true,
"mfa_approved": true,
"identity_provider": "Okta"
},
"session": {
"session_id": "a3f92b1c-4d78-11ee-be56-0242ac120002"
}
}
In plain English: On March 15th, 2025, at 9:04 am UTC, the HR Portal recorded a login from jsmith. This user initiated a login request using Chrome browser version 120.0. The application confirmed the password was correct, sent an MFA notification through Okta, and recorded that jsmith approved it. The application assigned a session ID to track all of jsmith’s activity for the duration of their visit, and logged that the request came from IP address 192.168.1.105 on the corporate network. It also recorded that this application was running version 4.2.1.
Several characteristics make this a useful log. It answers the core questions: who took an action, what they did, where the request came from, and how they authenticated. Each field is distinct enough to serve as a baseline for normal behavior. In addition, each field is sufficiently unique to enable rules that can help distinguish a legitimate user from a potentially malicious one.
To further illustrate why this matters, consider a scenario in which the same username logs in from two different IP addresses within minutes of each other, using an unrecognized user agent, and no MFA is required. On its own, any one of these data points provides very little insight. But when correlated together, they tell a different story: the IP addresses suggest the user is in two places at once, the unfamiliar user-agent indicates an unusual or automated client, and the missing MFA step raises the question of whether authentication was bypassed entirely. A detection engineer can only build a rule that catches this pattern if all three fields are present in the log to catch malicious activity.
How Context-rich Logs Lead to Better Detection Engineering and an Efficient SOC
A context-rich log is the foundation of good detection engineering, and a good detection rule is the foundation of an efficient SOC. The relationship among the three is similar to that among fresh ingredients, a recipe, and a chef.
As the analogy above suggests, a context-rich log provides a detection engineer with the foundation needed to write high-fidelity rules. High-fidelity rules fire on genuine threats and rarely on legitimate activity. Making high-fidelity rules is only possible when a detection engineer has enough context to baseline normal activity. Context-rich logs provide that baseline, giving detection engineers the data points needed to identify suspicious activity and giving the SOC something worth investigating when an alert fires.
Having high-fidelity rules is important because it prevents SOC alert fatigue. Alert fatigue occurs when analysts are overwhelmed by a high volume of alerts, most of which are false positives. Most SOC teams are underfunded due to budget cuts, so it is more important than ever to have high-quality alerts to maximize resources and minimize alert fatigue. Just as a chef cannot save a dish made with bad ingredients, no matter how talented they are, a SOC analyst cannot investigate their way out of a flood of low-quality alerts rooted in poor log data.
Finally, by ingesting only context-rich, relevant logs, organizations can improve SIEM performance, leading to faster investigations and lower costs. Removing irrelevant data can help SIEM and detection rules execute faster and enable analysts to query more quickly. Increasing analyst efficiency reduces the time for a threat actor to move through the environment, escalate privileges, and cause more damage. In addition, by ingesting context-rich logs, security teams can save money on log storage and reinvest in analyst training and development.
Closing Thoughts
Good security is more than having the right tools or the right people; it’s about having the right mindset for digesting context-rich logs. Every detection rule, alert, and investigation depends on the quality of the data.
For every team responsible for an application that feeds into a security pipeline, the question is simple:
Is what is being logged actually useful? Does it give the security team enough context to do their job? Is the log just for compliance purposes?
This distinction is very important. This is particularly important for small security teams already stretched thin, as a flood of low-context logs without any actionable items is barely more helpful than no logging at all.
The security team’s ability to protect the organization traces back to their ability to provide context-rich logs.
Log with intention.
References