Network Computingwww.networkcomputing.com

Whether to improve performance, gather business intelligence, or detect security threats, log management boils down to three steps: Collect the logs, store the data, and analyze the data to identify patterns.

Yet, while the collection and analysis of log data is one of 20 critical security controls identified by the SANS Institute, most companies do not regularly collect and analyze their logs unless required by regulations. With so much data, information technology professionals can be confused as to where to start, says Nicole Pauls, product manager for SolarWinds, a maker of IT management and monitoring software.

"When people come to log management, they are flooded with a lot of data," she says. "What people are trying to find are the anomalies, the patterns that hint at something going on, but it's difficult."

Good security log analysis revolves around four principals, says Ben Feinstein, director of operations and development for Dell SecureWorks' Counter Threat Unit. First, companies need to monitor the right logs, including data from firewalls, virtual private networking (VPN) appliances, Web proxies, and DNS servers. Next, the security team must collect data on what "normal" looks like inside the company's network. Third, analysts must identify the indicators of attacks in their log files. Finally, the security group must have a procedure for responding to incidents identified by log analysis.