Aggregating and analyzing log data is an IT best practice—and a requirement in regulated industries—but it can also be a pain in the you-know-what. Many log aggregation products have purpose-built parsing engines that process logs as they're received and build up event databases. This works well if all your log sources have parsers built in, but not all do. That means for unsupported devices, events are stored as raw log data that is not easily searched.
While there are some formal and informal standards for log formats including syslog, HTTP logs and Windows Event Logs, there are no standards for log messages themselves. This makes extracting meaning from events difficult.
Meanwhile, the volume of data that network devices and servers generate can be staggering.
NEWS | REVIEWS | BLOGS | FORUMS TUTORIALS | STRATEGY | MORE
Enter Splunk 3.0, the latest software release from Splunk. This excellent analyzer accepts any plain text as unstructured log data, indexes keywords and stores the records. Splunk then uses a search-based interface for log analysis.
As a general IT tool, Splunk is an excellent log analysis system for organizations of all sizes. You may quickly find yourself doing away with the largely pathetic log analysis tools built into network products like firewalls and device management systems.