Rollout: Splunk 3.0

Posted by Mike Fratto, Editor on September 27, 2007

Tags: Snort,  ArcSight,  LogLogic,  LogRhythm,  Mike Fratto,  data,  field parse,  flash,  Data Networking & Management,  Network Management ,  Network Performance, Utilization, QoS monitoring,  Nework capacity planning, design, load balancing,  Servers,  Servers & Storage,  Splunk,  Splunkbase,  Systems Management,  WAN Optimization and Application Acceleration,  log

Channel: Networking & Mgmt, Servers & Storage, WAN & App Acceleration

Aggregating and analyzing log data is an IT best practice—and a requirement in regulated industries—but it can also be a pain in the you-know-what. Many log aggregation products have purpose-built parsing engines that process logs as they're received and build up event databases. This works well if all your log sources have parsers built in, but not all do. That means for unsupported devices, events are stored as raw log data that is not easily searched.

While there are some formal and informal standards for log formats including syslog, HTTP logs and Windows Event Logs, there are no standards for log messages themselves. This makes extracting meaning from events difficult.

Meanwhile, the volume of data that network devices and servers generate can be staggering.

Enter Splunk 3.0, the latest software release from Splunk. This excellent analyzer accepts any plain text as unstructured log data, indexes keywords and stores the records. Splunk then uses a search-based interface for log analysis.

As a general IT tool, Splunk is an excellent log analysis system for organizations of all sizes. You may quickly find yourself doing away with the largely pathetic log analysis tools built into network products like firewalls and device management systems.