Log Management Gets SLIM

Log management is a regulatory requirement and best practice. It has grown from simple aggregation and storage of logs to become another data resource that can be mined, trended and reported on.

Q1 Labs' Simple Log and Information Management—SLIM—platform stores logs from a variety of devices and can correlate events and create ad hoc and scheduled reports. The appliance is rated for 5,000 events per second; adding more devices increases this events-per-second ratio.

SLIM's event correlation feature can be useful for uncovering malicious or unwanted activity in real time and can be easily customized. It also includes report templates for regulations such as Sarbanes-Oxley and GLB. However, SLIM is not as agile with real-time data mining or arbitrary event data compared with products from Splunk or LogLogic, both of which create indexes of data as they stream from event sources. SLIM is a good fit for companies that want to automate report generation and event correlation from log data.

As tested, SLIM costs $24,000; the product ships with 2 terabytes of disk space, and raw data and indexes are compressed after two days, conserving space with minimal impact on searching. Splunk's commercial software starts at $5,000 for 500 MB of indexed data per day, and hardware typically runs to over $10,000 for a beefy server. Moreover, Splunk doesn't have SLIM's event correlation component. A more comparable product, LogLogic's LX 2010, lists for $28,000 plus an additional $14,999 for compliance and control suites. It has more robust archiving functions and powerful search capabilities.CAPTAIN'S LOG

Once SLIM is installed, you simply forward log sources to the appliance. SLIM ships with a large number of device support modules (DSMs) that parse events from common devices such as Cisco PIX, Linux syslog, Windows event logs, and Web server logs, to name a few. You can write custom DSMs to add your own parsing rules, but you will need to write regular expressions and know the format your logs are in, then write an XML file so SLIM can process incoming messages. It's no more difficult than writing add-ons for Splunk, though you probably want to inquire about the availability of additional DSMs as part of the purchase.

SLIM's log management capabilities revolve around search filters, which are used to investigate events and populate reports. Interactive searching for events is an iterative process of defining a search filter, running the search, refining the search filter and so on. This is where Splunk or LogLogic work better by auto-populating the search fields based on the indexed data. By contrast, with SLIM, you have to know what words to search for before you begin.

Search is where SLIM shows its event reporting roots. Searches are defined by specifying one of the predefined fields, selecting an operator (which changes based on the selected field) and then choosing the string you are looking for. In addition to keyword and numeric strings, regular expressions can be defined to search the packet payload, a useful feature when dealing with unparsed data. Searches can be saved for later use and shared with others.

Once the data is retrieved, we were able to view it in multiple ways using a drop down menu. For example, we created a filter that pulled up firewall deny events from our Sonicwall firewall, then we generated charts of aggregated data showing the top targets, top sources, top ports, and top protocols. Within a few minutes we found external hosts scanning ports commonly used for VNC, the open source desktop remote control program, as well as and Symantec Anti-virus. We could also drill into the aggregated data for further analysis.SLIM ships with a number of predefined reports for various regulations such as GLBA and SOX, and for standards such as COBIT. It also provides executive reports. Using a drag and drop interface, we built a number of reports that could be exported to common formats, including PDF, HTML, XML and CSV. Flexible scheduling and multiple formats eases integration with existing business processes and consumption by other products.

IT'S ALL ABOUT RELATIONSHIPS

SLIM's robust event correlation engine is somewhat unique to the log management market. It lets you create rules to match up events as they stream into the appliance. Using event correlation, disparate events can be related to generate a meta-event.

For example, an IDS may trigger on two separate events, such as an attack attempt, and a string that indicates a shell was opened on a host. It's up to the administrator reviewing raw logs to recognize that these two events are strong indication of a successful attack. With SLIM, you can write a rule to combine such events and notify an administrator. For instance, a rule might read "If I see multiple attacks followed by a command shell against the same destination IP within 1 hour, alert an administrator of a successful exploit."

SLIM can also forward events to other systems if needed, and can send data to an archive. A typical strategy is to archive one day's worth of data and save that file to external storage. Archived data can be re-imported into SLIM and searched, but it won't be archived a second time. Locating archived data is separate process not managed by SLIM, so you will have to determine how archives are named, saved and retrieved.SLIM is a well rounded log analysis product suited for report generation and event correlation. Its search capabilities aren't as slick as Splunk's or LogLogic's, but it is powerful enough to dig through mounds of data. The missing piece—which applies to many log analysis products—is the ability to easily add interpreters for log sources.