How To Protect Your Network's PCs

Enterprise PCs, once nestled behind perimeter security devices, are the new security frontier for 2005. Roving laptops may return to the corporate mother ship with malware that propagates itself throughout the soft chewy inside of the enterprise network. Even stationary desktops can fall victim to rogue programs that exploit OS and application vulnerabilities or are downloaded by end users.

While desktop anti-virus software has become the de facto security standard on enterprise PCs, it's clear that anti-virus alone can't protect these assets. For instance, spyware programs that track user surfing habits often aren't covered in anti-virus signature libraries and usually get passed over during search-and-destroy scans. And when it comes to zero-day attacks, all signature-based solutions are helpless until malware researchers can identify and distribute patterns to detect the new exploits.

Vendors have responded with a bumper crop of software security agents to help you lock down your desktops. The result is a bewildering number of options for beleaguered security professionals to choose from.

At the top of the list is anti-spyware technology, which aims to detect and remove keystroke loggers and Trojans, as well as the annoying adware programs that violate privacy and affect PC performance.

2005 may also be the year that Host-based Intrusion Prevention Systems (HIPSs) lay claim to a significant chunk of real estate on enterprise desktops. This security category has gained the attention of enterprise security pros for its ability to stop zero-day attacks that traditional anti-virus products can't catch right away.Lastly, vendors are putting together integrated suites that start with a personal firewall, then add multiple security features into a single product to simplify desktop security management.

All three categories offer stronger protection for enterprise PCs than anti-virus solutions can alone, but savvy network architects know that every silver lining carries a dark cloud. Anti-spyware detection and removal is an immature technology dominated by start-ups. As a result, many of the products lack the tools necessary for wide-scale deployment and management. HIPSs are notoriously finicky and require careful tweaking and tuning to ensure that harmless applications are allowed to run unhindered. And integrated suites don't include all the features you might want, leaving important holes that need to be filled with other products. The result is that network architects will have to make trade-offs, balancing security, convenience, and a feasible management burden.

We'll examine these three categories in more detail to help you make sense of all the choices. We'll look at the security issues that each category addresses, discuss the pros and cons of each technology, and analyze what the major players are up to.

SPY vs. SPY

In 2004, spyware emerged as a major threat to enterprise PCs. In a recent reader poll conducted by Network Magazine, 93 percent of respondents said spyware was a serious problem, even though nearly 100 percent had anti-virus software installed. In fact, a crop of upstarts, abetted by incumbent anti-virus vendors' slow response to the spyware problem, carved out market share by protecting consumers and corporations from this new breed of intrusive software. These upstarts include Webroot Software, Tenebril, InterMute, and Lavasoft.Spyware affects both PC performance and network security. For example, adware programs that serve pop-ups or track user Web activity can drag down a PC's operating speed or cause programs to fail. Such problems inevitably generate help desk calls from irate users. Other spyware is plain malicious and may leak sensitive information or render the machine vulnerable to remote control.

Traditional anti-virus products can, to some extent, handle the security risks. Desktop anti-virus software can catch known Trojans and keystroke loggers that arrive via e-mail or are discovered on the hard disk during a scheduled scan. Both anti-virus and anti-spy software use signatures to catch these malicious programs.

However, there are three reasons why anti-virus products have been slow off the mark to explicitly address spyware. First, spyware is often delivered via the Web rather than over e-mail. Sometimes spyware can insert itself through browser vulnerabilities. The Download.Ject Trojan, for example, loaded itself onto PCs by exploiting a bug in Internet Explorer. Most adware programs are bundled with free games, peer-to-peer programs, or utilities such as weather monitoring tools that are installed by the user, so traditional anti-virus scanners won't catch them during download.

Second is how spyware is defined. While Trojans and keystroke loggers are clearly malicious and usually illegal, adware exists in a gray area.

"There's a liability issue with spyware," says Bob Hansmann, product marketing manager at Trend Micro. "A lot of companies [that make user tracking programs] call what they do a legitimate marketing practice."This was the subject of a 2003 lawsuit by adware vendor Gator (which has since changed its name to Claria). The lawsuit, aimed at anti-spyware maker PC Pitstop, claimed that calling Gator's product "spyware" amounted to defamation. Following an out-of-court settlement, PC Pitstop no longer calls Gator's product spyware, but still helps people remove it.

"We'll never get a letter from the lawyer of the guy who wrote the Netsky worm saying stop blocking my stuff, but spyware vendors feel they can be as intrusive as they want because you visited the site or downloaded the software," says Hansmann.

Hansmann says Trend Micro has to proceed carefully when defining programs to be blocked or removed. Spyware researchers at Trend Labs have to coordinate with Trend Micro's legal department before including software in a detection database.

Finally, spyware (and adware in particular) is often more difficult to remove than viruses. Many viruses exist as discrete entities on host machines, making them easy to find and delete. Spyware spreads itself throughout a PC, so portions of the program can exist throughout the file system and in dozens or hundreds of registry keys and directories. The result is that anti-spyware engines have to conduct detailed searches and attempt to remove all traces of spyware without affecting the PC's normal operations.

Failure to fully remove the program can affect the PC's performance. That's because spyware often inserts itself into the boot functions of the host PC. If the spyware program has been deleted elsewhere but entries remain in the boot process, the OS may return an error message because it can't find the program to run at startup.It's true that some complex malware, such as the Blaster worm, can insert itself into many points throughout the PC. Anti-virus vendors usually deal with this by creating a separate removal tool, but to do so for every incarnation of spyware would be unwieldy for both anti-spyware vendors and users. While anti-spyware start-ups can build their engines from the ground up with these conditions in mind, anti-virus vendors are forced to tinker with existing architectures.

That said, the incumbent anti-virus vendors are moving into the spyware market with a vengeance. An enhanced engine is being added to Symantec's AntiVirus Corporate Edition 10.0 to help repair and remove spyware on host machines. The new version will be available at the end of March. McAfee is also releasing an add-on module specifically for spyware detection and removal. The module, which snaps into AntiVirus 8.0i, is available for an additional charge (see table).

Trend Micro's OfficeScan 6.5 can intercept spyware being downloaded on host machines, but it doesn't have the ability to remove existing spyware, making it only half a solution. Trend says a removal capability is likely to be rolled out later this spring.

Of course, not everyone is willing to wait for the anti-virus giants to catch up. A case in point is Edward Bailey, director of IT for the Materials Science and Engineering Department at the University of Florida. He turned to Webroot's Spy Sweeper Enterprise to weed out malware on 100 desktops that McAfee's AntiVirus 8.0i wasn't catching.

"Computers were slowing down and crashing," says Bailey. "Webroot cleaned out the computers in about an hour. Now we scan and update every day, and that's the last of the problems I've had with those desktops."Other anti-spyware start-ups are also finding success in the market. Yahoo used a software development kit from PestPatrol (which was recently acquired by Computer Associates) to build a free anti-spyware tool. Sygate licenses detection and removal technology from Lavasoft, which makes the popular Ad-Aware detection and removal software, to power the spyware engine in its Sygate Secure Enterprise suite.

The newest entry to the anti-spyware market is Microsoft. In January, the company acquired Giant Company Software, which made a consumer anti-spyware product that included a user-driven forum known as SpyNet for reporting suspected spyware to Giant's researchers.

The current product, known as Microsoft AntiSpyware, is being offered as a free beta to consumers, but has been hardcoded to expire at the end of July. Microsoft says that on or before the expiration date, it will announce whether the software will remain a free product. The SpyNet forum is still active.

At press time, Microsoft wouldn't comment about a corporate version of AntiSpyware, but Paul Bryan, director of product management at the company, acknowledged that spyware was both a consumer and an enterprise problem. Given the company's plans to enter the anti-virus market, a corporate anti-spyware product seems likely. But even if Microsoft enters the enterprise anti-spyware market, it doesn't mean game over for everyone else. For one thing, Microsoft's competitors have a significant lead against the Redmond giant. It's also not clear that enterprise buyers would buy a security cure from a company whose OS and applications are such a significant enabler of the disease.

The upsurge in activity in the anti-spyware market may also have masked an important point: Anti-spyware technology is still immature, and security departments are factoring this into their buying decisions."We're waiting for a spyware solution with a strong management solution," says Gene Fredriksen, vice president of information security at Raymond James, a financial services company. "I can't afford to manage 15,000 separate instances of a product. I need a good control panel."

In addition to management concerns, no anti-spyware tool has proven to be 100 percent effective in identifying all the spyware on PCs. Like the early days of anti-virus, IT administrators often find themselves turning to multiple products to detect all instances of unwanted programs.

HIPS: OH BEHAVE

HIPSs are drawing interest from enterprises for two reasons. First, they provide real-time protection against attacks. By contrast, signature detection is reactive, meaning the attack has to appear before vendors can create a signature to detect it, and enterprises have to wait for those vendors to distribute the signature before they can start blocking the attack.

Second, HIPSs provide coverage for endpoints that haven't been patched. Because a HIPS can block both known and unknown exploits, administrators can test and deploy software fixes during regular maintenance windows instead of during emergencies.Five vendors with HIPS products are battling for market domination: McAfee, Cisco Systems, Sana Security, WholeSecurity, and eEye Digital Security.

The dominant technology for HIPSs is behavioral analysis, which uses various methods to examine the kinds of actions taken by a program or application. Actions that appear malicious, such as attempting a buffer overflow or opening a network connection, will trigger the HIPS agent. Behavioral analysis can catch malicious programs without the need for signatures, making it ideal for zero-day attack detection.

Behavioral analysis agents sit between the applications and OS kernel where they monitor system and API calls to file, network, and registry sources. They correlate system call behavior to a set of rules that define inappropriate behavior and can make real-time decisions whether to allow or deny an operation. Entercept, Cisco, and Sana sell behavioral analysis HIPS software.

WholeSecurity and eEye take a different approach to HIPS technology. Rather than intercept system calls, WholeSecurity's Confidence Online uses Windows APIs to learn how processes should behave. It then uses dozens of detection modules to examine active processes on an endpoint for behaviors that might indicate a malicious program. For instance, modules will check if the program attempts to log keystrokes, perform screen captures, or open a communications channel.

With Confidence Online, each potentially malicious behavior is assigned a score. Once these are tallied, the total score can trigger a response mechanism. Responses include alert-only, which sends a report to the management console; disable, which prevents the process from running unless the machine is rebooted or the user manually restarts the process; and permanently quarantine, which prevents the process from running ever again. Only administrators can unquarantine a program.eEye's Blink agent doesn't intercept system calls, either. Rather, after examining thousands of attacks, eEye has determined the most common exploit methods used and condensed them into a rule set for Blink. As packets and frames come into the PC from the network stack, Blink matches them against its rule set. Based on the number and type of rules violated, the agent can then drop potentially harmful packets and log the action.

While eEye wants to differentiate Blink from its system call brethren, version 2.0 of the product hedges its bets by also including a buffer overflow protection module similar to those found on Entercept and other system call interceptors. The module examines program processes as they're loaded into the computer's memory. If a program attempts to overwrite a buffer, Blink will kill the process.

Other vendors, such as Sygate, Symantec, McAfee, and Check Point, are also bundling HIPS capability with other products. For instance, Check Point's Integrity 6.0 is an integrated security suite that includes a HIPS. The behavior-based analysis engine, called the Malicious Code Prevention (MCP) module, decompiles network traffic coming into the PC and looks for patterns that may indicate the presence of a buffer overflow. Buffer overflows are the most common exploit used by malware to install programs or gain control of a target machine.

HIPS vendors know their claims about stopping unknown attacks are seductive. To resist being seduced into a buying decision, remember to keep an eye on the drawbacks. At the top of the list are false positives. HIPSs may stop valid programs from running, which will generate help desk calls and require you to create exception lists for applications on your desktops. (One customer reported that his HIPS blocked his patching software.) Every HIPS product will cause false positives, especially on initial deployment, so be wary of any vendor that claims otherwise. During product evaluation, look for a management interface you can be comfortable with because you'll likely be spending a lot of time with it.

Another issue with HIPSs is that without signatures to identify zero-day attacks, administrators have to puzzle out for themselves whether processes are malicious or not. Reporting may not be as intuitive as the person manning the console would like.HIPSs also aren't a replacement for anti-virus and anti-spy software. They won't catch macro viruses, file infectors, boot sector viruses, and e-mail worms because these classes of malware tend to operate inside known good applications. Also, HIPSs can't prevent malware from being loaded onto a machine; they have to wait until a program executes before they can check for malicious behavior. And while HIPSs can catch keystroke loggers, Trojans, and other malware that gets lumped into the spyware category, they have difficulty identifying user-tracking adware. Lastly, HIPSs don't remove any of the malware they detect. While they can prevent malicious programs from functioning, you'll need other tools to get the malware off your PCs.

INTEGRATED SUITES--SWEET!

One of the problems with PC security solutions is that they multiply the administrators' management burdens. Separate security solutions generate events that need to be collected, aggregated, and analyzed. Every security agent that sits on a machine requires policy, signature, and software updates, not to mention the licenses that need to be tracked. And of course, it goes without saying that deploying multiple solutions can be prohibitively expensive. In short, agent-based cures can almost be as much trouble as the disease.

2005 saw an explosion of new products that combine multiple functions into a single package, including anti-virus, firewall, HIPS, and anti-spyware features.

Andre Gold, director of information security at Continental Airlines, was looking into eEye's Blink 1.6 to run on a limited number of workstations. But after learning that version 2.0 was going to include a new anti-spyware capability, he decided to roll it out across 20,000 devices, including customer-facing kiosks, reservation servers, and corporate desktops and laptops.As the table shows, a popular combination includes anti-virus software to detect viruses and other malware and a personal firewall to control which ports the computer can use for network communication.

Many integrated suites also include application control. This feature lets administrators determine which applications are allowed to run on the computer. Subsequently, if an unknown application starts, the security agent can alert an administrator or simply prevent the application from running.

However, administrators would be wise to quiz vendors carefully regarding bundled solutions. For instance, many of the products listed in the table have anti-spyware capabilities, but you have to dig deeper to find out just what that means.

Check Point's Integrity 6.0, for example, can detect and quarantine some spyware (that is, prevent the spyware program from operating), but it can't actually clean the files off your machines--you'll need another product to do that. And as noted previously, while Trend Micro's OfficeScan 6.5 can stop spyware from being downloaded onto your machines, it can't stop any existing spyware from running (though it can tell you it's there).

You may also need to invest in other products to take full advantage of a desktop suite. eEye's Blink 2.0 includes host vulnerability assessment, so you can scan each of your hosts for problems that may lead to security exploits. However, to aggregate scan results and remediate the vulnerabilities you discover, you'll need eEye's REM management console.Andrew Conry-Murray, technology editor, can be reached at [email protected].

Risk Assessment: New Desktop Security Software

Anti-spyware and HIPSs aren't as mature as desktop anti-virus software. Anti-spyware still lacks enterprise-class management capabilities, and no product boasts 100 percent detection. HIPSs will generate false positives and require extensive tuning. Vendors are attempting to address these issues, but no clear leader has emerged.

Despite the maturity issue, anti-spyware and HIPSs can still be deployed in the enterprise, especially on PCs that face significant risks, such as laptops that spend significant time outside the corporate network. In particular, HIPSs can act as an ad hoc patch for PCs that may miss critical corporate software updates.Anti-spyware and HIPSs offer a higher level of protection for corporate PCs than anti-virus solutions can provide alone. Administrators can use them to enforce security policies regardless of the PC's location and help protect against zero-day threats.

Any software that tinkers with registries, excises executables, or generally deletes files runs the risk of damaging computers. Removing spyware can cause legitimate programs to crash. As for HIPS software, the greatest risk is that it will block legitimate applications and generate help desk calls from irate users.

*Assumes large, distributed, self-maintaining commercial enterprise with no special needs related to this technology