New Generation of Anti-Spyware Targets Network Safety

Promise: Anti-spyware vendors are developing behavior-based detection technologies that prevent spyware from reaching enterprise desktops without the use of signatures. The relentless pace of spyware development--and the irrepressible urge for users to install spyware-laden programs--may drive security architects toward proactive solutions.

Players: Much of the innovation in anti-spyware technology comes from a host of small companies, including Tenebril, Webroot, Aluria, Finjan, and Eset.

Prospects: Security architects and IT staff struggle daily with spyware and adware infestations and may thus be willing to experiment with proactive anti-spyware technology. Over time, they may grow more comfortable extending the approach to a broader class of malware.

Anti-spyware start-ups are rolling out proactive solutions that can stop new and unknown programs from invading PCs. Over time, anti-spyware software will likely evolve from threat-specific technologies into Host-based Intrusion Prevention Systems (HIPS) designed to protect desktops and laptops from a broad class of malware.Current HIPS software hasn't enjoyed widespread desktop deployment because of concerns over false positives and the complexities of policy creation for a diverse population of enterprise users. Proactive anti-spyware software may be a logical avenue for security architects to introduce HIPS-like technology to the enterprise.

It may also be a logical avenue for small, innovative vendors to gain a foothold in corporate desktops. One reason is that spyware and adware have an immediate and persistent impact on PC performance, and IT staff are desperate for relief. Another is that end users are often responsible for infestations. Despite years of warnings against clicking links and downloading software, users are the number-one reason why unwanted programs get installed on enterprise machines. Proactive blocking technology could save users from themselves.

ANTI-SPYWARE LEADS THE WAY

Spyware may be the entrance point of HIPS into the enterprise because it--and its sinister cousin, adware--cause IT administrators daily pain. Traditional HIPS solutions such as McAfee's Entercept, Cisco Systems' Cisco Security Agent, and Sana Security's Primary Response are sold primarily as protection against widespread malware outbreaks from zero-day worms exploiting OS or application vulnerabilities. These outbreaks are devastating when they occur, but also relatively infrequent. In addition, though traditional HIPS products are evolving to block spyware installations, at this point they can't remove spyware from infested PCs.That's a significant drawback, considering that IT departments are currently beleaguered by spyware and adware. These programs suck up processing power and can render PCs essentially inoperable. Besides affecting user productivity, spyware and adware drain time and money from the IT department. Jeff Pelot, CTO at Denver Health Hospital, knows firsthand. He says 25 percent of all his support calls were spyware- or adware-related, and that simply generating the help desk tickets to deal with infested PCs cost him $6,600 a month.

Part of the problem is self-inflicted. Some spyware and adware gets installed through drive-by downloads in which the user is blameless. But the great majority comes bundled with other software, including games, screen savers, file-sharing software, utilities, and add-ons such as weather trackers and emoticon generators. And it's the users who bring all this junk software onto the PCs.

EVOLVING PREVENTION

Most anti-spyware software includes some prevention features, such as the ability to stop Browser Helper Objects (which are companion applications for Internet Explorer and a popular vehicle for adware) from being installed, or create white lists of programs that are acceptable on the PC while blocking all others. They can also warn users if a program attempts to install itself or perform other behaviors, such as changing registry entries.

Anti-spyware software is also digging deeper into the OS. Aluria Software's corporate anti-spyware solution, Paladin, includes a kernel driver to prevent spyware programs from installing on PCs.However, these prevention measures rely on signatures and will only stop programs that already have a definition in the threat database.

At the same time, the need for proactive prevention is rising because spyware is getting more difficult to remove once it infects a PC. In fact, cutting-edge spyware is beginning to mimic rootkits. "We've seen some spyware that will hook the disk access API. So if you're scanning the hard disk, the spyware tells Windows API not to tell you it's there," says Mike Green, director of product management at anti-spyware maker Webroot Software. "If Windows won't let you see it, how can you delete it?"

Other spyware programs can monitor their own registry keys so that if a portion of the code gets removed, they can call home and get those portions reinstalled.

"The really good developers are reading Microsoft Systems Journal and looking at the same boards as virus creators to learn the darkest Windows API secrets to delve deeper and deeper into the OS," says Fred Felman, senior vice president of marketing at anti-spyware company Tenebril.

To address this problem, vendors are moving beyond signature detection by analyzing the behavior of unknown programs. This September, Tenebril will introduce SpyCatcher 4.0. In addition to using a database of known spyware definitions, the 4.0 version will add a kernel-based software agent that monitors system and API calls to look for potentially malicious behavior from programs that aren't listed in the spyware database.SpyCatcher's approach is identical to that of established HIPS products from McAfee, Cisco, and Sana. But unlike those products, SpyCatcher focuses exclusively on spyware and adware. It won't look for buffer overflows, which are commonly used by new worms and Trojans to gain entry into target machines.

"Eventually it may have applications in other areas, but right now we're just pursuing spyware with it," says Felman.

Webroot's Spy Sweeper Enterprise 2.5, which was released late this summer, uses heuristic analysis to prevent malicious behavior, such as changing a user's home page or resetting the host file to point to an unwanted Web site or spoofed versions of e-commerce or online banking sites. Webroot's Green says future versions of Spy Sweeper will also include kernel-based components to monitor system calls and provide more proactive protection against unknown attacks.

Other small vendors are pushing proactive solutions at the desktop. Eset, which makes the NOD32 desktop software, uses advanced heuristics to catch unknown spyware, viruses, Trojans, and other malware without signatures. An emulation engine on the desktop analyzes incoming programs from the Web and e-mail.

"You analyze the file to see what it's trying to do, see what it's calling--for instance, trying to hook a registry key at startup, or trying to install a Bowser Helper Object," says Andrew Lee, Eset's CTO. Lee says this emulation process adds about 6 percent of overhead to the desktop processing load. NOD32 backs up its heuristic analysis with a signature database.Panda Software's TruPrevent offers a full-blown HIPS solution that includes the ability to block new spyware and other malware. The software includes a signature-based IPS, a heuristic-based engine to analyze the intent of incoming programs, and a kernel-based rules engine to monitor file systems, registries, and active processes. Panda also claims to have an artificial intelligence engine that can correlate events from all the security components to help evaluate whether a piece of code is malicious.

Some products eschew behavioral analysis altogether when dealing with unknown programs. Host-based software from start-up GreenBorder creates a virtual environment that allows any untrusted executable to run without hooking into essential files and registries. At the end of the user session, these untrusted programs are simply flushed from the computer. (For more on GreenBorder, see "IPS Odyssey" July 2005)

Of course, the most significant drawback of behavioral-based prevention is the risk of false positives. Security architects must weigh the benefits of proactive security against potential disruptions in employee productivity and irate calls to the help desk.

Anti-spyware vendors also insist that signature-based detection will continue to be an essential weapon in their arsenals. Not only are signatures essential for removing known spyware from infected machines, but they also reduce the likelihood of false positives and help administrators track and report on remediation efforts.

ADVANCES AT THE GATEWAYAnti-spyware protection is also making its way to the network gateway. Several products, including Trend Micro's InterScan Web Security Suite, SurfControl's Web Filter, and Blue Coat Systems' ProxySG, can scan incoming Web traffic for spyware and adware. These gateway products also prevent end users from surfing to known spyware or adware sites and can stop adware or spyware on a PC from connecting to a remote server on the Internet. In addition, Trend Micro's InterScan can deploy an anti-spyware cleaner in an ActiveX control to clean desktops that have spyware or adware programs attempting to access the Internet.

Denver Health's Pelot turned to Blue Coat's gateway solution to protect approximately 50 PC kiosks deployed around the hospital. Adware and spyware infestations affected the performance of the PCs at the kiosks, which are used by doctors and other staff to track patient care and send prescriptions to the hospital pharmacy.

Pelot says before deploying Blue Coat, doctors might spend two minutes simply waiting to log on to the PC. With doctors seeing as many as 200 patients a day, they had little tolerance for poor-performing machines.

After testing the product, Pelot was satisfied enough to deploy it full time. "Depending on what a person is doing on the Internet, we may have to clean one or two machines. But the problem has virtually disappeared," he says.

Note that all gateway products suffer the same drawback--mobile users have no protection outside the corporate environment. Thus, enterprises with mobile workers should augment gateway solutions with a desktop-resident agent.BEHAVIOR BLOCKING AT THE GATEWAY

Blue Coat and other gateway solutions rely on signatures, which means new or unknown programs can still slip by. Security gateway vendor Finjan Software says that's not good enough.

Finjan offers a pair of appliances--Vital Security and a standalone anti-spyware version of it--that performs behavioral analysis of active content at the gateway. It compares the intended behavior of the active content--such as ActiveX controls, VBScript, JavaScript, or Java applets--with the content behavior policy defined by the enterprise.

For example, a customer may set a policy that prevents JavaScript or ActiveX controls from having access to file systems or registries on the desktop. The Finjan gateway will block active content that includes such capabilities.

Finjan's gateway doesn't create a sandbox to run the code inside. Instead, virtual scanners built into the gateway decompile each type of active content, examining commands and programmatic sequences to understand what the content will attempt to do when it executes. The result is that the gateway can detect new or unknown malicious programs and prevent them from infecting enterprise desktops.The product will introduce some latency to Web transactions, but Finjan declined to offer any figures. According to the company, the appliance does support load balancing to improve performance.

Finjan also recently announced that Microsoft had licensed several of Finjan's patents regarding its proactive content security used in the gateway. The patents will allow Microsoft to develop technology for preventing new and unknown attacks. Microsoft has also become a minority shareholder of Finjan.

Nick Sears, president of U.S. operations at Finjan, wouldn't say whether the deal was a prelude to an acquisition, but it does underscore the growing interest in prevention-oriented solutions. "The market is warming up to the notion that existing signature-based solutions aren't providing adequate malware prevention," says Sears. "Customers are looking to alternative solutions."

Technology Editor Andrew Conry-Murray can be reached at [email protected].