Centerfold: Civil Engineering Firm "White Lists" Vulnerable Apps

When Patton Harris Rust & Associates found out that Microsoft's Windows MetaFile (WMF) had a security vulnerability that might be exploited for zero-day attacks, PHR&A's IT team quickly removed the offending WMF dynamic link library file from its application whitelist.

"We took the DLL and put it in an unauthorized group so it couldn't run," says John Loyd, vice president and director of information technology for PHR&A, a multimillion-dollar Chantilly, Va.-based civil engineering and consulting firm with commercial, state and federal government clients. The strategy let the company take the unsafe app offline before an attack could take place, he said. When Microsoft issued a patch for the WMF vulnerability, PHR&A re-listed the DLL as an approved application.

Today's whitelisting is a new spin on an old-school (and some might say paranoid) security method that spells out which applications specific users and user groups can run. So if a user tries to access an unauthorized application, or if a rogue executable gets through to a workstation, the whitelist software blocks it. Only the defined apps and executables can run, period.

PHR&A has deployed SecureWave's Sanctuary whitelisting software, which uses ACLs (access control lists) to specify who can run what, as well as when they can run it and from where. The privately held PHR&A purchased the Sanctuary software last year after Loyd and his IT team became concerned the firm's antivirus tools could no longer keep on top of the increasing number of zero-day attacks that were flooding the Internet.

"We had no defenses for a zero-day attack," Loyd says. The firm previously scrapped an IDS (intrusion detection system) that flooded the firm with alerts but did little to secure its network.

But the new whitelisting software came with some baggage of its own. Remote users experienced excruciatingly slow boots when they started their workstations in the mornings. Turns out the 1-MB whitelist database was amassing about 2 MB's worth of data traffic over the company's already-strained frame relay WAN when the Sanctuary server at headquarters updated the workstations.

With help from SecureWave, PHR&A was able to initially remedy the problem, which was exacerbated by a combination of constraints imposed by the company's 384-Kbps remote frame-relay links and a 30-second time-out setting in the Sanctuary client software that had to be reconfigured. They later pinpointed the culprit--a problem with how Sanctuary interacts with its Ghost system-imaging and deployment software--which caused the workstations to constantly request updates from the Sanctuary server (and generate heavy traffic).Meanwhile, the firm is implementing a MPLS (Multiprotocol Label Switching)-based WAN service from Sprint that will upgrade its 16 sites to T1 WAN connections. PHR&A had asked its remote users to shut down their workstations on Friday evenings and reboot on Monday mornings.

"We had to educate users about the delay and make sure they're not doing cold boots every morning," Loyd says. Most of them still also log off their workstations when away from them for an extended period of time, he says.

No Pain, No Gain

Setting up PHR&A's whitelist was labor-intensive. "There's a lot of administration at first," Loyd says. The firm had to painstakingly list each and every executable it would allow, including permissions for specific groups of users. The database contains about 43,000 lines, most of which make up the firm's AutoCAD-based land development applications suite. "The AutoCAD suite is quite a monster," Loyd says. PHR&A hopes to whittle the database down by about 15,000 by weeding out any duplicate entries.

The administrative process has become more streamlined since the firm initially populated the whitelist. PHR&A regularly culls the whitelist to eliminate redundant user groups, for instance, and Sanctuary software wizards automate the Windows patch process. The firm now also uses VMWare's Virtual Machine for installing new software on its Windows XP Professional workstations.PHR&A also runs Sanctuary's Device Control tool, which protects the network from hand-held devices--digital cameras, USB memory sticks and so on--its users bring to the workstation for their jobs. Loyd and his team set permissions for accessing these devices, much the way they do with applications.

"We had our first SD card for a camera with a virus on it, which had come from someone's hard drive," Loyd says, and the firm's antivirus software stopped it in its tracks, he says. The Sanctuary Device Control tool also audits file-copying onto CD read/write disks. "We have it set to shadow all data copied to external devices--file names only," he says. "It can also shadow a full copy of the transferred files."

But like any other security technology, whitelisting alone is no panacea. "A whitelist doesn't protect you from the illegitimate use of a legitimate program," Loyd says. So if a user exploits an allowed executable and builds a tunnel to a hostile Web site, he says, SecureWave Sanctuary won't notice. But Sanctuary's shadow-logging feature, which audits any data transfer activities, does help the firm protect itself from internal data or intellectual property theft. "I'm protecting $15,000 to $20,000 an hour on that CAD workstation," both from malicious attacks as well as inadvertent deletions or breaches from the inside, says Loyd.

Most of the firm's Windows XP Professional workstations are outfitted with Sanctuary, and the firm's Sanctuary application server also runs an SQL database server in the background that houses the whitelist. But PHR&A is taking its time rolling Sanctuary out to the rest of its Windows 2000/2003 servers. "We're being careful with the servers," Loyd says, because they are critical systems, and there are complicated licensing issues with many of them.

The company also uses Sprint's Email Protection Service (SEPS), which includes antivirus and e-mail "scrubbing." Next for the firm will be a hardware-based firewall appliance on the other side of its server-based firewalls, as well as another layer of defense for its security architecture: an IPS (intrusion prevention system) to secure the wire side. The IPS will do what whitelisting can't: "It will help protect against illegitimate and inappropriate use of technology," Loyd says.[15 minutes]

John Loyd
Patton Harris Rust & Associates

John Loyd, 45, is director of information technology for Patton Harris Rust & Associates (PHR&A), a Chantilly, Va.-based civil engineering and consulting firm. He's responsible for the company's security strategy and architecture, as well as its overall IT infrastructure and planning. He's been with the company for 24 years and in IT for 20 years.

The trouble with today's security tools: "[Security vendors] are not offering a complete solution to the changing civil of security, and these products take a ton of our time. That's causing a mini-paradigm shift in where we spend our time and resources. Security has taken over our IT activities in the last three to five years."

Why measuring security ROI isn't possible: "It has to do with protecting yourself from the unknown. In some cases, it could pay for itself [right away], and in other cases, you may experience a benefit over time."How life imitates IT: "Communicating with young children is like communicating IT issues with [some] users."

Most bizarre IT inquiry: "We get people who want to work from home and open 200-MB CAD drawings on a dial-up or broadband connection, and they wonder why they can't. We end up the bad guys, because it can't happen."

What Loyd's co-workers don't know about him: "I've lived in Germany, Iceland, Alaska and Iran."

Favorite sports team: "The Washington Redskins."

Best geek joke: "I don't do geek jokes."Favorite hangout: "I've got three kids and I'm working on an addition to my house, so [we] don't get out much ... when we do, we like small music clubs."

Subject that makes him rant: "Political extremes."

Wheels: "Volkswagen Passat wagon. I like German engineering and the wagon is a nod to the family."

In Loyd's car CD player now: "The EARLIES, Bloodshot Records compilations and other alternative music."

Must-see TV: "I don't get to watch TV much, but I like Six Feet Under, Carnivale ... and mostly delayed, TiVo-based TV."Comfort food: "Thai."

After hours: "Working on my home renovation."

Dream job: "Chef."

Security technology is a tough sell. It's all about managing risk--a moving target--and few IT organizations can muster a meaningful return on investment (ROI) to help their case. "Security projects should be driven by reality--look at what's coming in through the network and who's coming in through the door and try to match the technology to your firm," says John Loyd, vice president and manager of IT for Patton Harris Rust & Associates (PHR&A), in Chantilly, Va.

PHR&A executives look to Loyd and the IT group to stay on top of the latest threats and technologies, so they didn't flinch when Loyd asked for approximately $35,000 to purchase the whitelisting software for the civil engineering and consulting firm. "They've been around long enough to see the benefits" of security technology and how we've dodged some bullets, he says.The firm's approach to IT purchases is a pragmatic one, and the budget for security projects isn't separated from the overall IT purchasing budget of around $300,000. "It's user-driven IT," Loyd says. "We try to fashion solutions that both protect [us] and let users do what they need to do."

The key is maintaining an ongoing dialogue with management on security issues and threats, Loyd says. If you have that regular dialog, you won't be stuck pitching a purchase while under the gun or after suffering an attack. "Don't start campaigning for a project the day you want to make the purchase," he says. "I've spent years going before management and explaining where the threats are and what to do about them, so when SecureWave came along, I could explain we needed it because of the changing security landscape."

Aside from preventing attacks from the outside, PHR&A's security concerns revolve around its intellectual property--large CAD files of surveys, plats and other engineering data for its clients--and maintaining safe and "clean" VPN connections for its remote users and two of its largest clients, a telecommunications firm and a power utility. PHR&A would be held liable for any damage if it were to inadvertently bring any malware to these customers' networks, Loyd says.