Next-Generation Malware On The Horizon

Computer security has always been a cat-and-mouse game. Just as IT deploys technology to plug the last round of holes, along comes a set of exploits opening up a new set. Indeed, that's exactly what's about to happen to the most recent weapon in the network security arsenal: next-generation application-aware firewalls. These devices worked well enough while the attackers played by the rules--in this case, properly formed TCP packets or well-behaved Web pages--but since when have malefactors been bound by rules?

At Focus 2013 last week, McAfee executives gave a powerful and alarming demonstration of so-called advanced evasion techniques (AET) designed to bypass even the best edge security devices. All are deviously creative and crafted to exploit weaknesses in the Internet's underlying technology. The first set operates at the network protocol level to bypass firewall and IPS systems by hiding malicious traffic within abnormal, but still compliant TCP/IP packets.

These exploits manipulate low-level IP packets by altering rudimentary parameters like TTL, packet length and sequence numbers, while fragmenting data streams in ways that still look normal and safe to a security appliance, but can be easily reassembled by malicious code on the host into an exploit.

Stonesoft, a recent McAfee acquisition, outlined the technical strategy of these new attacks in a paper on its IPS technology. For example, at the IP layer, a common evasion technique involves fragmenting malicious datagrams and then sending them out of order, only to be reassembled on the target client. The problem is that any edge security device that doesn't store the entire stream for reassembly prior to inspection is vulnerable. Sadly, many don't since it's now trivial to spread fragments across hundreds of packets, meaning security appliances would need big, fast buffers.

Things are no better at the TCP layer, where segments may arrive out of order and with different payload sizes; indeed, multiple copies of the same data may be sent if the receiving client doesn't acknowledge it. This allows attackers to send malicious code in arbitrary order while ignoring TCP flow control. At the application service layer, protocols like SMB/CIFS, MSRPC, Sun (ONC) RPC and even HTTP are equally subject to nefarious misuse.

While one class of evasion techniques operates entirely at the network-protocol layer another class also demonstrated at Focus works entirely within common applications using normal rules for Web traffic. These don't so much as trick network security software as bypass it. For example, one exploit demonstrated by McAfee CTO Mike Fey used steganography to embed a malicious binary payload within an innocuous image file. Once downloaded, the code/image must be decrypted, extracted and executed on the target.

Here's where HTML5, with its rich JavaScript and CSS support, comes in. Essentially, a small piece of HTML5 code on the malicious Web page can embed the necessary decryption instructions such that when the victim visits a website, an innocuous-looking picture is automatically downloaded. Malware is extracted and then executed via an embedded shell script.

Such exploits, particularly those using HTML5, are virtually impossible to detect and prevent using edge security. Even if an edge device managed to descramble and identify a known attack signature, sophisticated attackers know to subtly morph the bit-level details of each attack to evade hash/signature-based identification techniques.

[Even though traditional security systems aren't doing much to help us manage risk, we keep buying them. Read Michele Chubirka's analysis of the problem in "Security Snake Oil For Sale."]

But all is not lost as there are a number of techniques that can be integrated to detect and thwart all but the most sophisticated attacks. At the network edge, Stonesoft has pioneered security appliances that buffer, assemble and inspect even the most obfuscated packets and data streams before passing them on to the target system.

While these help--as Fey's Focus keynote demonstrated--the ability of attackers to easily build packet fragment permutations of arbitrary complexity means edge detection isn't perfect against a determined and sophisticated attacker. This is a key reason security expert and frequent InformationWeek contributor Michael Davis believes network-based approaches are not enough. "The endpoint is where the code is executed and it is where the analysis needs to be," he says.

A better approach builds detection and prevention intelligence into the client, where the actual malicious code executes. The problem here is implementing the strategy without bogging down clients under the overhead of increasingly CPU- and memory-intensive analysis code that must be executed in real time.

Enter the cloud, in the form of remote security software that does the heavy lifting in response to requests from client devices. McAfee describes this bifurcated architecture as thin client and thick cloud, where the endpoint has just enough intelligence to intercept local executable behavior and passes suspicious data onto central cloud back ends for sandboxing, static code analysis, reputation ranking, signing certificate verification and whatever else security experts can think of to validate the safety and integrity of the underlying code before it's executed on the client system. The beauty of this hybrid approach is that by offloading the computational effort to a central cloud service, it works equally well on a Windows laptop or Android smartphone.

According to McAfee, also important for improved cyber defense is integrating all security layers--such as SIEM, analytics, reputation databases or edge device detection--into a unified security control system. Indeed, this is a major component of McAfee's product strategy, its Security Connected Framework, and how it sells a homogeneous vendor strategy to prospective customers.

Integrating data from multiple security systems into one management and analysis system makes perfect sense, but the question for IT is whether it creates unintended consequences from vendor lock-in. Therefore, IT should demand open APIs and data exchange formats as security software providers develop integrated products.

AETs, HTML5 code distribution and other advanced malware techniques portend a new era of endpoint-centric security where edge defenses are increasingly marginalized with client-side code serving as the last and best defense. We'll be watching to see who builds the best distributed yet integrated security systems to replace decaying Maginot Line-style perimeter defenses.