Thwarting Malicious Java Attacks

NEWS BLOGS FORUMS NEWSLETTERS RESEARCH EVENTS DIGITAL LIBRARY CAREERS

IMMERSE YOURSELF:

The JVM consists of three parts: the CL (Class Loader) the BCV (Byte-Code Verifier) and the SM (Security Manager). Each adds a level of security while executing Java applications or applets. For instance, when an HTML page containing a Java applet is launched, the JVM loads the applet and creates a sandbox for it. The applet then invokes the CL, loading all the necessary Java classes, before passing the byte code to the BCV. Local applications do not invoke the CL; instead the byte code is created by the Java compiler during compile time. The byte code is then passed on to the BCV as it is executed. Applet byte code must pass through the CL in order to make sure that it is valid Java code and meets the language specifications.
The BCV ensures that all byte cod e adheres to Java language specifications. Regardless of a program or applet's origin, all Java byte code must pass through the BCV. However, applets are precompiled byte code (not compiled by the browser). And so, an attacker could create byte code by hand, bypassing the built-in security mechanisms of the development environment. The BCV prevents this type of attack by scanning the byte code and locating potential security risks and poor language syntax. In addition, the BCV also checks for name space violations and object accesses that would violate the Java security model.
The last and most important component of the JVM is the SM, which is really a Java class. The SM has the final say in all potentially dangerous Java actions. It monitors file access, system I/O, network I/O, Class Loader instantiation, process/thread creation and access to Java class objects. When an applet performs one of the above actions, it first consults the SM for approval. The SM decides if the action is permissible based on t he origin of the application or applet. Whenever a possibly dangerous function is called from within the applet or application, the SM grants or denies access to specific resources based on the origin of the application or applet.
It's Who You Know The JVM also restricts access to applets and applications based on their origins. Applets that are embedded in Web pages are the most restricted, while local Java applications are trusted almost entirely without restrictions. However, users may wish to grant more access to certain applets. Granting greater access to these applets is possible by denoting specific applets as trusted. By default, all applets are untrusted and cannot access local information, such as network configuration and user information. Locally installed applications are trusted because they reside on the local hard disk drive.
Java assumes that users know what is on their hard disk and that in itself is a security risk. For example, if a user were to download a Java applet from th e Internet and execute it locally, outside of his or her Web browser, the JVM will grant access to local system information because its local origin makes it trusted. If the user instead viewed the applet from within a Web browser, the applet would have far less access to the local system. Additionally, it would be watched closely by the JVM's security mechanisms.

Locking Down Your Network In most cases, Java itself can detect rogue applets and programs before they do harm. But don't get too comfortable because most of the security risks posed are new to users and system administrators. Malicious and badly written applets can be grouped into three categories: denial of service, Trojan horses, and programs that intend to compromise your privacy and gain access to important and sometimes damaging information.
Denial-of-service applets eat system resources until all applications come to a grinding halt. These programs typically spa wn process thread after process thread until all of your system's available memory or processing power is in use for nonproductive purposes. Trojan horse applets penetrate your system's security mechanisms and place a seemingly friendly program onto your computer. In most cases, a user downloads a file with the desire to execute a new demonstration or application. Once the user willingly executes the program, it has complete and unrestricted access to the computer and its resources.
The remainder of hostile Java applets are concerned with gaining access to privileged information, such as your financial records or address books. Once they gain access to your computer the information is sent from the attacked machine back to a central location for retrieval by the author. Then your computer is susceptible to new attacks made possible by the retrieved information.
Placing the Blame By far the majority of the security risks associated with Java applets are cracks in Java's armor. Take a look at Sun' s "Chronology of Security-Related Bugs" (at java.sun.com/sfaq/chronology.html). You will find there are more implementation bugs than authentic flaws in the Java security model. The next most common attack using Java applets are denial-of-service programs. However, because of the communal nature of the Internet, these applets are quickly identified and advertised as malicious, often before any damage is done.
The point-to-point nature of network communications provides us with a common means of identifying the source of applets. If we remember where an applet originates, we can deny or permit access based on previous applets from the same location. In addition, a digital signature included within the applet can guarantee its validity.
Java is capable of using digital signatures to verify content during the download. Incorporated within JDK 1.1 and greater, Java offers the ability to use AuthentiCode signatures as well as X.509 version 3 certificates and Java's own JavaKey for signing JAR (Java Archive) files. JAR files are single files containing all class files and images and resources.
Standing Guard A much better alternative to client-side protection against Java applets is to secure your network's access points. Most corporate networks employ firewalls to guard against unauthorized traffic. However, a firewall alone will not protect your clients from requesting corrupt or hostile Java applets. With the addition of software at the firewall, it is possible to inspect Java content as it enters your network. This strategy will depend on the firewall vendor and your network's configuration.
With certain software, an administrator can allow or deny specific access to the network based on an applet's origin. Additionally, the administrator can restrict access to an applet based on its digital signatures or author information. Signing is particularly effective in identifying valid applets because signatures are quickly recognized and may be stored for future access.
Third-party software also ca n generate custom signatures using public knowledge of Java byte code. However, though these keys can indicate the origin of an applet, they also can be misleading and a potential security risk. Projects, such as the University of Washington's Kimera Project, are extending Java's security by implementing better byte-code verifiers. In turn, vendors like Finjan are creating products that use the byte code of Java applets to create security profiles. These profiles can determine the relative security risk of each applet when it enters your network for the first time. Finjan's product, SurfinGate, accomplishes this by examining the Java byte code and recording the resulting profile for future access. Of course, examining all Java byte code as it passes through your network will slow down delivery to the desktop. SurfinGate's recording capability eliminates unnecessary byte-code scanning.
Digital signatures also provide an active approach to protecting your network. But products like SurfinGate may not correct ly interpret all Java applets. The applet and byte-code databases that these products use for comparisons can become outdated and may not reflect new threats and attacks. Other products, such as Digitivity's Cage product, shield end users from malicious activity by isolating the execution of Java applets.
Cage accomplishes this by altering the appearance of downloaded content and directing all Java references through the Cage server, which resides outside the firewall. This method limits security risks, while letting applets function. Cage transparently transfers the results of the executed Java applet back to the workstation for the duration of the applet's execution via a proxy applet, which runs on the client's browser. If the applet is malicious, it is only capable of crashing the Cage machine.
Gregory Yerxa is an assistant NetWare administrator for the Computer-Aided Engineering Laboratory at the University of Wisconsin-Madison. He can be reached at yerxa@cae.wisc.edu.