An improved method has been devised for defending a server or other computer against a code-injection attack, in which an attacker exploits a hardware or software vulnerability to inject harmful or otherwise unwanted code into an application program that is being executed and then causes the injected code to be executed. The present improved defense method provides for a secure and efficient implementation of instruction-set randomization (ISR), incorporating several advances beyond related prior methods that utilize ISR.

ISR is a theoretically strong approach to defending against a code-injection attack, irrespective of the nature of either the attack or the vulnerability that the attacker exploits. In a computer defended by ISR, an instruction set for the desired application program is created by a randomization algorithm. The encrypted instruction set is sent to an emulator for execution. The emulator is augmented to decrypt the instructions before execution. When an attacker exploits a vulnerability to inject code, the injected code is also subjected to the decryption process. Unless the injected code has been encrypted by means of the same key as that used in the defending computer (in effect, unless the attacker knows the encryption key), the decryption process transforms the injected code into, in essence, a random stream of bytes that give rise to an exception (e.g., invalid operation code or invalid address) when execution is attempted.

Overhead Levels of an Apache Web Server using the present method (SDT-Based ISR), an alternative method (SDT only), and native execution were measured for requests of various sizes. The overhead values plotted here are normalized to the corresponding native-execution values.
The security of ISR depends on the strength of the encryption process, the protection of the encryption key, the security of the underlying execution process, and the probability that an attempt to execute injected code will result in exception. The practicality of ISR is affected by the overheads in execution time and storage space introduced by the encryption and decryption processes. The improvements incorporated into the present method were made with consideration of both security and practicality (of which efficiency is an important component).

In the present method, ISR is implemented by software dynamic translation (SDT) using the Advanced Encryption Standard (AES), which has been approved by the United States government for protecting information classified at the SECRET level with a 128-bit key and at the TOP SECRET level with either a 192- or 256-bit key. The method does not require storage of the encryption key on the hard disk of the defending computer: the key is generated dynamically when the program is loaded. A further security benefit of this method is that a different key is used for each execution of an application program.

The SDT system used in this method provides a small, robust virtual execution environment for ensuring safe execution. The SDT system loads and encrypts the application program, decrypts the application-program instructions in preparation for execution, and determines whether the decrypted instructions are valid application instructions prior to execution. Hence, unlike related prior methods, this method does not involve reliance on detection of an exception during attempted execution of randomized injected code: instead, injected code is detected as it is prepared for execution.

In tests of the security of this method, vulnerabilities of various types were seeded into several popular server application programs and then attempts were made to exploit the vulnerabilities to inject code. In every test case, the injected code was detected and execution of the detected code was prevented.

The method has also been shown to be efficient enough (and, hence, practical) to be useful in protecting critical server application programs that are often targets of attacks. Measurements on an Apache Web server protected by this method showed a performance loss of only 5 to 15 percent relative to a natively executing Apache web server (see figure). Similar measurements on a domain name server protected by this method showed a performance loss between 5 and 10 percent.

This work was done by Wei Hu, Jason Hiser, Dan Williams, Adrian Filipi, Jack W. Davidson, David Evans, John C. Knight, Anh Nguyen- Tuong, and Jonathan Rowanhill of the University of Virginia for the Defense Advanced Research Projects Agency.

This Brief includes a Technical Support Package (TSP).
Thwarting Code-Injection Attacks Using SDT-Based ISR

(reference DARPA-0005) is currently available for download from the TSP library.

Don't have an account? Sign up here.

Defense Tech Briefs Magazine

This article first appeared in the October, 2007 issue of Defense Tech Briefs Magazine.

Read more articles from the archives here.