Data Execution Prevention

Data Execution Prevention (DEP) is a security feature included in modern operating systems. It is known to be available in Linux, Mac OS X, and Microsoft Windows operating systems and is intended to prevent an application or service from executing code from a non-executable memory region. This helps prevent certain exploits that store code via a buffer overflow, for example.^[1] DEP runs in two modes: hardware-enforced DEP for CPUs that can mark memory pages as nonexecutable, and software-enforced DEP with a limited prevention for CPUs that do not have hardware support. Software-enforced DEP does not protect from execution of code in data pages, but instead from another type of attack (SEH overwrite).

DEP was introduced on Linux in 2000, on Windows in 2004 with Windows XP Service Pack 2,^[2] while Apple introduced DEP in 2006.^[1]

Mode of enforcement

Hardware enforcement

Hardware-enforced DEP enables the NX bit on compatible CPUs, through the automatic use of PAE kernel in 32-bit Windows and the native support on 64-bit kernels. Windows Vista DEP works by marking certain parts of memory as being intended to hold only data, which the NX or XD bit enabled processor then understands as non-executable.^[2] This helps prevent buffer overflow attacks from succeeding. In Windows, from version Vista, whether DEP is enabled or disabled for a particular process can be viewed on the Processes tab in the Windows Task Manager.

In some instances, enabling Data Execution Prevention can have the unintended consequence of preventing legitimate software from executing, typically exposing a hitherto unknown bug in the process. In these cases, the affected software needs to be remediated by its developer; in the short term it can be flagged as being allowed to execute code in those parts of memory, but this itself leads to a possible attack if the application isn't rigorous in validating data that is passed into a region of memory that is marked as being executable.

Microsoft Windows

If the x86 processor supports this feature in hardware, and if the system's BIOS provides support for this feature and it has been enabled, either by the manufacturer or the user,^[3] then the NX features are turned on in Windows, on a limited "OptIn" basis.^[4] This setting provides protection only for a limited set of Windows system and binary files. To achieve full protection, the user must choose either "OptOut", covering all programs and processes not specifically exempted, or "AlwaysOn", covering all such without exemption. These are configurable through the System Properties interface.^[5] If the feature is not supported by the x86 processor, then no protection is given. Outside of the x86 architecture, a version of NX also exists for Intel's IA-64 architecture that is supported by Windows.

Software enforcement

Software DEP, while unrelated to the NX bit, is what Microsoft calls their enforcement of "Safe Structured Exception Handling". Software DEP/SafeSEH simply checks when an exception is raised to make sure that the exception is registered in a function table for the application, and requires the program to be built with it. However, even though it creates an impression that software DEP is related to the prevention of executing code in data pages, it is a different form of protection.^[2]

Issues

Limitations

DEP provides some protection against attacks, but of course is not sufficient to protect against all forms of attack. Other security features such as address space layout randomization, structured exception handler overwrite protection (SEHOP) and Mandatory Integrity Control, can be used in conjunction with DEP.^[6]

Where code is executed at runtime—a JIT compiler is a prominent example—the compiler can potentially be used to produce exploit code (e.g. using JIT Spray) that has been flagged for execution and therefore would not be trapped by DEP^{[citation needed]}.

Compatibility

DEP occasionally highlights software problems, usually with older software that was not compiled and tested to take advantage of the feature. Users^[who?] have experienced problems using various command line commands that are a part of Microsoft's Services for Unix, which is included as part of Vista as well as Windows Server 2003 R2.

These problems may be masked by disabling DEP, but at the risk of increasing exposure of the system to malware. DEP can be turned off on a per-application basis, or turned off entirely for all non-essential Windows programs and services.^[7] Microsoft recommends that DEP not be globally disabled where an application crashes due to a DEP error. Instead, the author or vendor of the offending software should firstly be contacted for an updated version that does not violate DEP, or failing that, that DEP be disabled on an exception basis for the offending application only.^[8]

DEP is applied to an entire process, so even if an application runs perfectly with DEP, it may need to be disabled if a non-DEP compliant extension is added that runs in the same process space. For example, DEP-related problems can occasionally occur with DEP-compliant core operating system components such as Windows Explorer, Internet Explorer and Windows Installer as they support in-process third party extensions or plugins that may not be DEP-compliant.^[9]

Configuration

Windows

Performance Options dialog box in Windows Vista allows Data Execution Prevention (DEP) to be configured.

Four DEP modes are available in Windows. The OptIn and OptOut modes can be controlled through the Performance Options dialog box of System applet in Control Panel; the AlwaysOn and AlwaysOff modes can only be controlled through the boot.ini file on Windows XP, with the bcdedit command in the command prompt on Windows Vista or later,^[2] or by using the Microsoft TechNet Enhanced Mitigation Experience Toolkit.^[10]

OptIn

This setting is the default configuration for Windows XP, Vista and 7. In this mode, DEP is only enabled for limited Windows system applications and services. In general, applications are not forced by the Operating System to opt-in—applications that wish to opt in must explicitly submit a protection request.^[11] 64-bit Windows Vista and newer operating system are an exception: 64-bit applications and services are automatically protected in OptIn mode unless they submit an explicit opt-out request.^[12][13]

OptOut

This setting is the default configuration for Windows Server 2003 SP1. In this mode, DEP is enabled by default for all programs and services. A list of specific programs that should not have DEP applied can be entered using the System dialog box in Control Panel. Network administrators can use the Application Compatibility Toolkit to "opt out" one or more programs from DEP protection. Programs and services that wish not to have DEP protection must explicitly submit an opt out request.^[11][13]

AlwaysOn

This setting provides full DEP coverage for the whole system. All processes always run with DEP applied. The exception list, application compatibility shims and opt-out requests are disregarded.

AlwaysOff

This setting disables DEP entirely. All opt-in requests are disregarded.

The Boot.ini file parameter /noexecute=policy_level adjusts the DEP setting on older versions of Windows, where policy_level can be either OptIn, OptOut, AlwaysOn or AlwaysOff.^[2]

See also

• NX bit – list of names of similar techniques used in other operating systems
• Executable space protection
• Buffer overflow
• Heap overflow
• Stack buffer overflow
• Stack-smashing protection

References

1. ^ ^{a b} Engler, Tobias (22 February 2011). "„Schwere der Lücken ist besorgniserregend“" (in German). Mac & i. Heinz Heise. http://www.heise.de/mac-and-i/artikel/Schwere-der-Luecken-ist-besorgniserregend-1194782.html. Retrieved 22 February 2011. 
2. ^ ^{a b c d e} "A detailed description of the Data Execution Prevention (DEP) feature in Windows XP Service Pack 2, Windows XP Tablet PC Edition 2005, and Windows Server 2003". Microsoft. 2006-09-26. http://support.microsoft.com/kb/875352/EN-US/. Retrieved 2008-07-11. 
3. ^ "Microsoft Knowledge Base kb912923". 2007-10-07. http://support.microsoft.com/kb/912923. Retrieved 2009-03-14. 
4. ^ "Microsoft Knowledge Base kb875352". 2006-09-26. http://support.microsoft.com/kb/875352. Retrieved 2009-03-14. 
5. ^ "System-wide configuration of DEP". 2006-09-26. http://support.microsoft.com/kb/875352#5. Retrieved 2009-03-14. 
6. ^ http://pax.grsecurity.net/docs/aslr.txt elaborates
7. ^ Marc Liron. "Adding Software Exceptions In Data Execution Prevention (DEP)". Windows XP Update. http://www.updatexp.com/dep-exceptions.html. Retrieved June 8, 2006. 
8. ^ "Data Execution Prevention: frequently asked questions". Microsoft. http://windowshelp.microsoft.com/Windows/en-US/help/186de3d0-01af-4d4c-981d-674637d2f4bf1033.mspx. Retrieved 2008-09-15. 
9. ^ "Windows Installer custom action code must be compatible with Data Execution Prevention in Windows Vista". Microsoft. 2007-03-15. http://support.microsoft.com/kb/929710. Retrieved 2008-07-10. 
10. ^ "The Enhanced Mitigation Experience Toolkit 2.0 is Now Available". Microsoft TechNet. 2010-09-02. http://blogs.technet.com/b/srd/archive/2010/09/02/enhanced-mitigation-experience-toolkit-emet-v2-0-0.aspx. Retrieved 2011-02-17. 
11. ^ ^{a b} "SetProcessDEPPolicy Function". Microsoft Developer Network. Microsoft Corporation. 3 December 2009. http://msdn.microsoft.com/en-us/library/bb736299%28VS.85%29.aspx. Retrieved 5 December 2009. 
12. ^ Robert Hensing. "DEP on Vista exposed!". http://blogs.technet.com/robert_hensing/archive/2007/04/04/dep-on-vista-explained.aspx. Retrieved February 26, 2009. 
13. ^ ^{a b} Hensing, Robert (12 June 2009). "Understanding DEP as a mitigation technology (Part 1)". Microsoft Security Research & Defense Blog. Microsoft TechNet Blog. http://blogs.technet.com/srd/archive/2009/06/12/understanding-dep-as-a-mitigation-technology-part-1.aspx. Retrieved 5 December 2009. 

External links

• A description of DEP and how to configure it
• Guide to hardware-based DEP protection
• The "Understanding Data Execution Prevention" help topic incorrectly states the default setting for DEP in Windows Server 2003 Service Pack 1
• Easy steps to define DEP exceptions in Vista and XP