Device driver synthesis and verification

The device driver is a program which allows the software or higher-level computer programs to interact with a hardware device. These software components act as a link between the devices and the operating systems, communicating with each of these systems and executing commands. They provide an abstraction layer for the software above and also mediate the communication between the operating system kernel and the devices below.

The device driver.

Usually the operating systems comes with a support for the common device drivers and usually the hardware vendors provide the device driver for their hardware devices for most platforms. The aggressive scaling of the hardware devices and the complex software components has made the device driver development process cumbersome and complex. When the size and functionality of the drivers started increasing the device drivers became a key role in defining the reliability of the system. This has developed an incentive towards automatic synthesis and verification of device drivers. This article sheds some light into some approaches in synthesis and verification of device drivers.

Motivation for automatic driver synthesis and verification

The device drivers are the principal failing component in most systems. The Berkeley Open Infrastructure for Network Computing (BOINC) project found that OS crashes are predominantly caused by poorly-written device driver code.^[1] In Windows XP, drivers account for 85% of the reported failures. In the Linux kernel 2.4.1 device driver code accounts for about 70% of the code size.^[2] The driver fault can crash the whole system as it is running in the kernel mode. These findings resulted in various methodologies and techniques for verification of device drivers. An alternative was to develop techniques which can synthesize device drivers robustly. Less human interaction in the development process and proper specification of the device and operating systems can lead to more reliable drivers.

The other motivation for driver synthesis, is the large number of flavors of operating systems and device combinations. Each of these have its own set of input/output control and specifications which makes support of hardware devices on each of the operating systems difficult. So the ability to use a device with an operating system requires the availability of corresponding device driver combination. Hardware vendors usually supply the drivers for Windows, Linux and Mac OS but due to the high development or porting costs and technical support difficulties they are unable to provide drivers on all platforms. An automated synthesis technique can help the vendors in providing drivers to support any devices on any operating system.

Verification of Device Drivers

There are two challenges that limit testing the device drivers.

• It is very hard to determine the exact operation or time, when there is a fault in the interaction between driver and the kernel. The system could go in to some inconsistent state and the crash is reported after a long time, blurring the real cause of the crash.
• The drivers which work properly in normal circumstances can go wrong in rare and exceptional cases and the traditional testing techniques may not help in detecting the corner case behavior of the drivers.

The verification wave of device drivers, initiated by Microsoft through their SLAM project as early as the year 2000. The motivation for the project was that 500,000 crashes reported a day was caused by one video driver and thus became cautious about the great vulnerability in using complex device drivers, More details can be found here, in the speech delivered by Bill Gates. A large number of static and runtime techniques have since been proposed for bug detection and isolation.

Static Analysis

Static analysis means analyzing the program without executing to check whether it comply with the safety critical properties specified. For example, the system software should constrain themselves to rules such as "check user permissions before writing to kernel data structures", "referencing null pointer without check", "overflowing buffer size" etc. Using the traditional testing process (dynamic execution) requires writing many testcases to exercise these paths and drive the system in to error states. This process can take a long time and effort and is not a practical solution. Another approach is manual inspection but the code size is in millions of lines of code and too complex to be analyzed by humans.

Compiler Techniques

The rules that have a straight forward mapping to source code can be checked using a compiler. Rule violations can be found by checking if the source operations does not make sense. For example, rules like "enabling an interrupt after being disabled" can be checked by looking at the order of function calls. But if the source code type system cannot specify the rules in its semantics, then the compilers cannot catch those kind of errors. Many type safe languages allow memory safety violations resulting from unsafe type casting to be detected by compiler.

Another approach is to use meta-level compilation (MC),^[3] they extend the compilers with lightweight, system specific checkers and optimizers. These extensions are to be written by system implementers in a high level language which are dynamically linked to the compilers to do strict static analysis.

Software Model Checking

Software model checking is the algorithmic analysis of programs to prove properties of their executions.^[4] This automates the reasoning about the program behavior with respect to the given correct specifications. Model checking and symbolic execution are used to verify the safety critical properties of device drivers. The input to the model checker is the program and the temporal safety properties. The output is the proof that the program is correct or violation of the specification as a counterexample in the form of a specific execution path.

The tool SDV (Static Driver Verifier)^[5] from Microsoft uses static analysis for windows device drivers. The back end analysis engine SLAM used model checking and symbolic execution for compile time static verification. The rules that are to be observed by the drivers for each API are specified in a C like language SLIC (Specification Language for Interface Checking). The analysis engine finds all paths which can lead to violations of the API usage rules and are presented as source level error paths through the driver source code. Internally, it abstracts the C code in to a boolean program and a set of predicates which are rules that are to be observed on this program. Then it uses the symbolic model checking ^[6] to validate the predicates on the boolean program.

The model checker BLAST (Berkeley Lazy Abstraction Software verification Tool) ^[7] is used to find memory safety and incorrect locking errors in linux kernel. It uses an abstraction algorithm called lazy abstraction ^[8] to build the model from the driver C code. It has been successful in verifying temporal safety properties of C programs with up to 50K lines of code. It is also used to determine if a change in the source code affects the proof of property in the previous version and is demonstrated on a Windows device driver.

Avinux ^[9] is another tool that facilitates the automatic analysis of linux device drives and is build on top of bounded model checker CBMC.^[10] There exist fault localization methods to find the bug location as these model checking tools return a long counter example trace and it is hard to find the exact faulty location.^[11]

Run Time Analysis

Dynamic program analysis is performed by running the program with sufficient test inputs to produce interesting behaviors. Safe Drive ^[12] is a low overhead system for detecting and recovering from type safety violations in device drivers. With only 4% changes to the source code of linux network drivers they were able to implement SafeDrive and give better protection and recovery to linux kernel. A similar project using hardware to isolate the device drivers from the main kernel is Nook.^[13]They place device drivers in separate hardware protection domain called "nooks" and they have separate permission setting for each pages making sure that a driver does not modify pages which are not in its domain but can read all kernel data since they share the same address space.

Another similar work in this area is on automatic recovery of operating systems due to driver faults. MINIX 3 ^[14] is an operating system which can isolate major faults, defects are detected and failing components are replaced on the fly.

Device driver Synthesis

An alternative to verification and isolation of faults is to deploy techniques in device driver development process to make it more robust. Given a device specification and operating system functions, one method is to synthesize device driver for that device. This helps to reduce the human introduced errors as well as the cost and time involved in developing the system software. All the synthesis methods rely on some form of specification from the hardware device manufacturers and operating system functions.

Interface Specification Languages

Hardware operating code is usually low level and is prone to errors. The code development engineer rely on the hardware documentation which typically contains imprecise or inaccurate information. There are several Interface Definition Languages (IDL) to express the hardware functionalities. The modern OSes uses these IDLs to glue components or to hide heterogeneity, like remote procedural call IDL. The same applies to hardware functionalities as well. In this section we discuss writing device drivers in domain specific languages which helps to abstract the low level coding and use specific compilers to generate the code.

Devil ^[15] allows high level definition of the communication with the device. The hardware components are expressed as I/O ports and memory-mapped registers. These specifications are then converted to a set of C macros which can be called from the driver code and thus eliminates the error induced by programmer while writing low level functions. NDL ^[16] is an enhancement to Devil, describing the driver in terms of its operational interface. It uses the Devil's interface definition syntax and includes set of register definitions, protocols for accessing those registers and a collection of device functions. Device functions are then translated in to a series of operations on that interface. For a device driver generation, one have to first write the driver functionalities in these interface specification languages and then use a compiler which will generate the low level driver code.

HAIL (Hardware Access Interface Language) ^[17] is another domain-specific device driver specification language. The driver developer needs to write the following.

1. Register map description, which describes various device registers and bit fields from the device data sheet.
2. Address space description for accessing the bus.
3. Instantiation of the device in the particular system.
4. Invariant specification, which constraints accessing the device.

The HAIL compiler takes these inputs and translates the specification in to C code.

Hardware Software Co-design

In hardware software co-design, the designer specifies the structure and behavior of the system using finite state machines which communicate among themselves. Then a series of testing, simulation and formal verification are done on these state machines before deciding which components go in to the hardware and which of these in to the software. The hardware is usually done in field programmable gate arrays (FPGAs) or application specific integrated circuits (ASICs), whereas the software part is translated in to low-level programming language. This approach mostly applies in embedded systems which is defined as a collection of programmable parts that interact continuously with environment through sensors. Existing techniques ^[18] are intended for generating simple micro-controllers and their drivers.

Standalone Driver Synthesis

In the standalone synthesis both the device and the system software are done separately. The device is modeled using any Hardware Description Language (HDL) and the software developer does not have access to the HDL specifications. The hardware developers put forth the device interface in the data sheet for the device. From the data sheet the driver developer extracts register and memory layout of the device and the behavioral model in the form of finite state machines. This is expressed in the domain specific languages described in the Interface language section. Final step involves generating the code from these specifications.

The tool termite ^[19] takes three specifications to generate the driver.

1. Device specification : The device register, memory and interrupt services specification obtained from the device data sheet.
2. Device class specification : This can be obtained from the relevant device I/O protocol standard. For example for ethernet the Ethernet LAN standard describes the common behavior of these controller devices. This is usually encoded as a set of events like packet transmission, completion of auto negotiation and link status change etc.
3. OS specification : This describes the OS interface with the driver. More specifically the request OS can make to the driver, the order of these requests and what the OS expects the driver in return for these requests. It defines a state machine where each transition corresponds to a driver invocation by OS, the callback made by driver or a protocol specified event.

Given these specifications the termite will generate the driver implementation that translates any valid sequence of OS request in to a sequence of device commands. Due to formal specification of the interfaces, termite can generate the driver code which holds the safety and liveness properties.

Another very interesting hacking effort is done by RevNIC,^[20] which generate a driver state machine by reverse engineering an existing driver to create inter-portable and safe drivers for new platforms. To reverse engineer a driver, it wiretaps the hardware I/O operations by executing the driver using symbolic and concrete executions. The output of the wiretap is fed to a synthesizer, which reconstruct a control flow graph of the original driver from these multiple traces along with the boiler plate template for the corresponding device class. They have ported some of windows drivers for network interfaces to other linux and embedded operating systems.

Conclusion

The various verification and synthesis techniques surveyed in this article have their own advantages and disadvantages. Like the runtime fault isolations have performance overhead, where as, the static analysis does not cover all classes of errors. The complete automation of device driver synthesis is still in its early stages and has a promising future research direction. The existence of many languages for interface specification would eventually consolidate in to a single format, which is supported universally by device vendors and operating systems team. This can result in complete automated synthesis of reliable device drivers in future.

References

1. ^ Archana Ganapathi, Viji Ganapathi and David Patterson. "Windows XP kernel crash analysis". In Proceedings of the 2006 Large Installation System Administration Conference, 2006.
2. ^ A. Chou, J. Yang, B. Chelf, S. Hallem and D. Engler. An Empirical Study of Operating Systems Errors. In SOSP, 2001
3. ^ Engler, Dawson and Chelf, Benjamin and Chou, Andy and Hallem, Seth. "Checking system rules using system-specific, programmer-written compiler extensions". In Proceedings of the 4th conference on Symposium on Operating System Design and Implementation, 2000
4. ^ Jhala, Ranjit and Majumdar, Rupak. "Software model checking". In ACM Computation Survey. 2009
5. ^ Thomas Ball, Ella Bounimova, Byron Cook, Vladimir Levin, Jakob Lichtenberg, Con McGarvey, Bohus Ondrusek, Sriram Rajamani. and Abdullah Ustuner. "Thorough static analysis of device drivers", In SIGOPS Oper. Syst. Rev, Vol. 40, 2006.
6. ^ McMillan, Kenneth L. "Symbolic Model Checking". Kluwer Academic Publishers, 1993.
7. ^ Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar and Gregoire Sutre. "Software Verification with BLAST". In SPIN, 2003.
8. ^ Thomas A. Henzinger, Ranjit Jhala, Rupak Majumdar and Gregoire Sutre. "Lazy Abstraction", In ACM SIGPLAN-SIGACT Conference on Principles of Programming Languages, 2002.
9. ^ H. Post,W. Küchlin. "Integration of static analysis for linux device driver verification". In The 6th Intl. Conf. on Integrated Formal Methods, 2007.
10. ^ Edmund Clarke, Daniel Kroening and Flavio Lerda. "A Tool for checking ANSI-C Programs". In TACAS, 2004
11. ^ Thomas Ball, Mayur Naik, and Sriram K. Rajamani. "From symptom to cause: localizing errors in counterexample traces". SIGPLAN Not.., 2003.
12. ^ Feng Zhou, Jeremy Condit, Zachary Anderson, Ilya Bagrak, Rob Ennals, Matthew Harren, George Necula and Eric Brewer. "SafeDrive: Safe and Recoverable Extensions Using Language-Based Techniques". In 7th OSDI, 2006.
13. ^ Michael M. Swift, Steven Martin, Henry M. Levy, and Susan J. Eggers. "Nooks: an architecture for reliable device drivers ". In 10th ACM SIGOPS, 2002.
14. ^ Jorrit N. Herder, Herbert Bos, Ben Gras, Philip Homburg, and Andrew S. Tanenbaum. "MINIX 3: a highly reliable, self-repairing operating system". In SIGOPS Oper. Syst. Rev. 40, 2006.
15. ^ Fabrice Merillon, Laurent Reveillere, Charles Consel, Renaud Marlet, and Gilles Muller. " Devil: an IDL for hardware programming". In Proceedings of the 4th conference on Symposium on Operating System Design & Implementation, Vol. 4, 2000.
16. ^ Christopher L. Conway and Stephen A. Edwards. "NDL: a domain-specific language for device drivers". SIGPLAN Not. 39, 2004.
17. ^ J. Sun, W. Yuan, M. Kallahalla, and N. Islam. "HAIL: A Language for Easy and Correct Device Access". In Proc. of ACM Conference on Embedded Software, 2005.
18. ^ Felice Balarin et. al. "Hardware-Software Co-design of Embedded Systems. The POLIS Approach." Kluwer Academic Publishers, 1997.
19. ^ Leonid Ryzhyk, Peter Chubb, Ihor Kuz, Etienne Le Sueur, and Gernot Heiser. "Automatic device driver synthesis with Termite". In Proceedings of the 22nd ACM Symposium on Operating Systems Principles, 2009.
20. ^ Vitaly Chipounov and George Candea. "Reverse Engineering of Binary Device Drivers with RevNIC". 5th ACM SIGOPS/EuroSys, 2010.

External links

• Future Chips: A website dedicated to hardware/software co-design
• Avinux, A Linux driver verification
• BLAST: Berkeley Lazy Abstraction Software Verification Tool
• Static Driver Verifier by Microsoft
• SafeDrive - Safe and Recoverable Extensions Using Language-Based Techniques
• Nook : Improving reliability of commodity Operating Systems
• BugAssist : A fault location tool
• Device driver reverse engineering
• HAIL A Language for Easy and Correct Device Access