Pointer swizzling

In computer science, pointer swizzling is the conversion of references based on name or position to direct pointer references. It is typically performed during the deserialization (loading) of a relocatable object from disk, such as an executable file or pointer-based data structure. The reverse operation, replacing pointers with position-independent symbols or positions, is sometimes referred to as unswizzling, and is performed during serialization (saving).

Example

For example, suppose we have the following linked list data structure:

struct node {
        int data;
        struct node *next;
};

We can easily create a linked list data structure in memory using such an object, but when we attempt to save it to disk we run into trouble. Directly saving the pointer values won't work on most architectures, because the next time we load it the memory positions the nodes now use may be in use by other data. One way of dealing with this is to assign a unique id number to each node and then unswizzle the pointers by turning them into a field indicating the id number of the next node:

struct node_saved {
        int data;
        int id_number;
        int id_number_of_next_node;
};

We can save these records to disk in any order, and no information will be lost. Other options include saving the file offset of the next node or a number indicating its position in the sequence of saved records.

When we go to load these nodes, however, we quickly discover that attempting to find a node based on its number is cumbersome and inefficient. We'd like our original data structure back so we can simply follow next pointers to traverse the list. To do this, we perform pointer swizzling, finding the address of each node and turning the id_number_of_next_node fields back into direct pointers to the right node.

Methods of unswizzling

There are a potentially unlimited number of forms into which a pointer can be unswizzled, but some of the most popular include:

• The offset of the pointed-to object in the file
• The index of the pointed-to object in some sequence of records
• A unique identifier possessed by the pointed-to object, such as a person's social security number; in databases, all pointers are unswizzled in this manner (see foreign key)

Potential security weaknesses

For security, such methods must be implemented with a great deal of caution. In particular, an attacker's presentation of a specially crafted file may allow access to addresses outside of the expected and proper bounds. In systems with weak memory protection this can lead to exposure of confidential data or modification of code likely to be executed. If the system does not implement guards against execution of data the system may be severely compromised by the installation of various kinds of malware.

Methods of protection include verifications prior to releasing the data to an application:

• That an offset does not leave the bounds of the data read.
• That a table of indexes and the records pointed to is similarly constrained.
• That identifiers are both unique, and if sensitive, encrypted.
• That all variable–length data is restrained to lengths not exceeding the actual allocation.
• That allocations are of reasonable size
• That allocations made that are not loaded with data read are cleared, or loaded with some specific pattern.

This list is incomplete; you can help by expanding it.

Methods of swizzling

Swizzling in the general case can be complicated. The reference graph of pointers might contain an arbitrary number of cycles; this complicates maintaining a mapping from the old unswizzled values to the new addresses. Associative arrays are useful for maintaining the mapping, while algorithms such as breadth-first search help to traverse the graph, although both of these require extra storage. Various serialization libraries provide general swizzling systems. In many cases, however, swizzling can be performed with simplifying assumptions, such as a tree or list structure of references.

The different types of swizzling are:

• Automatic swizzling
• On-demand swizzling

References

• Paul R. Wilson: Pointer swizzling at page fault time: efficiently supporting huge address spaces on standard hardware, ACM SIGARCH Computer Architecture News, Volume 19, Issue 4, pp. 6–13. June 1991.
• Alfons Kemper and Donald Kossman: Adaptable Pointer Swizzling Strategies in Object Bases: Design, Realization, and Quantitative Analysis (2.56 MB), The International Journal on Very Large Data Bases, Volume 4, Issue 3, pp. 519–567. July 1995.
• Derek Crawford: 'Derek's ABC of C', Volume 2, pp 340–343. June 1992.

External links

• Swizzle, from the Free On-line Dictionary of Computing