Structured storage

Structured storage (variously also known as "COM structured storage" or "OLE structured storage") is a technology developed by Microsoft as part of its Windows operating system for storing hierarchical data within a single file. Strictly speaking, the term "structured storage" refers to a set of COM interfaces that a conforming implementation must provide, and not to a specific implementation, nor to a specific file format (in fact, a structured storage implementation need not store its data in a file at all). In addition to providing a hierarchical structure for data, structured storage may also provide a limited form of transactional support for data access. Microsoft provides an implementation that supports transactions, as well as one that does not (called "simple-mode" storage, the latter implementation is limited in other ways as well, although it performs better).

Structured storage is widely used in Microsoft Office applications, although newer releases (starting with Office 2007) use the XML-based Office Open XML by default. It is also an important part of both COM and the related Object Linking and Embedding (OLE) technologies. Other notable applications of structured storage include MSSQL, the Windows shell, and many third-party CAD programs.

Motivation

Structured storage addresses some inherent difficulties of storing multiple data objects within a single file. One difficulty arises when an object persisted in the file changes in size due to an update. If the application that is reading/writing the file expects the objects in the file to remain in a certain order, everything following that object's representation in the file may need to be shifted backward to make room if the object grows, or forward to fill in the space left over if the object shrinks. If the file is large, this could be a costly operation. Of course, there are many possible solutions to this difficulty, but often the application programmer does not want to deal with low level details such as binary file formats.

Structured storage provides an abstraction known as a "stream", represented by the interface IStream. A stream is conceptually very similar to a file, and the IStream interface provides methods for reading and writing similar to file input/output. A stream could reside in memory, within a file, within another stream, etc., depending on the implementation. Another important abstraction is that of a "storage", represented by the interface IStorage. A storage is conceptually very similar to a directory on a file system. Storages can contain streams, as well as other storages.

If an application wishes to persist several data object to a file, one way to do so would be to open an IStorage that represents the contents of that file and save each of the objects within a single IStream. One way to accomplish the latter is through the standard COM interface IPersistStream. OLE depends heavily on this model to embed objects within documents.

Format

Microsoft's implementation uses a file format known as "compound files", and all of the widely deployed structured storage implementations read and write this format. Compound files use a FAT-like structure to represent storages and streams. Chunks of the file, known as "sectors" (these may or may not correspond to sectors of the underlying file system), are allocated as needed to add new streams and to increase the size of existing streams. If streams are deleted or shrink leaving unallocated sectors, these sectors can be reused for new streams.

Native Structured Storage

During the beta testing phase of Windows 2000, it included a feature titled Native Structured Storage (NSS) for storage of Structured Storage documents (like the binary Microsoft Office formats and the thumbs.db file Windows Explorer uses to cache thumbnails) with each "Stream" that makes up a document stored in a separate NTFS data stream. It included utilities that automatically split up the streams in a regular Structured Storage document into NTFS data streams and vice versa. However, the feature was withdrawn after Beta 3 due to incompatibilities with other OS components, and any NSS files automatically converted to the single data stream format. [cite web | url = http://www.windowsitpro.com/Article/ArticleID/13785/13785.html | title = What is Native Structured Storage? | accessdate = 2007-12-03]

Competition

A number of competing formats and technologies exist, including:
* CoreFSIF, cross-platform including Windows, Windows CE, Linux, Solaris. C++ and C# interfaces.
* [http://www.gemboxsoftware.com/CompoundFile.htm GemBox.CompoundFile] , for Microsoft .NET.
* [http://www.gnome.org/projects/libgsf/ GNOME Structured File Library]
* [http://jakarta.apache.org/poi/poifs/index.html POIFS] , for Java.
* [http://pole.berlios.de/ POLE]

References

External links

* For Linux: the [http://www.gnome.org/projects/libgsf/ GNOME Structured File Library] can read Microsoft structured storage files. Another option is [http://pole.berlios.de/ POLE] .
* For Java: [http://jakarta.apache.org/poi/poifs/index.html POIFS] – Java implementation of the OLE 2 Compound Document format.
* For Microsoft .NET: [http://www.gemboxsoftware.com/CompoundFile.htm GemBox.CompoundFile] – .NET component for accessing structured storage files.
* [http://user.cs.tu-berlin.de/~schwartz/pmh/guide.html FYI: LAOLA file system, Mar/23/97]