Virtual file system

A virtual file system (VFS) or virtual filesystem switch is an abstraction layer on top of a more concrete file system. The purpose of a VFS is to allow client applications to access different types of concrete file systems in a uniform way. A VFS can, for example, be used to access local and network storage devices transparently without the client application noticing the difference. It can be used to bridge the differences in Windows, Mac OS and Unix filesystems, so that applications can access files on local file systems of those types without having to know what type of file system they're accessing.

A VFS specifies an interface (or a "contract") between the kernel and a concrete file system. Therefore, it is easy to add support for new file system types to the kernel simply by fulfilling the contract. The terms of the contract might change incompatibly from release to release, which would require that concrete file system support be recompiled, and possibly modified before recompilation, to allow it to work with a new release of the operating system; or the supplier of the operating system might make only backward-compatible changes to the contract, so that concrete file system support built for a given release of the operating system would work with future versions of the operating system.

Implementations

One of the first virtual file system mechanisms in Unix-like systems was introduced by Sun Microsystems in SunOS 2.0 in 1985. It allowed Unix system calls to access local UFS file systems and remote NFS file systems transparently. For this reason, Unix vendors who licensed the NFS code from Sun often copied the design of Sun's VFS. Other file systems could be plugged into it also: there was an implementation of the MS-DOS FAT file system developed at Sun that plugged into the SunOS VFS, although it wasn't shipped as a product until SunOS 4.1. The SunOS implementation was the basis of the VFS mechanism in System V Release 4.

John Heidemann developed a "stacking" VFS under SunOS 4.0 for the experimental Ficus file system. This design provided for code reuse among file system types with differing but similar semantics ("e.g.", an encrypting file system could reuse all of the naming and storage-management code of a non-encrypting file system). Heidemann adapted this work for use in 4.4BSD as a part of his thesis research; descendants of this code underpin the file system implementations in modern BSD derivatives including Mac OS X.

Other Unix virtual file systems include the File System Switch in System V Release 3, the Generic File System in Ultrix, and the VFS in Linux. In OS/2 and Microsoft Windows, the virtual file system mechanism is called the Installable File System.

The Filesystem in Userspace (FUSE) mechanism allows userland code to plug into the virtual file system mechanism in Linux, FreeBSD, OpenSolaris, and Mac OS X.

In Microsoft Windows, virtual filesystems can also be implemented through userland Shell namespace extensions; however, they do not support the lowest-level file system access application programming interfaces in Windows, so not all applications will be able to access file systems that implemented as namespace extensions. KIO and the GNOME VFS provide similar mechanisms in the KDE and GNOME desktop environments, with similar limitations, although they can be made to use FUSE techniques and therefore integrate smoothly into the system.

ingle-file virtual file systems

Some virtual file systems are implemented in such a way that the illusion of a file system can be created by using access to a single file on the underlying file system. The primary benefit to this type of implementation is that it is centralized and easy to remove. A single-file virtual file system may include all the basic features expected of any file system (virtual or otherwise), but access to the internal structure of these file systems is often limited to programs specifically written to make use of the single-file virtual file system (instead of implementation through a driver allowing universal access). Another major drawback is that performance is relatively low when compared to other virtual file systems. Low performance is mostly due to the cost of shuffling virtual files when data is written or deleted from the virtual file system.

Implementation of single-file virtual filesystems

Direct examples of single-file virtual file systems include emulators, such as PCTask and WinUAE, which encapsulate not only the filesystem data but also emulated disk layout. This makes it easy to treat an OS installation like any other piece of software -- transferring it with removable media or over the network.

PCTask

The Amiga emulator PCTask emulated an Intel PC 8088 based machine clocked at 4.77MHz (and later an 80486SX clocked at 25MHz). Users of PCTask could create a file of large size on the Amiga filesystem, and this file would be virtually accessed from the emulator as if it were a real PC Hard Disk. The file could be formatted with the FAT16 filesystem to store normal MS-DOS or Windows filesref num|PC Drives|1ref num|PCTask|2.

WinUAE

The UAE for Windows, WinUAE, allowed for large single files on Windows to be treated as Amiga file systems. In WinUAE this file is called a "hardfile".ref num|WinUAE Hardfile|3

UAE could also treat a directory on the host filesystem -- (Windows, Linux, MacOS, AmigaOS) -- as an Amiga filesystem.ref num|WinUAE Add Directory|4

ee also

*9P - Distributed file system protocol that maps directly to the Plan 9 from Bell Labs VFS layer making all file system access network transparent.
*Toronto Virtual File System, a VFS for OS/2 developed by IBM Toronto that allows mounting diverse filesystems under a common structure.

Notes

# [http://www.simon.mooli.org.uk/AF/8.html Emulation on Amiga] Comparison between PCX and PCTask, Amiga PC emulators.
#See also [http://www.unitechelectronics.com/emul.htm This article] explaining how it works PCTask.
# [http://winuaehelp.back2roots.org/gui/hard-drives.htm Help About WinUAE] (See Hardfile section).
# [http://winuaehelp.back2roots.org/gui/hard-drives.htm Help About WinUAE] (See Add Directory section)

References

* [http://www.ibm.com/developerworks/library/l-sc12.html Put virtual filesystems to work]
* [http://www.arl.wustl.edu/~fredk/Courses/cs523/fall01/Papers/kleiman86vnodes.pdf Vnodes: An Architecture for Multiple File System Types in Sun UNIX]
* [http://www.cse.unsw.edu.au/~neilb/oss/linux-commentary/vfs.html The Linux Virtual File-system Layer]
* [http://www.science.unitn.it/~fiorella/guidelinux/tlk/node102.html#SECTION001120000000000000000 Linux kernel's Virtual File System]
* cite conference
first = R.
last = Rodriguez
coauthors = M. Koehler; R. Hyde
year = 1986
month = June
title = The Generic File System
booktitle = Proceedings of the USENIX Summer Technical Conference
publisher = USENIX Association
location = Atlanta, Georgia
pages = 260–269
* cite conference
first = M.
last = Karels
authorlink = Mike Karels
coauthors = M. K. McKusick
year = 1986
month = September
title = Towards a Compatible File System Interface
booktitle = Proceedings of the European UNIX Users Group Meeting
publisher = EUUG
location = Manchester, England
pages = 481–496
* Cite techreport
first=John
last=Heidemann
title=Stackable Design of File Systems
number=CSD-950032
institution=UCLA
year=1995
url=http://www.isi.edu/~johnh/PAPERS/Heidemann95e.html
* "The Linux VFS", Chapter 4 of "Linux File Systems" by Moshe Bar (McGraw-Hill, 2001). ISBN 0-07-212955-7
* Chapter 12 of "Understanding the Linux Kernel" by Daniel P. Bovet, Marco Cesati (O'Reilly Media, 2005). ISBN 0-596-00565-2

External links

* [http://www.scalingweb.com/embedded_file_system.php Embedded File System (EFS)] - Open Source cross-platform C++ implementation of Virtual File System
* [http://www.inf.bme.hu/~mszeredi/avfs/ AVFS] - A Virtual File System for mounting compressed or remote files
* [http://www.fs-driver.org/ fs-driver] Ext2 Installable File System for Microsoft Windows
* [http://www.viksoe.dk/code/gmail.htm GMail Drive] shell extension for Microsoft Windows
* [http://richard.jones.name/google-hacks/gmail-filesystem/gmail-filesystem.html GMail Filesystem for Linux] , based on FUSE (Linux)
* [http://www.ibm.com/developerworks/linux/library/l-linux-filesystem/ Anatomy of the Linux file system by M. Tim Jones]
* [http://www.boxedapp.com/ BoxedApp SDK] - a solution to create a private virtual file system
* [http://www.eldos.com/solfs/ Solid File System] - (SolFS) cross-platform single-file virtual filesystem with encryption and compression
* [http://www.eldos.com/cbfs/ Callback File System] - virtual filesystem for Windows that lets the developers create file systems in user mode
* [http://fuse.sourceforge.net/ FUSE - Filesystem in Userspace] - virtual filesystem for Linux. Latest file release: 2006-04-10
* [http://sourceforge.net/projects/lufs LUFS - Linux Userland FileSystem] - virtual filesystem with support of localfs, sshfs , ftpfs, gnutellafs, locasefs, gvfs, cardfs, cefs and more. Latest file release: 2003-10-29
* [http://jakarta.apache.org/commons/vfs/index.html Commons VFS] - virtual filesystem for Java, with support for Cifs, ftp, http, Zip (file format), Tar (file format), gzip, bzip2, and more.
* [http://millscript.org/projects/millscript-vfs/ MillScript VFS] - virtual filesystem for Java, influenced by the KIO subsystem in KDE, Steve Leach's work on a VFS in JSpice and to a limited extent the Apache Commons VFS.
* [http://developer.kde.org/documentation/library/3.5-api/kdelibs-apidocs/kio/html/index.html KIO] - (KDE IO) - a network-enabled file management system
* [http://www.flipcode.com/articles/article_vfs01.shtml flipcode - Programming a Virtual File System]