Hierarchical storage management

Hierarchical storage management (HSM) is a data storage technique which automatically moves data between high-cost and low-cost storage media. HSM systems exist because high-speed storage devices, such as hard disk drive arrays, are more expensive (per byte stored) than slower devices, such as optical discs and magnetic tape drives. While it would be ideal to have all data available on high-speed devices all the time, this is prohibitively expensive for many organizations. Instead, HSM systems store the bulk of the enterprise's data on slower devices, and then copy data to faster disk drives when needed. In effect, HSM turns the fast disk drives into caches for the slower mass storage devices. The HSM system monitors the way data is used and makes best guesses as to which data can safely be moved to slower devices and which data should stay on the fast devices.

In a typical HSM scenario, data files which are frequently used are stored on disk drives, but are eventually migrated to tape if they are not used for a certain period of time, typically a few months. If a user does reuse a file which is on tape, it is automatically moved back to disk storage. The advantage is that the total amount of stored data can be much larger than the capacity of the disk storage available, but since only rarely-used files are on tape, most users will usually not notice any slowdown.

HSM is sometimes referred to as tiered storage.

HSM (originally DFHSM, now DFSMShsm) was first^{[citation needed]} implemented by IBM on their mainframe computers to reduce the cost of data storage, and to simplify the retrieval of data from slower media. The user would not need to know where the data was stored and how to get it back; the computer would retrieve the data automatically. The only difference to the user was the speed at which data was returned.

Later, IBM ported HSM to its AIX operating system, and then to other Unix-like operating systems such as Solaris, HP-UX and Linux.

HSM was also implemented on the DEC VAX/VMS systems and the Alpha/VMS systems. The first implementation date should be readily determined from the VMS System Implementation Manuals or the VMS Product Description Brochures.

Recently, the development of Serial ATA (SATA) disks has created a significant market for three-stage HSM: files are migrated from high-performance Fibre Channel Storage Area Network devices to somewhat slower but much cheaper SATA disks arrays totalling several terabytes or more, and then eventually from the SATA disks to tape.

The newest development in HSM is with hard disk drives and flash memory, with flash memory being over 30 times faster than disks, but disks being considerably cheaper.

Conceptually, HSM is analogous to the cache found in most computer CPUs, where small amounts of expensive SRAM memory running at very high speeds is used to store frequently used data, but the least recently used data is evicted to the slower but much larger main DRAM memory when new data has to be loaded.

In practice, HSM is typically performed by dedicated software, such as IBM Tivoli Storage Manager, CommVault [1], VERITAS Enterprise Vault, Sun Microsystems SAMFS/QFS, Quantum StorNext, or EMC Legato OTG DiskXtender.

Use Cases

HSM is often used for deep archival storage of data to be held long term at low cost. Automated tape robots can silo large quantities of data efficiently with low power consumption.

Some HSM software products allow the user to place portions of data files on high-speed disk cache and the rest on tape. This is used in applications that stream video over the internet -- the initial portion of a video is delivered immediately from disk while a robot finds, mounts and streams the rest of the file to the end user. Such a system greatly reduces disk cost for large content provision systems.

Tiered storage

Tiered storage is a data storage environment consisting of two or more kinds of storage delineated by differences in at least one of these four attributes: Price, Performance, Capacity and Function.

Any significant difference in one or more of the four defining attributes can be sufficient to justify a separate storage tier.

Examples:

• Disk and Tape: Two separate storage tiers identified by differences in all four defining attributes.
• Old technology disk and new technology disk: Two separate storage tiers identified by differences in one or more of the attributes.
• High performing disk storage and less expensive, slower disk of the same capacity and function: Two separate tiers.
• Identical Enterprise class disk configured to utilize different functions such as RAID level or replication: A separate storage tier for each set of unique functions.

Note: Storage Tiers are NOT delineated by differences in vendor, architecture, or geometry except where those differences result in clear changes to Price, Performance, Capacity and Function.

See also

• Archive
• Automated Tiered Storage
• Backup
• Computer data storage
• Data proliferation
• Disk storage
• Information Lifecycle Management
• Information repository
• Magnetic tape data storage
• Repository (disambiguation)
• Storage virtualization

Implementations

• DFSMShsm for z/OS
• Atempo Digital Archive (ADA) (HSM available on Windows, Mac, Linux, UNIX featuring end-user initiated Archive) by Atempo
• CASTOR by CERN
• IBM Tivoli Storage Manager for Space Management (HSM available on UNIX (AIX, HP UX, Solaris) & Linux)
• IBM Tivoli Storage Manager HSM for Windows formerly OpenStore for File Servers (OS4FS) (HSM available on Microsoft Windows Server)
• HPSS by IBM
• QuickSilver by IBM
• OpenArchive OpenSource Archive Manager by GRAU DATA for Windows/Linux
• GRAU Archive Manager (GAM) by GRAU DATA for Windows/Linux
• Automated Tiered Storage (SmartMove) by Enigma Data Solutions for Windows/Linux
• DataGlobal ERS
• EMC DiskXtender, formerly Legato DiskXtender, formerly OTG DiskXtender
• Caminosoft Managed Server by Caminosoft Corporation for Netware/Windows/Linux
• Moonwalk [2] Moonwalk (Columbia and Eagle), for NetWare/Windows/Linux
• SAM-QFS
• StorFirst EAS (Enterprise Archival Software) by Seven10 Storage Software
• CommVault QiNetix DataMigrator
• Remote Storage Services by Microsoft (available on Server 2000 and Server 2003 only)
• SGI DMF [3] (Data Migration Facility)
• Symantec VERITAS Enterprise Vault (KVS acquisition and Veritas acquisition)
• QStar [4] Network Migrator - Server and Client based Data Migrator for Windows, Linux, UNIX and Mac
• Quantum StorNext Storage Manager [5]
• Compellent Data Progression Automated Tiered Storage [6]
• Hitachi Data Systems HNAS Intelligent Tiering [7]
• Hitachi Data Systems HDT (Hitachi Dynamic Tiering)
• Zarafa_(software) Zarafa Archiver (component of ZCP)
• PoINT Software & Systems GmbH
• OpenVMS