Hold Your Breath for Novell Storage Services

At BrainShare '97 in Salt Lake City, Utah, Novell engineers demonstrated a new storage system called Novell Storage Services (NSS), which is scheduled to ship with the next release of IntranetWare. These engineers claimed that IntranetWare could mount any size NSS volume in less than one minute.

To prove their claim, the engineers periodically asked someone from the audience to shut down the busy server, which was running a utility that created 750 files per second. Then they turned on the server and remounted the NSS volume under a sign reading "Hold Your Breath," which begged the question, "Can the Novell engineers remount an NSS volume before you take your next breath?"

By the end of the conference, the utility had generated 400 million files on a single NSS volume on which three name spaces were installed. Over one billion directory entries (400 million per name space) pointed to these files. After a participant from the audience turned off the server, onlookers dutifully held their breath when an engineer stepped forward to remount the billion-entry NSS volume. Contrary to what you might expect, no one passed out. The NSS volume was remounted in only three seconds.

IT'S FAST--IT'S REALLY FAST

NSS volumes mount far more quickly than volumes in the existing IntranetWare file system. Currently, IntranetWare loads a volume's entire File Allocation Table (FAT) into memory, creating a Directory Entry Table (DET). When mounting an NSS volume, however, IntranetWare doesn't load a FAT into memory.

Instead of using FATs to organize data, NSS uses more sophisticated organizational structures called balanced trees, or B-trees, which provide better performance and consume less memory than FATs. B-trees enable the IntranetWare file system to retrieve any file blocks that are not in memory in four I/O cycles at most. B-trees also enable more file blocks to be stored in memory than FATs do, making better use of your memory.

NSS makes better use of your memory in another way as well. Under the existing IntranetWare file system, the larger a volume gets, the more RAM the volume requires to be mounted. For example, a 10 GB volume might require 160 MB of RAM, and a 100 GB volume might require 1 GB of RAM.

In contrast, the size of an NSS volume has no effect on the memory requirements. Whether an NSS volume is 100 KB, 10 GB, or 1 TB, the volume requires only 32 MB of RAM.

RAPID RESTART

IntranetWare can also repair NSS volumes far more quickly than it can repair volumes in the existing IntranetWare file system. Currently, IntranetWare records file operations (such as creating, modifying, and deleting a file) first to the DETs in memory and then, when the server is not busy, to the hard disk drive. If users make changes and the server crashes before these changes are written to the hard disk, you have a problem. To restore the server, you must run the VREPAIR utility, which scans the entire volume and may recreate FATs from scratch to match whatever data this utility finds on the volume. This process is time-consuming, especially if the volume is large.

In contrast, NSS writes all file operations directly to the hard disk and associates each step required to complete these operations within a transaction. For example, renaming a file actually involves several steps, including removing the file from the current directory and placing this file in a new directory. The transaction associated with renaming a file would list all of these steps.

NSS then records every transaction in a separate, carefully-managed journal on the hard disk and uses the journal to redo or undo transactions to restore a server. After a transaction is completed (that is, after each step necessary to perform a particular change has been fully written to the hard disk), the steps associated with the transaction are deleted from this journal. If the server crashes before the transaction is completed, you have little to worry about. To restore the server, NSS simply replays the journal, redoing transactions that were fully written to the hard disk to ensure that these transactions were performed correctly and undoing transactions that were only partially written to the hard disk.

Because the size of a journal is based on the rate at which users modify files rather than on the volume's size, journals tend to be small. As a result, NSS can replay the journal quickly--far more quickly than the VREPAIR utility can scan an entire volume. Thus, IntranetWare takes only seconds to restore an NSS volume of any size--and may take even less time than that. After a clean dismount, which leaves no significant journal entries to replay, an NSS volume remounts in one tenth of a second, regardless of the volume's size.

THE SKY'S THE LIMIT

NSS supports many more volumes per server and files per volume than the existing IntranetWare file system, which supports as many as 64 volumes per server and 16 million files per volume. In contrast, NSS supports an unlimited number of volumes per server and theoretically supports 264 files per volume--more files than you will probably generate during your lifetime. Novell engineers have already tested NSS volumes with as many as 500 million files.

In addition, NSS supports much larger files and volumes than the existing IntranetWare file system, which supports files up to 2 GB and volumes up to 1 TB. In other words, one volume in the existing IntranetWare file system is large enough to hold all of the data you could store on 50 20 GB tape cartridges.

NSS supports files that can hold at least eight times that amount: One NSS file is large enough to hold all of the data you could store on 400 20 GB tape cartridges. Specifically, NSS supports files up to 8 TB and volumes up to 32 zetabytes--that's 32 sextillion (a 32 followed by seven sets of triple zeros) bytes. In other words, one NSS volume is large enough to store more data than you can probably imagine and certainly more data than you can find on most of today's networks.

AN EYE TOWARD THE FUTURE

In its initial release, NSS will provide a major enhancement to the existing IntranetWare file system. However, NSS is designed to go even farther. For example, Novell engineers have demonstrated an NSS version of BorderManager and Oracle8, and plans for an NSS-powered Network File System (NFS) are currently under way. In addition, NSS can support all types of mass storage devices, such as today's DVD-ROM and DVD-RAM drives and tomorrow's holographic storage systems. As Prashant Shukla, NSS product manager at Novell, points out, NSS "has truly been architected with an eye toward the future."