A clustered file-system, for windows?

| 2 Comments
Yesterday I ran into this:

http://blogs.msdn.com/clustering/archive/2009/03/02/9453288.aspx

On the surface it looks like NTFS behaving like OCFS. But Microsoft has a warning on this page:
In Windows Server® 2008 R2, the Cluster Shared Volumes feature included in failover clustering is only supported for use with the Hyper-V server role. The creation, reproduction, and storage of files on Cluster Shared Volumes that were not created for the Hyper-V role, including any user or application data stored under the ClusterStorage folder of the system drive on every node, are not supported and may result in unpredictable behavior, including data corruption or data loss on these shared volumes. Only files that are created for the Hyper-V role can be stored on Cluster Shared Volumes. An example of a file type that is created for the Hyper-V role is a Virtual Hard Disk (VHD) file.

Before installing any software utility that might access files stored on Cluster Shared Volumes (for example, an antivirus or backup solution), review the documentation or check with the vendor to verify that the application or utility is compatible with Cluster Shared Volumes.
So unlike OCFS2, this multi-mount NTFS is only for VM's and not for general file-serving. In theory you could use this in combination with Network Load Balancing to create a high-availability cluster with even higher uptime than failover clusters already provide. The devil is in the details though, and Microsoft aludes to them.

A file system being used for Hyper-V isn't a complex locking environment. You'll have as many locks as there are VHD files, and they won't change often. Contrast this with a file-server where you can have thousands of locks that change by the second. Additionally, unless you disable Opportunistic Locking you are at grave risk of corrupting files used by more than one user (Acess databases!) if you are using the multi-mount NTFS.

Microsoft will have to promote awareness of this type of file-system into the SMB layer before this can be used for file-sharing. SMB has its own lock layer, and this will have to coordinate the SMB layers in the other nodes for it to work right. This may never happen, we'll see.

2 Comments

You hit all of the points that I was going to. Clustered filesystems are easy if you've got one lock per file and you know where it is at all times. It gets complex when you need a lock manager. Nice to see that it is an option, though. As you mentioned, near-instantaneous switchover would be great! I have to wonder how it would deal with the memory handoff in the event of an unexpected shutdown, though.

It's really not a general-purpose clustered filesystem at all. Only one node really mounts the filesystem at any moment in time, and that node handles all filesystem metadata operations and coordinates all disk I/O. As an optimization other nodes can get permission to directly read or write data blocks to shared storage, so that in a SAN environment they can avoid funneling all the read and write data through one node like you would if you were just using CIFS, but that's really all it's doing, and that's why they didn't need to change the filesystem format at all or implement a real lock manager.