A friend of mine has until very recently worked in the storage industry writing drivers for Fibre Channel and iSCSI devices. This has granted those geeks among us who have access to this august personage a certain insight into the realities of what goes on at the storage layer. And in his words:

In general, I've found that working in the storage industry has greatly increased the value I see in a good backup strategy. I don't really trust any storage technology anymore.

And in more specific terms, he distrusts iSCSI as immature. When comparing Fibre Channel (FC) and iSCSI, he has waxed eloquent in his beliefs. For one, the fact that iSCSI runs over Ethernet and TCP/IP means that such problems long-solved at the network-driver level as out of order arrival and bad latency are essentially unknown to Operating Systems (Win2003 is new enough it MIGHT not qualify here) at the storage-driver level. For another, the dependency on the TCP/IP stack by the storage-driver is another new condition that can lead to deadlocking. In short, the industry is immature.

In his opinion, a correctly configured iSCSI environment would include:
  • Hardware based iSCSI cards that present to the OS as a storage adapter, not a NIC
  • A dedicated Ethernet segment for iSCSI
  • Guaranteed low latency on the iSCSI path
  • Guaranteed bandwidth on the iSCSI path
Quite a list. Especially when you see that one of iSCSI's key marketing points is that it isn't limited to distance the way FC is. The other key marketing point, price, is true; however, when correctly designed, iSCSI isn't nearly as cheap as it looks on first blush. The first item on that list is very key, as it presents to the OS as a device that was designed for storage and also is built to have its own TCP-stack to handle the networking issues.

Fibre Channel has several things going for it. For one, it has been around a number of years and has been well battle tested. Second, it uses a networking infrastructure that was designed from the ground up for storage-networking and its demands. Third, it is still far more reliable that iSCSI. Even if it is more expensive on a per-port basis.

And I agree with him on these. But as anyone who has attempted to get something expensive approved, the siren call of 'cheaper... Cheaper...' can deafen budgetary ears to the potential downsides.