ISCSI vs. NFS for virtualization shared storage

Virtualization experts debate ISCSI vs. NFS for shared storage in a virtual environment.

Among the many decisions IT managers face when deploying server virtualization is what form of shared storage to...

use: block-based storage such as Fibre Channel and iSCSI, or file-based NFS?

According to storage expert Nigel Poulton, the vast majority of VMware deployments rely on block-based storage, despite usually being more costly than NFS. That is a testament to the performance and reliability that block-based storage provides, especially in mission-critical production environments.

At the same time, influential virtualization experts like Scott Lowe advocate strongly in favor of NFS storage, citing management simplicity, large datastores and the availability of cost-saving features like data deduplication on some NFS arrays.

Read the following columns to help you choose whether to pick block-based Fibre Channel or iSCSI shared storage, or an Ethernet-based NFS array.

Block storage better than NFS for performance, reliability
By Nigel Poulton, Contributor

Should you use block storage or file storage for your VMware environment?

There is never a perfect answer to a question like this. But in many cases the solution can be boiled down a tried-and-tested guideline: If you plan on doing real work with your VMware estate, the answer is block. If you just plan on playing around, the answer is file. Here's why.

Real work vs. playing around
When I say real work, I think of these common terms: production, critical, high-availability, online-trading, high-performance, mission-critical, data center, non-stop, zero-downtime.

On the other hand, playing around refers to anything that is not described by the above terms. For example, playing around might include scenarios like home networks, labs and maybe test and development scenarios where cost is king.

Performance and reliability
It is common for block storage to have its own dedicated high-speed network, giving it naturally superior performance and reliability. Dedicated networks mean predictability, less contention and usually higher throughput. Less contention reduces the risk of somebody else's network problem becoming your network problem.

Protocol offloads are also a factor in increasing performance. Fibre Channel and iSCSI host bus adapters (HBAs) normally provide protocol offloads. These HBAs perform protocol-related functions faster than the main CPU and also free up CPU resources for other core ESX tasks.

Another factor to note is that VMware has invested heavily in improving and tuning the iSCSI software initiator in vSphere 4.0, hugely enhancing its performance over version V 3.5.

Going with the pros 
Away from pure technological superiority, it is probably fair to say that the vast majority of production VMware deployments over the years have been on block storage -- especially Fibre Channel. When it comes to the crunch, that fact should give you the warm fuzzy feeling we all like when doing real work.

To confirm that point, I recently pinged some friends and colleagues who use and deploy VMware day in and day out to ask what they predominantly deploy. Interestingly, all of them deploy VMware on block storage the vast majority of the time!

I also put a quick poll up on my website recently asking the same question. As of Jan. 5, block storage has 72% of the votes, file storage has 10%, with the remaining percentages made up of options such as a mix of block and file.

The bottom line is that the most mature, well-known and trusted configurations are clearly block-based, despite the fact that block-based solutions are usually more expensive than file-based. So if you plan on doing real work with your VMware estate where you need performance, reliability and peace of mind, you should deploy on block storage. Don't be wooed by jazzy features offered by file-based approaches -- you will likely never use them.

Six reasons to use NFS for VM storage
By Scott Lowe

These are six reasons to choose Network File System (NFS) for your virtual machine (VM) storage over traditional block-based storage.

1. Simplified operational model. NFS offers a greatly simplified operational model versus traditional block storage. Resizing LUNs can sometimes be problematic -- both the LUN and the Virtual Machine File System (VMFS) datastore must be resized. Resizing NFS filesystems/exports is generally much easier. VSphere's new VMFS expansion functionality helps, but it's still generally more cumbersome than with NFS. In addition, the use of deduplication functionality (for those arrays that support this functionality) is far simpler and easier to use when used with NFS instead of block-based storage.

2. Larger datastores. While VMFS LUNs top out just shy of 2 TB in size, NFS has no such limits -- some arrays go as high as 16 TB.

3. Ethernet-based network infrastructure. NFS uses the existing Ethernet infrastructure. On the other hand, the same can be said for iSCSI, so that's an even draw between the two protocols.

4. Advanced functionality via filesystems. NFS can offer advanced functionality above what a traditional block device can offer because the storage device has control of the filesystem. Features like snapshots and clones are more widely supported on NFS than on block storage. For example, consider storage devices like the Sun Storage 7000 or the NetApp FAS, both of which offer nearly instantaneous snapshot functionality and space-conserved clones.

5. Open access. VMware designed VMFS from the ground up to support virtualized environments, but that makes VMFS a bit specific to VMware environments.This means that it can be difficult to gain access to VMFS datastores. NFS, on the other hand, is a mature cross-platform specification that makes it much easier to provide access to virtual machines for backup, replication or other purposes.

6. A bright future. A significant part of NFS' future lies in parallel NFS (pNFS), which provides a standard for scale-out storage involving multiple storage controllers (NAS heads) and multiple volumes. It's expected that pNFS will bring performance and throughput benefits to a wide variety of workloads, including virtualization.

Dig Deeper on Introduction to virtualization and how-tos