Based on the volume of media coverage, explosive growth of iSCSI storage area network (SAN) companies and my own experiences, I think it’s safe to say that NFS and iSCSI SANs have begun to prove their worth as valid storage platforms with VMware’s Infrastructure 3, and both are also a lot cheaper than 2 GB and 4 GB Fibre Channel SANs. This has made it very interesting to those who are in the lower-tier of purchasing power – namely the many small and medium-sized companies that can’t afford the initial acquisition costs (the bulk of my clientele, by the way). That brings in the great equalizer – open source NFS and software iSCSI solutions, though given a choice I prefer iSCSI, because it can support VM partitions.
One of these OSS iSCSI packages is called IET, the iSCSI Enterprise Target, and it’s rolled into a Linux distro called Openfiler, a distribution focused on being a storage platform. Before we all rush out and use Openfiler for all our needs, IET itself has had some problems with supporting VMotion, so it’s earned a bit of a bad rap. In fact, there are often wails of protesters screaming about how IET is unsupported by the good folks at VMware because of the way two SCSI commands are structured and how that can hose a VMotion move. Luckily, there’s some good news on that front.
The truth be told, the good news there is only half-good. IET’s release 0.4.5 addresses those problems. The better news is that this update is rolled into the latest patches for OpenFiler (which has a self-update option, just like any other distro). There’s a nice link in a blog here. The reason it’s only half-good… IET is still not supported, patch or no patch. Still, I’ve been using it in the lab, and it works just fine for me. I’ve done 1, 2, and 10 VMotion moves by manipulating DRS, both manually and automatically, and haven’t hit a snag yet. That said, it isn’t supported.
Hitting an Openfiler target with VI3.5 is easy. First, install OpenFiler, ESX and VirtualCenter and configure them. I will skip most of the details of basic setup, as it’s superflous. Openfiler’s install guide is here, but the Petri IT Knowledgebase has a specific article that covers soup-to-nuts here. I’ll post another below that’s a little shorter and to the point. You’ll need to create the virtual machine on the local disk of the ESX host (otherwise, what’s the point?) and configure it to take up just enough, but not too much of that precious internal disk space. In my example, I built a virtual machine with two virtual disks (you can do it with one, if you want). The trip from an unallocated lump o’ disk to iSCSI-enabled volumes went like this:
Part One: Disk Management
Set up your physical storage from the get-go. During setup, I chose NOT to do anything to the second drive – which is to say, I didn’t let Disk Druid touch it at all. When Openfiler boots, and you have your basic settings done, things like ntp, the admin password, etc., go to the Volumes button, and click on Physical Storage Mgmt. From there, create the extended and logical partitions you will need to get your iSCSI volumes up. The steps, once Openfiler is up, look a little like this:
The basic setup. System, and nothing else:
Creating the Extended Partition:
Creating the Logical (Physical) Partition:
The Partitons on the Second Drive:
During creation, I ran into a problem on both machines I did this build on – in each case I had created the logical partition, but OF didn’t see it. I had to go delete it, and create it again.
Part 2: Creating the LUN
First up, enable the server to act as an iSCSI Target. This is done from the Services tab, under Enable/Disable. In this example, I’m only using iSCSI, but if you wanted to enable NFS, this is where you would do so. Mine looked like this:
Then, go back to Volumes, under Volume Group Mgmt, and create the iSCSI volume group. In this example, I’m using the whole disk, but you don’t have to.
The before and after looks like this:
Next, head over to Create New Volume (one tab left). Creating a volume is, like the rest of the process, relatively straightforward. I made one volume, comprising the entirety of my one Volume Group. Having chosen the name”san” (how creative of me) for the process, I kept with the same simplicity for the volume I created.
After that, you are forwarded to the List of Existing Volumes tab. Congratulations, you have an iSCSI SAN. Now, about configuring that SAN…
Part 3: Security
There are two ways to lock down your SAN. One is via the network, and choosing only to allow certain hosts or networks access to the SAN. From the List of Existing Volumes tab, select the Edit link under the Properties column. You will then be presented with a screen called Volume’s Properties giving you a number of options for securing the LUN. I won’t dwell too much on the network access restrictions, as my screen captures will be different from other networks. The long and short of it is that in the General tab there is a sub-tab called Local Networks. You can create individual hosts and subnets there, and then allow or deny access to these networks on the Volume’s Properties tab.Using MS-CHAP to lock down who can connect to your target is a much smarter plan.
Part 4: Getting VMware to See Your LUN
This is extremely straightforward, and is no different than any other iSCSI implementation. The only thing I’ll mention here is something that is left out of many, many documents. Open the ports for iSCSI on your ESX hosts! In 3.5, it looks like this:
There are a lot of considerations that need to be addressed about what you keep there, chiefly around reliability and performance. For example, if you put virtual machines in the iSCSI SAN on the physical host, when the physcial host dies, it takes the SAN with it, along with all those virtual machines. Another consideration is networking. The long and short of it is this – if you have a whole lot of unused disk space on your ESX hosts, you can use Openfiler to put stuff there. As to what stuff, that’s entirely up to you. I personally prefer iso files and other junk that doesn’t need to go anywhere or do anything, and isn’t going to hurt me when something fails. Of course, you could also use NFS, which Openfiler supports.
The end result, Openfiler gets a solid 8 pokers from me. I give it a ten for being a soup-to-nuts full-featured distro that I can use at almost any client site for almost any purpose. If this were a storage column, it would remain a ten. It drops back because they haven’t yet earned a nod from VMware, which is something I would have expected them to pursue, considering their growth and media attention lately. What makes it frustrating is that Xinit systems, the company behind the project, makes storage appliances. Since they’ve been producing Openfiler since 2003, I think it’s about time they got on the move with this!