The price of an extremely fast 10 Gigabit Ethernet (GbE) infrastructure has fallen dramatically. But when it comes to upgrading to a 10 GbE network, simply swapping out a few network interface cards (NICs) and cables won't suffice. To get the most bang for your buck, you may need to upgrade your entire network backbone.
Incorporating 10 GbE is a big decision -- one with associated costs and benefits. To help you understand the implications, here are three important points to consider before you upgrade a twork infrastructure and move a 10 GbE network.
Obviously, 10 Gigabit Ethernet upgrades your network bandwidth performance, but its faster throughput is useful only for servers and virtual machines (VMs) that need it. Many virtual hosts use single GbE in bonded configurations (where devices such as NICs are joined to increase bandwidth but appear as a single device), which provide good-enough performance for today's workloads. Additionally, virtual environment traffic can be segregated for storage needs, production networking and inter-server communication for management and live migration purposes.
From a cost perspective, your network utilization may be better served through the addition and bonding of single GbE network cards, rather than a wholesale upgrade to 10 GbE network hardware. Before making a decision, however, it's imperative that you figure out the amount of data that passes through your virtual hosts' network cards.
A 10 Gigabit Ethernet network still requires security zone isolation
A 10 GbE infrastructure can involve less virtual environment complexity. After all, many virtual hosts that run single Gigabit Ethernet require a larger-than-usual number of network cards to process various types of network traffic. You need separate NICs for management and live migration, for example. Additionally, you require separate sets of bonded NICs for production and storage networking.
Segregating traffic is fundamentally important because each traffic type has a different level of assigned trust. Storage traffic, for instance, requires special attention because of its tendency to consume bandwidth as well as its direct connection between servers and storage and the built-in limitations of the iSCSI protocol. Management traffic, on the other hand, is segregated to keep VM actions separate from regular network traffic.
A 10 GbE network doesn't eliminate the need for security zone separation, but it does provide a wider path for multiple zones of traffic -- if you trust it.
The use of virtual local area networks (VLANs) to segregate traffic is a topic of contention among security administrators. Some administrators suggest that some exploits allow the monitoring of VLAN traffic from another VLAN. Other experts, however, claim that those exploits are difficult to employ, and the risk of exposure -- particularly within a locked-down data center -- is relatively low.
The additional bandwidth provided by 10 Gigabit Ethernet results in fewer NICs, but this arrangement is possible only when internal security policies allow for traffic segregation through VLANs. If not, you may have the same number of NICs, including combinations of single GbE and 10 GbE.
Are you prepared to update your backbone?
Upgrading to a 10 GbE network usually requires an investment in your entire network backbone. It's not enough to simply install 10 GbE NICs into your virtual servers and expect those data speeds with a single GbE backbone.
As a result, consider where a 10 GbE upgrade makes the most sense in your network. Do the connections between your servers and storage experience the greatest bottlenecks? If so, consider physically segregating their network path with the intent of exclusively upgrading that path. But this upgrade may also require updates to your storage components so they can handle 10 GbE bandwidth.
If a bottleneck affects your production networking, on the other hand, are you prepared to upgrade your production network backbone to handle the increased throughput? Also, if you segregate traffic with a VLAN, will you interconnect that traffic through single GbE or 10 GbE network equipment? You should make these considerations after analyzing the existing network utilization for each virtual host traffic type. Then, make your decisions based on the specific traffic needs.
Does a 10 GbE network make sense for your virtual environment? Probably, if your network performance is less than stellar. Before updating your infrastructure, however, seriously weigh the benefits and drawbacks.
This was first published in August 2010