HP EVI vs. Cisco OTV: A Technical Look

HP announced two new technologies in the late summer, Multitenant Device Context (MDC) and Ethernet Virtual Interconnect (EVI), that target private clouds. Mike Fratto outlined the business and market positions, particularly in regard to Cisco's Overlay Transport Virtualization (OTV) and Virtual Device Context. However, the technology is also interesting because it's a little different than Cisco's approach. This post will drill into HP's EVI and contrast it with Cisco's OTV, as well as with VPLS.

HP EVI supports Layer 2 Data Center Interconnect (L2 DCI). L2 DCI technology is a broad term for technologies that deliver VLAN extension between data centers. Extending VLANs lets virtual machines move between data centers without changing a VM's IP address (with some restrictions). The use cases for such a capability include business continuity and disaster recovery. For a more extensive discussion of L2 DCI, please see the report The Long-Distance LAN.

HP EVI is a MAC-over-GRE-over-IP solution. Ethernet frames are encapsulated into GRE/IP at ingress to the switch. The GRE/IP packets are then routed over the WAN connection between the data centers.

EVI adds a software process to act as control plane to distribute the MAC addresses in each VLAN between the EVI-enabled switch. Thus, the switch in data center A updates the MAC address table in data center B and vice versa. By contrast, in traditional use, Ethernet MAC addresses are auto-discovered as frames are received by the switch.

Because GRE packets are TCP/IP packets they can be routed over any WAN connection, making it widely useful for customers. In a neat bit of synergy, the HP Intelligent Resilient Framework (IRF) chassis redundancy feature means that WAN connections are automatically load-balanced because switches that are clustered in an IRF configuration act as a single switch (a Borg architecture, not an MLAG architecture). Therefore, multiple WAN connections between IRF clusters are automatically load-balanced by the control plane either as LACP bundles or through ECMP IP routing, which is a potential improvement over Cisco's OTV L2 DCI solution.

However, note that load balancing of the end-to-end traffic flow is not straightforward because there are three connections to be considered: LAN-facing, to the servers using MLAG bundles; WAN-facing, where the WAN links go from data center edge switches to the service provider; and intra-WAN, or within the enterprise or service provider WAN. Establishing the load balancing capabilities of each will take some time.

Because HP has chosen to use point-to-point GRE, the EVI edge switch must perform packet replication. Ethernet protocols such as ARP rely heavily on broadcasts to function. In a two-site network this isn't problem, but for three sites or more, the EVI ingress switch needs to replicate a broadcast EVI frame to every site. HP assures me that this can be performed at line rate, for any speed, for any number of data centers. That may be so, but creating full mesh replication for n* (n-1) WAN circuits could result in poor bandwidth utilization in networks that have high volumes of Ethernet broadcasts.

Next Page: Cisco's OTVCisco's OTV is also MAC-over-GRE-over-IP (using EoMPLS headers), but it adds a small OTV label into the IP header. The OTV control plane acts to propagate the MAC address routing table.

Like HP's EVI, OTV can complicate load balancing. Cisco's Virtual Port Channel (vPC) shares the control plane, while HP's IRF shares the data plane. Although a vPC-enabled pair of Nexus 7000 switches run as autonomous control planes, NX-OS can load balance evenly using IP. OTV load balances by using a 5-tuple hash and will distribute traffic over multiple paths for the WAN.

OTV also supports the use of multicast routing in the WAN to deliver a much more efficient replication of Ethernet broadcasts in large-scale environments. Instead of meshing a large DCI core, a Source Specific Multicast (with good reasons) should be more efficient for multiple sites. Badly designed applications, such as Microsoft NLB, will be much more efficient using multicast.

EVI Compared To MPLS/VPLS

For many enterprises, MPLS is not a consideration. MPLS is a relatively complex group of protocols that requires a fair amount of time to learn and comprehend. However, building mission-critical business services that aren't MPLS is really hard. Service providers can offer L2 DCI using their MPLS networks with VPLS. Operationally, enterprise infrastructure is diverse and customised to each use case. Service provider networks tend toward homogeneity and simplicity because of the scale.

Some enterprises will buy managed VPLS services from service providers. They will also discover that such VPLS services are of variable quality, offer poor loop prevention, and can be expensive and inefficient. (For more, see the above-referenced report.) This is what drives Cisco and HP to deliver better options in OTV and EVI.

Other EVI Claims

HP notes that its solution doesn't require "multicast in the default configuration." HP wants to contrast itself to Cisco's OTV, which uses Source Specific Multicast in the WAN core, because many network engineers might think configuring multicast to be too hard. Building an SSM design over a Layer 3 WAN core is a substantial requirement and not a technology that most enterprise engineers would be comfortable configuring. On the other hand, configuring SSM over a Layer 2 WAN core (using Dark Fibre or DWDM) is trivial.

However, Cisco OTV has a unicast mode that works in a similar way to HP EVI, which most engineers would choose for simplicity. That said, the SSM WAN core offers scaling and efficiency if you need it, while HP's EVI does not.

The HP EVI approach is potentially more effective at load balancing WAN circuits than 5-tuple hashing in Cisco OTV, but it's unlikely to make much difference in deployment.

EVI's Enterprise Value

HP EVI is aimed at enterprises with private clouds. The technology looks like a good strategy. HP says EVI will be available in the A12500 switch in December. HP has a poor history of delivering on time (we're still waiting for TRILL and EVB), so plan accordingly. Cisco OTV is shipping and available in the Nexus 7000 and ASR products (for substantial license fees). HP says it won't charge for EVI.

Private clouds are shaping up to be a huge market in the next five years, and HP is addressing this space early by bringing L2 DCI capabilities to its products. HP EVI looks to be a good technology to meet customer needs. Combined with Multitenant Device Content, it should keep HP on competitive footing with Cisco. Of course, it's easy to make this kind of technology work in a PowerPoint. We'll have to wait and see how it works in real deployments.

Note: Thanks to Ivan Pepelnjak from ipspace.net for input and proof reading.