Networking In The Cloud

In discussing network design with my clients, I talk about how routers, firewalls and other network gear can be interconnected to provide a scalable and redundant IP network. These foundations of network design apply to networks composed of tangible components or the virtualized infrastructure that extends corporate data centers to the cloud. I find that the networking aspects of cloud computing are frequently overlooked or addressed as an afterthought. In this article, I'll share my thoughts and experiences on data center/cloud integration and discuss the network ramifications of some services to the cloud.

There's much hype associated with the cloud. Until late last year, I found the non-stop talk about the cloud a bit off-putting, especially when mentions of the cloud began showing up in TV ads. What changed for me? Two things. First, I started to deploy services in on Amazon Web Services (AWS), including my company's website. I started to see the value of the cloud, not just for the big cloud consumers such as Netflix and Foursquare, but also for enterprises and Web-based services providers.

Second, companies began approaching me about integrating their data centers into the cloud. This customer interest made me realize that the network design services I provide have a new application in constructing data center extensions to the cloud.

Why are companies moving services to the cloud, and why would you want to consider moving your critical business applications from your data centers to the cloud? I see three primary drivers of this shift: cost, speedy introduction of new applications/services and availability. The large cloud providers realize economies of scale in operating virtualized platforms that few enterprises can replicate.

These providers offer GUIs and APIs that allow users to spin up new virtual machines (VMs) and services in minutes. Even if you already use virtualization in your data center, you'd have to invest a lot of money in systems and tools to match the flexibility of the cloud.

The last driver is availability. You probably have redundancy in your data center design to keep applications available to the user base. Imagine if you could host applications from many different geographic locations and have the infrastructure managed by the world's leading experts of high availability services.

In the infrastructure-as-a-service (IaaS) model, the tenants of the cloud provider can select VM characteristics such as CPU, memory and storage. Administrators have complete control over the VMs; they choose the operating systems and applications. Some IaaS providers allow administrators to control networking resources. This control of the network positions users to integrate their corporate network with the cloud in a way that is largely invisible to the end users.

The largest and most capable IaaS provider is AWS, which I will be focusing on in this article. AWS' competitors can't match its networking features at the time of this writing. No other provider offers the networking flexibility of AWS' service. Tying your business to one vendor is obviously not ideal. Let's hope that the competition improves so that multiple IaaS vendors that can offer rich networking features. Ideally, IaaS providers will adopt open standards so that enterprises can avoid vendor lock-in and proprietary technology. If cloud standards are adopted, businesses will be able to easily migrate services between different providers' clouds.

AWS offers two cloud environments--Amazon Elastic Compute Cloud (EC2) and Virtual Private Cloud (VPC). EC2 is intended for delivering services to Internet users without data center integration. Web servers offering content is one example. VPC is better suited for integrating with your corporate network and users. By default, the VPC has no connectivity to the Internet unless explicitly configured.

Amazon makes data center integration possible by letting IT:

VPC has limitations that administrators should understand. The VPC supports only RFC1918 space within the VPC. If this presents problems for your network, you can use NAT in your data center to make the VPC appear to be numbered from another address space. AWS built the VPC to scale to massive size. To accomplish this feat, the engineers chose a Layer 3 (that is, IP) foundation for networking. A ramification of this decision is that VPC does not support broadcast and VLANs. Traffic separation must be done at the subnet level. Since enterprise networks rely heavily on VLANs for separations, this is a significant problem if you expect to port VLAN-centric designs to the cloud.

Let's turn to an example. You want to provision a set of VMs in the cloud to run your Web-based expense reporting system. Only users on your corporate network need to access the application. You need two Web servers and one database server. Data between your data center and your VPC will be encrypted using the IPSec protocol.

AWS's VPC Creation Wizard makes the configuration of this set-up simple. You'll need one unused subnet from the address space that you use on your corporate network. Since the 10.0.0.0/8 is often used, we'll go with the 10.10.50.0/24 subnet. The wizard will automatically create the VPC router, which is a virtual router. While you can't log in to this router as you would a physical router, you can make routing tables changes that affect how this router does its work. In this example, no routing table changes are needed; the wizard sets up routing automatically.

Next, you'll create the VPC gateway and IPSec tunnel to your data center. In your data center, you must have a router that supports IPSec and the Border Gateway Protocol (BGP). AWS has tested Cisco and Juniper routers. While other routers will probably work, I recommend using one of these vendors for at least the initial turn-up. I've seen several organizations spend days trying to get IPSec to the VPC working with other vendors' equipment. You probably have a Cisco or Juniper router lying around somewhere. Use it. AWS provides the IPSec and BGP configuration for these routers. You can attempt to use another router once you've confirmed that the tunnel is working.

Now your data center has been extended to the cloud. Your users will access the expense reporting application no differently than they would applications hosted in your data center. You can use the VPC for much more involved setups that include multiple subnets for public and private use. The VPC can be configured to allow users on the Internet to reach the subnets you specify. This is useful for deploying e-commerce and other customer-facing services. AWS allows you to create up to 10 VPN connections to data centers per VPC and provides a form to request more connections. I've never needed more than 10 VPN connections, so I don't know if Amazon approves all these requests. Administrators can create up to five VPCs per region. Traffic between VPCs must traverse your data centers; Amazon does not offer a method for direct VPC-to-VPC connectivity. Alternatively, you could roll your own VPC-to-VPC connectivity with open source VPN software such as Openswan and OpenVPN. This option can be very complex, and I wouldn't advise pursuing it unless your network engineers and sys admins understand how the failure of these homegrown tunnels will affect the services in the cloud.

If you take away one point from my article, let it be this: The integration of your data center with the cloud can't be performed by system administrators and application developers alone. Do your sys admins know how to configure IPSec and the BGP? Probably not, and potential mistakes makes on-the-job learning a major business risk. Your network engineers must work hand-in-hand with your systems team in all aspects of migrating to the cloud and maintaining cloud services. I've seen many application teams take ownership of the cloud without understanding the effects on networking. You pay your network engineers for their subject-matter expertise. Insist on their involvement.

Network engineers bring experience to the project that helps ensure a smooth user experience. You don't want your users to groan each time they hear that another application or service will be deployed in the cloud, because of slower response time or other issues that lead to a degraded experience. Moving services to the cloud has an impact on the network. Let's cover what that means.

Network characteristics such as latency and jitter play a more prominent role when users access services and applications in the cloud. The primary driver of latency is distance. Applications that once existed in a data center in the same building as employees may be located in distant cities. Applications must be capable of dealing with increased latency without affecting the user experience. Most business-class ISPs will have jitter SLAs of 4 milliseconds or less, so the change in jitter probably won't have the same impact as the latency increase. Of course, if your applications have stringent jitter requirements, you will have to assess how the minor increase affects your application.

Data confidentiality is critical in the cloud. As mentioned earlier, the AWS VPC has no connectivity to the Internet unless explicitly configured. Purchasing dedicated links from your data center to Amazon's data center would not be feasible or needed for the most companies. The use of IPSec tunneling across Internet circuits provides a secure, standards-based tunneling mechanism for encrypting data. You won't have to worry about your data being comprised as it is transmitted across the network.

Since connectivity to the cloud typically uses existing Internet connections, you must take the traffic between your data center and the cloud into consideration when doing network capacity planning. Incremental service deployment in the cloud gives you visibility into bandwidth needs. The lead times associated with many ISPs for circuit turn-up are long. For this reason, an abrupt and complete switch to the cloud could degrade service for extended periods until Internet circuits are upgraded to higher capacities.

You might be wondering if the cost of the increased bandwidth for cloud services outweighs the benefits of the cloud. In the majority of instances, I would argue that the added cost is minor compared with the cost savings introduced by running services in the cloud. On a per-megabit basis, bandwidth prices continue to decrease, particularly as businesses move to Ethernet access for the WAN. One location type that must be examined is branch offices connected at lowers speeds using access technologies such as T1 and DSL. Depending upon current bandwidth usage, you should evaluate your options for connecting the branch office with Ethernet or business-class cable services.

Another bandwidth-related issue is the implementation or modification of quality of service (QoS) policy. Applications that formerly did not have to contend for bandwidth within the enterprise may have to do so on WAN links that do not have the capacity of internal LAN links. I recommend purchasing sufficient Internet bandwidth such that packets are not dropped under normal conditions. QoS should be relied on in abnormal states, such as increases in traffic due to denial-of-service attacks and link failures.

Still unsure about the cloud? You can perform extensive testing on AWS's VPC for less than $100. You may find that the cost and availability benefits of the cloud make it a valuable tool for your IT infrastructure. Don't forget that transitioning applications and services to the cloud will not always be simple and painless, depending on the complexity of the service. Maintaining existing levels for security and user experience will require a lot of planning. Applications developers and sys admins can't do this alone. The addition of network expertise to integration team will help ensure successful migrations and day-to-day upkeep of your data center's extension to the cloud.