Bright Computing Targets Cloud Bursting

San Jose-based Bright Computing is looking to take cloud bursting to the masses with Bright Cluster Manager 6.0. The new release incorporates technology to support cloud bursting, a technology that allows private clouds to scale out to public clouds when loads outgrow private cloud compute resources. It supports all Amazon instance types, including Cluster GPU Instances.

While cloud bursting may be relatively new, the basic concept of balancing workloads has been an ongoing concern for IT managers, especially those dealing with high-performance computing (HPC) applications. The age-old problem has always been one of scale--how to scale performance and capacity up or down based upon workloads, without overspending on technology or services. Simply put, compute cycles can be expensive, and no one wants to pay for under-used or, worse yet, unused capacity.

The company, which is known for its cluster management and HPC grid solutions, has partnered with Amazon to extend on-site clusters and servers into the Amazon EC2 cloud. Bright Cluster Manager 6.0 allows network administrators to control and manage those resources as if they are based locally, which simplifies cloud bursting and removes many of the barriers normally found when trying to dynamically add compute capacity.

"One important barrier to cloud computing adoption in HPC is data movement and data management," says Steve Conway, IDC research VP for HPC. "Cloud latency issues can make it challenging to ensure that the right data are in position when applications need to run on the cloud, and that the data are returned in a timely way afterward. Bright addresses this challenge by designing data-aware scheduling directly into the company's cloud-bursting capability. This integrated approach could make it easier for mainstream HPC applications to exploit cloud computing."

The Bright Cluster Manager 6.0 Data Aware Scheduling capability eliminates one of the primary drawbacks of cloud bursting solutions--lack of automation. Normally, running a compute job forces cloud computing users to manually initiate the data transfer, monitor the completion of this task, and then submit the job into the queue. Upon completion, the user needs to retrieve the output data. That process can be complex and consumes both user time and usage fees. By contrast, Bright Cluster Manager 6.0 runs the process as a single step, dramatically improving productivity and cycle times.

Bright Cluster Manager 6.0 provides cloud-readiness as a standard feature, enabling system administrators to expand capacity as needed or to evaluate the use of GPUs for applications. Bright provides two alternatives for cloud bursting: extending on-site clusters into the EC2 cloud and managing these nodes as if part of the local system; or creating entirely new clusters in EC2 and providing the full provisioning, scheduling, monitoring and management capabilities that Bright provides for local clusters.

The cluster extension scenario is particularly useful for users with varying workload demands on their clusters. Users can determine whether to run their applications locally, in the cloud or, for suitable parallel applications, split between the local cluster and the cloud. In all cases, the system administrator and user have complete visibility, as if the entire cluster was on-site.Bright removes the complexity from using the cloud for HPC. Much of the complex functionality is performed without the need for user intervention, such as the data-aware scheduling and automatic VPN creation. Bright Cluster Manager allows all instance types to be initialized with equal ease, including cluster compute instances that utilize hardware virtual machine (HVM) virtualization. The tasks are managed with Bright’s intuitive GUI, making it easy to get the most out of cloud bursting into EC2.

Not everyone has bought into the utility of cloud bursting. It is one of the more hyped use cases for hybrid cloud but people don't always stop to think about the ever-more-important "why," notes Network Computing blogger Joe Onisick, technical solutions architect at a large systems integrator. Why would I want to burst? What business advantage would I gain? Is bursting the right solution to the problem of capacity? Will bursting help me accomplish my organization's mission?

The answer to those questions will most typically be no, he says. While there are circumstances where bursting works and use cases for it, they will be the exception rather than the norm.

Last August Network Computing editor Mike Fratto took a closer look at somesignificant hurdles that you need to get over to succeed at cloud bursting. They range from the fact that most enterprise applications, including Web applications, are not written in a way that allows them to scale properly to the little matter of getting the application and data out to the cloud provider that you want to burst to in a timely manner.

Aside from cloud bursting, Bright Cluster Manager 6.0 is shipping with dozens of enhancements to its system management capabilities:

- Integration of openlava, the open source version of LSF, in addition to the existing integration of LSF, SLURM, PBS Professional, TORQUE/Moab, TORQUE/Maui, Univa Grid Engine and Open Grid Scheduler- Power-saving mode for use with SLURM and PBS Professional to lower operating costs- Full support for CUDA 4.1- Enhancements to automatic failover capabilities- ISON-based API in addition to the existing SOAP API, for ease of JavaScript integration- Support for Mellanox and QLogic OFED stacks- Booting over IB added for QLogic- Head node installation on existing Linux installations, in addition to standard bare-metal installation

"In building Bright 6.0, we listened closely to what our customers and prospects needed to benefit from cloud computing," says Martijn de Vries, CTO of Bright Computing. "They wanted to access cloud resources from their existing clusters and cluster management solution--not as a separate entity. They wanted it to be straightforward--no learning curve, no process changes. They wanted to solve the data management issue without jumping through hoops every time. They simply wanted to extend their local clusters, as needed, without becoming EC2 experts. I believe we have delivered."

Learn more about Strategy: Monitoring and Measuring Cloud Provider Performance by subscribing to Network Computing Pro Reports (free, registration required).