Unified Communications

01:36 PM
Kurt Marko
Kurt Marko
Commentary
Connect Directly
LinkedIn
Twitter
Facebook
RSS
E-Mail
50%
50%

Big Data Means Big File Sharing

Need to move a few 60-MB CAD files cross-country? To the cloud, by way of Aspera's new virtual appliance.

In this age of Gmail, CloudApp, and Dropbox, it seems like sharing files across organizations and geographies would be a solved problem. Just email it, or upload it and send a link. But what if you're a video producer on location needing to send the day's clips, at about 500 GB per hour for 1080p video, back to Hollywood for editing? Or perhaps a medical researcher sharing with colleagues DNA sequences clocking in at 3 GB per genome, or even a road construction company getting 60-MB CAD file design updates from the local highway department?

Email is usually out of the question, since even Google limits attachment sizes to 25 MB. Ditto for Dropbox; even if you don't mind the $20-per-month price tag per 100 GB quota, the service caps uploads at 300 MB. Likewise for CloudApp, which caps file sizes at 50 MB. And even assuming you find a place in the cloud to share the data, you've still got to get it there over network pipes and file transfer protocols like FTP or WebDAV, which were designed for browser sessions or small downloads, not gigabyte uploads.

This is just the problem Aspera is looking to solve with its Aspera On-Demand virtual appliance, which facilitates moving data to and from Amazon Web Services at up to 350 Mbps, dependent on your WAN link. So, if you're an enterprise that has moved data- and processor-intensive applications into Amazon's cloud--particularly apps where users are constantly manipulating or updating the data--you now have a means of accelerating their access to near-LAN-level speeds.

From its roots in the media and entertainment market, Aspera is targeting the new offering to address the growing need for efficient "big data" transfer in industries ranging from defense to pharmaceuticals. Its first product, locally installed server software with an associated client application, amounted to little more than an FTP replacement--FTP 2.0 if you will, albeit one with sizable benefits, including orders-of-magnitude increases in throughput, in-flight AES encryption, and bandwidth-usage controls. But it's aimed at businesses with the IT resources and mega-file volume to justify purchasing yet another piece of server software and distributing another unfamiliar client application to end users. Smaller businesses that can't go that route but still need to collaborate on large data sets were out of luck--until now.

Aspera's niche was, until recently, a corner of the IT market of interest primarily to the media and entertainment industries. The cloud, and the simultaneous but largely unrelated growth in rich media generally, has changed all that as companies across the industrial spectrum now need to move large amounts of data across the Internet. And this problem of transferring unique data sets from disparate, sometimes mobile sources (like the aforementioned video producer on a shoot) is one that neither WAN-optimization appliances nor content-distribution networks are designed to solve. Aspera's technology is designed to improve content delivery in a number of scenarios--not only ad hoc file transfer and synchronization, but also data transfer into the cloud, from both PCs and, now, mobile devices.

Historically, file transfer optimization products like Aspera's have used a client-server software architecture. You run a proprietary server "receptacle" in the enterprise and distribute client applications to users who transfer big data. This works fine for the ad agency scenario filming a new car ad at the Bonneville Salt Flats (the middle of nowhere, Internet-wise) needing to copy the day's best clips back to Hollywood for editing. But cloud services introduce another, and likely much more common, use case, one in which you don't directly control the server hardware and are using the cloud to facilitate working with people outside your company. Yet using the cloud when the object of discussion is a 100-plus-MB video clip, scientific data set, or CAD file can feel like trying to drain a swimming pool with a drinking straw. Here is where Aspera's new virtual appliance, which turns its server software into an Amazon EC2 instance tied to an S3 storage pool, comes in handy.

Aspera isn't alone in addressing the need for more efficient transfer of and access to cloud-resident data. RocketStream, which has since been acquired by Tibco and incorporated into its Managed File Transfer service; FileCatalyst; and RepliWeb offer similar managed data transfer services. If you're in a business that routinely manages large data files, this new generation of services that move data transfer optimization technology into the cloud is worth investigating. Speeding up file transfers by one or two orders of magnitude does wonders to your ability to efficiently collaborate across organizations and geographies.

Get lessons from five companies on the front lines of implementing unified communications. Also in the all-digital supplement of Network Computing: Mike Fratto on how to make the case for UC. Download the supplement now. (Free registration required.)

Comment  | 
Print  | 
More Insights
Comments
Newest First  |  Oldest First  |  Threaded View
ExpeDat
50%
50%
ExpeDat,
User Rank: Apprentice
10/17/2011 | 9:42:34 PM
re: Big Data Means Big File Sharing
Kurt,

Thank you for this article and for shedding light on the UDP-based / high-performance data transport space. As the first company in this space of those listed above, and the only one with multiple US-patents, I think it worthwhile to mention to your readers Data Expedition, Inc.---creators of the Multipurpose Transaction Protocol (MTP/IP): www.dataexpedition.com

MTP/IP has been integrated into a wide variety of 3rd party software applications to enable their users to enjoy the benefits we present with our high-performance transport. Of course, integration is simply one angle, while another common choice among the world's most recognizable companies (Motorola, Disney, Lockheed Martin, Procter & Gamble, etc.) is to use our ExpeDat file-transfer software, built on top of MTP/IP.

With multiple integration packages available, easy to use and install software, convenient and inexpensive pricing, and the most adaptable UDP-based transport technology on the market, I'd encourage all of your readers to have a look at us.
dsf74
50%
50%
dsf74,
User Rank: Apprentice
10/15/2011 | 9:33:13 PM
re: Big Data Means Big File Sharing
Kurt, while cloud is one way [FTP being another] to transfer large files from one point to another, there is another alternative with which you can transfer large files directly from the sender to the receiver's computer without having to first upload them on the cloud or some central place. This alternative is called Binfer [http://www.binfer.com]. Binfer does all that you mention about Aspera[aes, high speed] with significant advantages over Aspera: 1. There is no complex installation of appliance etc. 2. Significantly cheaper. 3. You can setup Binfer under 2 minutes and be on your way to sending those 500 GB video files.
Cartoon
Slideshows
Audio Interviews
Archived Audio Interviews
Jeremy Schulman, founder of Schprockits, a network automation startup operating in stealth mode, joins us to explore whether networking professionals all need to learn programming in order to remain employed.
White Papers
Register for Network Computing Newsletters
Current Issue
2014 State of Unified Communications
2014 State of Unified Communications
If you thought consumerization killed UC, think again: 70% of our 488 respondents have or plan to put systems in place. Of those, 34% will roll UC out to 76% or more of their user base. And there’s some good news for UCaaS providers.
Video
Twitter Feed