Networking

02:55 PM
Connect Directly
RSS
E-Mail
50%
50%

Custom Optical Network Will Tackle Big Data

Rice University researchers are building a specialized network that features a new type of optical switch to carry mountains of data.

Researchers at Rice University in Houston have come up with an approach to networking that may hold implications for enterprise uses of big data.

Backed by a grant from the National Science Foundation, computer networking researchers at the school are creating a customized, energy-efficient optical network they say can feed rivers of data into Rice’s supercomputers.

"Advances in computing and sensing technologies have led to a similar problem across many disciplines in science and engineering today," T.S. Eugene Ng, associate professor of computer science and principal investigator on the project, said in a statement announcing the breakthrough. "Experiments produce mountains of data, and there is often no efficient way to process that data to make discoveries and solve problems.”

It’s a problem many enterprises face as they try to jury-rig their networks to cope with the massive data flows they’re facing--data flows that are growing every day. Ng said via email that he envisions the new network as a potential boon for researchers and enterprises that use technologies like Hadoop to process big data but struggle to keep up with mushrooming data flows.

The new network, dubbed “big data and optical lightpaths-driven networked systems research infrastructure,” or BOLD for short, will combine electronic and optical switches. It also will feature a new type of optical switch that has none of the moving parts associated with traditional optical switches. Qianfan Xu, an assistant professor of electrical and computer engineering who specializes in creating ultra-compact optical devices on chips, plans to build the new silicon-photonic switches in his laboratory.

[Read about the critical role of text analytics in big data's transformation of business in "Text Analytics Key To Unlocking Big Data Value."]

While Xu works on the new switch, Ng’s team is working on related tools that will allow networks to take advantage of the combination of electronic, optical and silicon-photonic switches.

"To make use of these three types of technology, we need an intelligent layer that can analyze data flow and demand, all the way up to the application layer, and dynamically allocate network resources in the most efficient way," Ng said in the announcement.

Ng is working with two co-principal investigators on optimizing network design and performance. Meanwhile, another co-principal investigator, Bill Symes, who figures to benefit greatly from the new technology, is helping with algorithm design and testing BOLD’s impact on tackling big data problems.

Symes, a professor of earth science, directs an industry-funded consortium that solves complex seismic data processing challenges. One such operation, used in 3-D seismic analysis, requires two time-dependent simulations--one running forward in time, the other backward. The resulting computation can generate hundreds of terabytes of intermediate data that has to be loaded, cached, recalled, modified and saved repeatedly.

Ng said Symes’ work is a good example of the kind of data-intensive computations that BOLD can streamline. He envisions BOLD improving the performance of computationally intensive research at Rice for many years.

Meanwhile, there are numerous applications for large corporations. For instance, Ng said via email that large Internet service providers rely on Hadoop for big data management tasks involving log processing, business intelligence apps and the deployment of new products and platforms. Those tasks, he said, will be made quicker and simpler by the BOLD network.

Get deep insight into how big data is managed, including governance and analytics, in "Navigating the Big Spectrum of Big Data's Solutions." at Interop New York Sept. 30-Oct. 4]

Comment  | 
Print  | 
More Insights
Slideshows
Cartoon
Audio Interviews
Archived Audio Interviews
Jeremy Schulman, founder of Schprockits, a network automation startup operating in stealth mode, joins us to explore whether networking professionals all need to learn programming in order to remain employed.
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Video
Twitter Feed