04:44 PM
Repost This

SDN: Is Big Data a Killer App?

A major promise of software-defined networking is that it will unleash new applications to take advantage of network programmability. Big data may be one category poised to leverage SDN.

Software-defined networking is a hot topic, and for good reason. Just as the hypervisor transformed the data center, SDN has the potential to transform the data center network, and in the process enable a new wave of powerful applications. If done right, SDN will speed the provisioning of network services and, perhaps more importantly, create a new software layer that can be used by applications to extract more value out of the network and the computing systems that they connect.

I've been involved with IT Infrastructure for pretty much all of my career. As much as I love it, I have to remind myself that infrastructure is important only to the extent that it supports the delivery of valuable applications. Applications produce results that have meaning and value. Infrastructure--that is, computing, storage and networking--are just the means to that end.

The term "application" shows up a lot in SDN conversations, and that's good, because it more closely links SDNs to where the real value is created. But I also wonder if the term as used today is potentially misleading. For example, consider that in the canonical architecture diagram that's emerged around OpenFlow-based SDN, firewalls and load balancers are "applications."

I'm not sure I buy that. I've always known those things to be "infrastructure," like routers and switches. Excel and Oracle Financials and Hadoop are applications. With SDN, the network infrastructure is now programmable (that is, software-driven), so I suppose firewalls and load balancers can be seen as applications in the sense that they are software programs that instantiate functions in the network via the SDN controller. That said, I think firewalls and load balancers are better labeled as "software-defined infrastructure."

OpenFlow Architecture

What I'm wondering is if SDN will cause an IT revolution in application development. Think of the PC: While PC makers had to get the hardware right, it was the emergence of killer applications such as spreadsheets and desktop publishing that really drove adoption.

A true killer app for SDN would be one with significant economic benefits to lots of companies, and one that requires SDN in order to be viable on a broad scale. I think big data may be one such application area. You're probably thinking, "Oh, great--let's take over-hyped SDN and multiply it by over-hyped big data and see what we get!" That's fair, and all we know that the answer will lie somewhere between huge and zero. But a few data points suggest there may be something here.

First, big data has unlocked big value for some companies. For example, it's been reported that Google generates 95% of its revenue from the ability to target ad placements based on search terms. That's an extreme and as-yet unduplicated example, but big data apps will be used pervasively if they can drive even a few percentage points of revenue (or save equivalent costs) for many organizations.

Second, a theoretical analysis presented this summer by researchers from IBM and Rice University suggested that network-aware applications could reduce execution times for key Big Data operations by 70%.

In a more recent experiment, which was conducted by Infoblox, an SDN-aware version of Hadoop achieved a 40% reduction on a key Hadoop benchmark when executed over an SDN network. The Hadoop application set up prioritized queues (flows) in the underlying OpenFlow switches at initialization, giving the highest priority to traffic from particular HTTP ports--specifically, those ports corresponding to the most latency-sensitive Hadoop traffic. As a result, critical Hadoop traffic was forwarded ahead of other, less critical traffic, and so the job completed faster, even when executing on a congested network.

The 40% performance improvement is impressive--even more so when you consider there was no feedback or runtime optimization. Applications that speak SDN can dynamically monitor network state along with app execution and make network optimizations on-the-fly.

The implications of this simple experiment could be significant for SDNs and big data. Applications like Hadoop rely on distributed computing in order to achieve acceptable performance. Distributed computing requires a network. If SDNs can enable applications to make better use of distributed processing resources, then big data applications can deliver acceptable performance using much less computing hardware, making them affordable for more organizations. That could make big data a killer app for SDNs.

This is just one example. Delivery of rich media like video and audio also has potential. Just as the PC (and now mobile devices) unleashed waves of programmer creativity, it will be exciting to see what programmers--perhaps flowgrammers?--will do when their applications can define the network.

Richard Kagan has spent the last 30 years working in telecommunications, networking and security.

Comment  | 
Print  | 
More Insights
Newest First  |  Oldest First  |  Threaded View
User Rank: Apprentice
12/19/2012 | 12:15:00 PM
re: SDN: Is Big Data a Killer App?
Good info Rick. Infoblox is misspelled as InfloBlox.
User Rank: Apprentice
12/18/2012 | 8:20:56 PM
re: SDN: Is Big Data a Killer App?
@lgarey: Yes, sort-of. The dynamics with video are somewhat different that those of the Hadoop example, which demonstrates the value of SDN to distributed computing. I'm not sure about what the 40% number would mean in the video context. That aside, service-aware routing using SDN technology is being eyed by service providers as a way to keep up affordably with the exploding demands that video is placing on their infrastructures. I'd also expect to see SDN making a big impact on videoconferencing, where the ability to dynamically provision links with the appropriate bandwidth, latency and jitter would is a big plus.
User Rank: Apprentice
12/18/2012 | 7:03:16 PM
re: SDN: Is Big Data a Killer App?
"Flowgrammers" ... love it. But more to the point, that 40% stat re Hadoop is pretty impressive. Could that same optimization be duplicated for other applications, like video for example?

Lorna Garey, InformationWeek
More Blogs from Commentary
Infrastructure Challenge: Build Your Community
Network Computing provides the platform; help us make it your community.
Edge Devices Are The Brains Of The Network
In any type of network, the edge is where all the action takes place. Think of the edge as the brains of the network, while the core is just the dumb muscle.
Fight Software Piracy With SaaS
SaaS makes application deployment easy and effective. It could eliminate software piracy once and for all.
SDN: Waiting For The Trickle-Down Effect
Like server virtualization and 10 Gigabit Ethernet, SDN will eventually become a technology that small and midsized enterprises can use. But it's going to require some new packaging.
IT Certification Exam Success In 4 Steps
There are no shortcuts to obtaining passing scores, but focusing on key fundamentals of proper study and preparation will help you master the art of certification.
Hot Topics
IT Certification Exam Success In 4 Steps
Amy Arnold, CCNP/DP/Voice,  4/22/2014
Edge Devices Are The Brains Of The Network
Orhan Ergun, Network Architect,  4/23/2014
Heartbleed Flaw Exploited In VPN Attack
Mathew J. Schwartz 4/21/2014
White Papers
Register for Network Computing Newsletters
Current Issue
2014 Private Cloud Survey
2014 Private Cloud Survey
Respondents are on a roll: 53% brought their private clouds from concept to production in less than one year, and 60% ­extend their clouds across multiple datacenters. But expertise is scarce, with 51% saying acquiring skilled employees is a roadblock.
Twitter Feed