Yahoo Opens Bandwidth Bottlenecks

ORLANDO, Fla. -- Storage Networking World -- If its the Jessica Simpson video from “The Dukes of Hazzard” movie that your customers are after, you want to make sure your storage, databases, and media servers don’t buckle under the load of 2.4 million hits a day, according to Ken Black, global storage architect for Yahoo Inc.

“When you’re serving video to 20 million customers per week, it can put a strain on infrastructure,” Black said during his keynote Wednesday, a rare moment of understatement at the otherwise marketing-rich SNW.

Yahoo, which Black claims is the world’s largest portal, has re-invented itself beyond its search roots. The company provides audio and video streaming for corporate and consumer users, offering .wmv, QuickTime, and mp3 formats among others; movie trailers; other advertising; and soccer tournaments. Yahoo can also conjure a flash crowd with two simple words: Victoria’s Secret.

But moving beyond its consumer base is a big part of expanding the company, which recorded $3.5 billion in revenue in 2004. Black said Yahoo’s corporate users now include Ford, General Motors, 20th Century Fox, MGM, BMG, Universal, Sony, Philips, and McDonalds.

Still, that growth has indeed strained the portal’s infrastructure. Its media servers were working with direct attached storage (DAS) that would feed disk-based content to Yahoo’s streaming servers. CPU issues and buffering problems were too common, and performance slowed to a glacial 30 Mbytes per second or less.“There was no shared capacity on the DAS, so it was difficult to scale, and it made for high cost per Mbyte delivered,” Black said. In addition, Black needed storage arrays of all sizes to accommodate various locations and requirements of companies that Yahoo had acquired. “We needed any-to-any connectivity. I didn’t want to be limited by any single brand and have it talk only to Windows or only to Unix clients."

As a result, Yahoo upgraded to NAS gateways made by ONStor, a change that immediately eliminated the disk and server bottlenecks. “Our content is read-only content for streaming users, so if one of the NAS heads starts getting busy, I make a copy of the data volume and share it to another NAS head on the fly,” Black explained. “It’s awesome on the scaleability.”

Using NAS has boosted performance to 200 Mbytes per second per device, and he got native CIFS and NFS built into the box, which obviates his Windows versus Unix issues. Black also likes the fact that the ONStor boxes offer n-clustering, another scaling boon. “We can also go to a single pane and manage it all."

As bandwidth, quality, and speed requirements increase each month, Black said the content company is well positioned going forward. “It allows us to plan for the next Jessica Simpson or Britney Spears video."

— Terry Sweeney, Editor in Chief, Byte and SwitchOrganizations mentioned in this article: