RSS
Email
Print
Share3Leaf Dynamic Data Center: True Hardware Virtualization
Posted by Mike Fratto, Editor on November 4, 2009
3Leaf's Dynamic Data Center (DDC) is a rack of servers, storage and networking that does something unique—it allows administrators to treat the entire rack of systems as one computer running a single OS over multiple AMD Opteron servers. The location of underlying hardware no longer matters. Now that is virtualization. 3Leaf is positioning the DDC to sell to OEM vendors and limited fixed configurations to end-users. Systems will be available in December, 2009.
At a high level, 3Leaf's DDC product line virtualizes and aggregates computing hardware across multiple platforms. A fully racked DDC rack operates as a single computer with 192 cores and 1TB of RAM. To provision a server, you select the number of cores, the amount of RAM and the required I/O. For example, an application that requires more memory than is available in a single server can access RAM on other hardware.
Consider that a DDC-Server configured with 1TB of shared memory, 192 cores of AMD processors at 2.8 GHz and 8TB of storage, all connected via an InfiniBand switch and complete with cables, Linux OS and DDC-Pool software lists for $250,000. A DDC-Server with 256GB of shared memory, 96 cores of AMD Istanbul processors at 2.4 GHz, 4TB of storage with an InfiniBand switch, cables, Linux OS and DDC-Pool software is listed at $99,000.
Three technologies come together to make this happen. Modern processors support nested page tables, which are a hardware memory management technology in the CPU that allows multiple virtual hosts to have the same view of RAM. 3Leaf leverages nested page tables and their custom ASIC to map memory across multiple computers. The servers are interconnected with a high throughput, low latency (approximately 100 nano seconds) Infiniband switch. Finally, native OS support for ACPI dynamically manages hardware resources.
Representatives from 3Leaf say that a similarly sized system from the likes of HP or IBM might run into the millions, but 3Leaf pricing starts at $99K because they use off the shelf parts for the hardware compared to specialized HPC equipment.
Comment by Sotuan on November 4, 2009 2:43 PM
I have an idea. Why don't you take a DDC-Pool and run ESX on it. Then you can take a whole bunch of machines, pool them together, and then split them back up again. Now *that* would be true virtualization. ;-)
Reply to this comment
Comment by Mike Fratto on November 4, 2009 3:40 PM
I think my head just exploded. LOL
Reply to this comment
Comment by NUMA Lives.....again! on November 4, 2009 5:48 PM
The architecture looks pretty much like a standard CC-NUMA architecture. This is not really new......SGI Origin Servers did it on MIPS.....and IBM still sells the 3950-M2. It also uses IB interconnect to stack up to 4 chassis together. They only go up to 128 cores vs. 192....but the bigger question is not the technology, but what is the business value behind it. IBM purchased Sequent Computer 10 years ago.....and dont ship many of their NUMA boxes. Windows has extensions for NUMA, and so does ESX.....so the OS support is there for most of what people need, yet none of the big guys decided to pursue this....outside of the IBM niche. CC-NUMA is great for some applications. Best of luck guys pushing NUMA around again.
Reply to this comment
Comment by Mike Fratto on November 4, 2009 6:02 PM
The things I don't know could fill a library. Hey, thanks for the heads up on NUMA. Here is a quick intro to NUMA http://www.redbooks.ibm.com/abstracts/tips0476.html and a FAQ from the Numa project on sourceforge http://lse.sourceforge.net/numa/faq/
Thanks!
Reply to this comment
Comment by gedw99 on November 4, 2009 8:16 PM
Great idea, BUT Totally dead in the water.
KVM can easily do the same thing in software itself at the hypervisor level.
And it can use all the grunt of the CPU's, rather than some ASIC.
So it will be free cost.
The interconnect to the other server nodes will requires serious bandwith too.
Reply to this comment
Comment by Mike Fratto on November 4, 2009 8:22 PM
Gedw99, so since I don't know much, let me ask this. Can an all-software technology keep up with need to keep latency really, really low? That is kind of the point of doing the memory mapping in the ASIC--low latency. The firmware is just to bootstrap the node. (from what I understand).
So while you may be able to do this in software with KVM, would you?
Reply to this comment
Comment by SRM on November 4, 2009 9:15 PM
I think the key is to be able to do this with COTS Commercial Off -the-shelf servers at a fraction of the cost
Reply to this comment
Comment by natsukun on November 5, 2009 1:46 PM
I'm interested that how they can handle storage - separate disk shelves with traditional raid system? own clustered array/file system? customized zfs? ... anyway, interesting :-)
Reply to this comment
Comment by Serve Technology on July 3, 2010 3:29 AM
It only requires the member to log into his MySpace account, and then open the page where he desires to upload the background. The MySpace backgrounds will come with simple codes, which anyone can follow. They just need to copy paste the code onto the page where they want the background to be seen. Backgrounds can be used with either images or texts.
================
Serve Technology
Reply to this comment