RSS
Email
Print
ShareunRAID But Not Unprotected
Posted by Howard Marks on January 7, 2010
One of the things I like best about this job is discovering hand-crafted software that elegantly solves the problem its designer had in mind, while putting a new twist on familiar technology. Looking for a reliable and yet expandable place to store the media for his home theatre PCs, Lime Technology's Tom Mortensen came up with unRAID, an innovation that lets users build parity-protected data stores from drives of different sizes and add capacity on the fly.
Like a RAID-4 array, an unRAID is made up of a group of data drives and a dedicated parity drive. Rather than stripe data across the data drives as a RAID-4 array would, unRAID writes an independent RFS file system to each data drive. When you save a file to the system, it's stored in one data disk's file system. unRAID updates the corresponding blocks on the parity disk to reflect the new data. As long as the parity disk is as large as or larger than all the other drives, unRAID can use all the space on all its data drives for user files.
To minimize the number of I/Os for each write, unRAID calculates the new parity from the old parity, the new data and the old data, so the parity calculation takes four I/Os for any set of data drives. Calculating parity from all the data drives could take several times that many on a large array. It also means an unRAID system only needs to spin up one data drive to play a movie and just two, the data drive that's being written to and the parity drive, to store data.
UnRAID's user shares create a CIFS share that exists on multiple data drives. Each file is stored on a single drive based on one of several available load balancing or spanning policies. A side effect of not striping data, although one much loved by unRAID's users, is that in the event of multiple drive failures the files stored on the other data drives are still available. This means that you can still watch seasons 1-24 of Dr. Who when the drive with the last few seasons crashes.
Not striping data does limit unRAID's performance, but you should remember that unRAID isn't supposed to be a general purpose replacement for standard RAID based NASes. It just has to be fast enough to deliver an HD video stream to a HTPC (Home Theater PC) and save data as fast as a Blu-Ray drive can rip it. Read performance is limited to the speed of a single drive and writes are normally somewhat slower. Users can dedicate a cache drive to hold new data until a scheduled process copies it to the unRAID. With a cache disk write performance is about that of the cache drive.
Comment by UnRAIDer on January 11, 2010 3:09 PM
Some article comments can be found here:
http://lime-technology.com/forum/index.php?topic=5052.0
Reply to this comment
Comment by Cinnamon on March 20, 2010 3:36 AM
Okay... I've been reading about this wherever I can, now, and I just can't see it - I'd like to know just how ONE dedicated parity drive can possibly be used to reconstruct the full contents of another drive in an array that could span 20 drives WITHOUT striping. That just doesn't make any sense to me - if that were actually possible, then why bother with the data drives at all? The parity drive could handle all the data - just rebuild the data as you need it on the fly. Sounds like so much snake oil to me.
Reply to this comment
Comment by anonymous on March 23, 2010 9:39 PM
If any one drive fails (except the parity drive!) then by using XOR logic, you can reconstruct the missing bit in the below example.
If the parity drive fails, then you can rebuild the parity drive. If more than one drive fails however, you are SOL.
For example, pretend these are the bits off of 5 drives for the same bit location
1 0 0 1 1
Parity is the XOR of all the bits together, so in this case (1 ^ 1) ^ 1 = (0) ^ 1 = 1
If the first drive goes missing...
? 0 0 1 1 with a parity bit of 1
Then you can reconstruct the first bit (drive) by using the parity, which simply keeps track if the number of 1's is even or odd.
Hope this helps!
Reply to this comment
Comment by annonymous on April 27, 2010 8:36 PM
Yeah, that really should help him... It's funny how the moron pretends to "know" how it would worked with striped drives, yet he doesn't know what basic parity is... Idiots are everywhere!
Reply to this comment
Comment by James Murphy on April 28, 2010 4:33 PM
No need to be offensive - I understand parity, that's easy however understanding the concept and understanding its application are not quite the same thing - I for one just hadn't quite grasped how it was applied in this case. Of course *once you've worked it out* its obvious.
Reply to this comment
Comment by Mellow on April 30, 2010 11:24 AM
The example works fine with striping. If you are not striping and lost the drive the whole 1 0 0 1 1 sequence is lost, not only the first bit.
Reply to this comment
Comment by Cinnamon on May 3, 2010 7:15 PM
Okay, see, this is the problem. The first respondent obviously assumed that the rest of the array was using striping - as near as I can tell. And I have no problem understanding how a RAID5 works - or even RAID4 - which does use a dedicated parity drive, but also uses striping.
What I do NOT understand, and what no one has, in any of the reviews or analyses of the UnRAID system, explained to my satisfaction, is how the contents of an entire lost drive can be recovered WITHOUT using striping.
The UnRAID system claims that worst case, if you lose one drive, the remaining drives can all be read individually. This suggests a non concatenated JBOD sort of setup. That's all fine and well, but what I do not get is just how the contents of any of those drives can be recovered from JUST the parity record.
In RAID 4 or 5, if a given drive fails, you have the parity record, plus the stripes from the remaining drives to reconstruct the lost data. With UnRAID the claim is that the stripe information isn't required - so HOW do they do that? HOW do they recover the entire contents of a drive with nothing but that parity record? And if they CAN do that, then why bother with data drives at all?
Maybe I'm just dense or something, but I just don't grok it.
Reply to this comment
Comment by Alan on May 15, 2010 10:06 AM
The parity drive works just like it would on a RAID.
When you stripe data on a RAID, you XOR the nth byte of data on all the data disks and write that to the nth byte of the parity drive.
With unRAID, it XORs the nth byte of data on all the data disks and writes that to the nth byte of the parity drive. The only difference is that files aren't striped across the data drives on unRAID.
.
Reply to this comment
Comment by arliejacobs on May 27, 2010 12:46 AM
Just wonder what happens in the event that a USB stick blows? Setting up with a new one, with a new UnRAID OS, will it recognize the drives properly, or would you have to rebuild drive information, because the stick has a different serial number. Just wondering if the USB stick serial number gets inter-twind somehow and if swapping sticks has any effects?
property for sale in london
Reply to this comment
Comment by anonymous on June 25, 2010 1:26 AM
There's no issue if your USB dies, just create another one and away you go. (The Unraid guys are quite happy to transfer your license to a new USB if the old one dies)
Reply to this comment