RAID 5 on multiple controllers
Hi all, I have been using a software raids for several years mostly due to inexperience with configuring and managing hardware raids. I am a little more comfortable with hardware raids now but still have some questions regarding setup/configuration. I have 6 drives, 2tb each. I have 2 4-port sata controllers. Is it possible for me to set the 6 disks up in a single raid 5 array even though they are on 2 different controllers?
Thanks in advance.
Thanks in advance.
0
Comments
As for doing an array across multiple controllers, it's not always impossible. Some modern cards do have the ability to span an array across multiple controllers (or so I've read). The caveat here is that you have to have cards that support that feature (and there aren't many of them, once again, from what I've read) and I'm pretty sure you have to have the exact same model of card for all your RAID cards.
If you're going to do RAID5, make sure you're using a true hardware RAID card and not a CPU assisted RAID card. If you paid less than 300 dollars for your RAID card, you're probably not using a true hardware RAID controller. If you are not using true hardware RAID, I would suggest strongly that you do not go RAID5 or 6 due to the fact that, as I mentioned before, you're going to take a serious performance hit. Add to that the fact that it is highly unlikely that you will be able to make a RAID array across the cards and what you're probably better off doing would be making a RAID10 array across the 6 disks. In case you're not familiar with RAID10, this would involve making 3 sets of 2-disk RAID1 arrays then using those 3 arrays to make one big RAID0 array. This gives you the redundancy of a mirrored array (ability to lose 1 disk out of each RAID1 array without data loss) as well as a decent performance boost because you are striping your data across the 3 arrays. You'll also be able to do it across cards (in a manner of speaking) because you can make the RAID1 arrays on the card itself, then use software RAID to make your RAID0 across the RAID1 arrays.
The idea that RAID6 somehow has some massive CPU overhead is a complete fallacy. RAID5 is N+Parity, RAID6 is N+P+Q. (Where N is your data disks.) The idea that P is expensive, is just silly. P is XOR - that's it. Q is XOR + Galois. Typically this is implemented functionally as XOR -> Linear Bitshift. So there's not actually enough CPU overhead to matter, unless you're running the disks ragged with sequential IO on something like a TBird 750MHz. (And even that can handle it with ease.) Honestly, if RAID5 or RAID6 actually incurred that much CPU loading, do you think NASes would work at all? You're talking about systems which don't have dedicated parity engines and are using very low power embedded CPUs. RAID just isn't that CPU expensive any more, relatively speaking.
As for cross-card spanning, nobody's doing that currently unless I missed a feature on the latest round of LSI cards. That feature was removed after the LSI Ultra320 family for lack of demand. Now I'm going to have to go look into things, and see if somebody brought it back. <joking>
(JOKE) Thanks a bunch for that, ardichoke! You're making me do research! Jerk! (/JOKE)</joking>
I don't disagree that true hardware is superior. Especially if you're only talking 6 disks. The problem is that you're talking 6 disks. That puts you into the higher end of RAID cards if you want something that isn't going to crap out. You can get true hardware RAID for cheap, but you're not going to get fast or more than 4 ports. 6 disks means 8 ports means LSI 8708 or 926n-8i, Areca ARC-1160 family, you get the idea. Very Not Cheap products.
The other problem here is that with 2TB SATA, your best case IOPS in real world is right around N * 90. So for 6 disks, that's a whopping 540 IOPS. (Compare this to 15k SAS @ 250+ per disk and even consumer SSD at >4K IOPS.) Your primary concern in this array is the disks are sloooooooow. They're also consumer disks, which like it or not, means that failure will occur. And rebuild will be hellish and insanely long. As disk size increases, rebuild time does not go down. And with disks that slow, good luck loading down any modern CPU. (You'll see load, oh yes! It'll just say "iowait.")
So I'm not recommending RAID6 because it's buzzword or because it's faster. I'm recommending it because for a 6 disk software RAID setup, it's going to give the least amount of pain. It's also going to make the inevitable expansion less agonizing. I'm not going to recommend RAID0 overlaying RAID1 because frankly, you aren't buying 2TB drives to lose half your capacity. And you're not going to actually gain anything over RAID6 either. RAID6 at least you've got all spindles involved; RAID1 halves your performance potential. Plus software RAID0 is the most fragile sort. Software RAID3, 4, 5, and 6 leave you parity recovery. RAID0 has no such protection, making data loss potential much much higher.
Yes. I do spend entirely too much time on storage. But then again, it is my job.
(EDIT) Oh my gods, I forgot to mention; hardware RAID cards aren't an option for 2TB drives except for Seagate Constellation 2TB, Hitachi A7K2000 and WDC RE4 drives. Without TLER and adaptive algorithm, they will drop off the controller. Joy!
Further, recommending RAID6 is rather silly as well. You get no performance benefit (in fact, you will take a slight performance hit thanks to the additional parity calculation) and you sacrifice 2 extra TB of storage. If you really want to go 5 or 6, just go 5 and invest in an extra disk to have on standby in case of failure. Of course to be fair, in a home environment your chances of failure are not all that high. I, for one, have never had a disk fail on me and I'm not exactly what you would call a standard user. I have a couple computers at home that run more or less 24/7. That said you should still have backups... but you should have backups no matter what RAID level you are running because RAID IS NOT BACKUPS.
Man, I can go on for about.. I think last test went four weeks on "OMG NO STOP PUTTING THIS IN DESKTOP." And that was on ICH9R Matrix RAID. I can probably add another week just from ICH10R alone, forget the Marvell stuff! (Let's not test it, please!)
Seriously. 6x2TB in desktop = :mad2:. RAID5 in desktop = :mad2:. RAID1 in desktop =
Well, this isn't true. RAID6 is striped parity, not purely N+P+Q. N+P+Q is your capacity calculation. Because RAID6 is striped parity across all disks, you get another spindle of IOPS available over RAID5 + Spare. RAID5 is also going to incur a penalty on any configuration besides 4+1 and 8+1 forever and ever amen because of full stripe writes. You can't do them on odd number disks at any stripe size, period. RAID6 suffers full-stripe write penalties on odd disks but not on even disks. Realistically, a 6 disk RAID6 will see better performance than a 4+P+Spare RAID5 and will always be faster than 5+P RAID5. You're not taking any capacity hit over RAID5 + Spare, and you're avoiding 24+ hours of praying a second glitch doesn't happen. It doesn't even have to be a full failure - just enough to trigger a rebuild.
And as I often preach, presuming no failure is nothing more than a guarantee you will. Failure will happen. Arrays that big at home are not backed up often enough or effectively enough. You're better served acknowledging the fact that a disk failure of some sort will occur, and planning for it with a priority on fault tolerance. All it takes is once.
Regardless, we've gotten way off topic here anyway. @BossHogg - It is highly unlikely you will be able to span an array across both RAID cards (unless you use pure software RAID).
I love it when the storage geeks go "Hammer and Tong"