Some general raid questions
Hi gurus!
As a Unix engineer working on big iron systems, I've always wished to be able to do some form of raid on my desktop, but shied away from tacky software raid. Now finally I have a system which allows me to play arround with motherboard based raid, call it hardware assisted raid or fake-raid or whatever you like. Since I started playing arround with it, I have discovered a few questions to which the answers still elude me.
#1 of these questions is Where does nForce raid and/or SI3114 raid arrays keep the partitioning information.
#2 is related to the above: Can I move a raid array, let's say a raid-0 stripe, from one motherboard to another, or even from one controller to another on the same motherboard. In particular, I have a motherboard with an SI3114 as well as nForce raid, so I may want to move an array from the one controller to the other.
#3 is there a way to add a mirror to a disk after the array was created? This is probably not possible using the default BIOS tools as these appear to destroy the simple disk partitioning information when you "commit" the array config changes made.
#4 related to #1 and #2: Is a JBOD disk usable on a non-RAID controller without losing the data and partitioning information? More accurately, if I have a disk configured on a RAID controller as a JBOD disk, can I take this disk to a non-raid S-ATA controller and still see the partition table and file systems? Conversely, if I have a controller set to non-raid mode and have a disk on it, and later want to add a mirror to the disk, which requires me to switch on the raid-controller mode to add the mirror, what happens to existing partition information on the disk?
#5 Are there any tools for backing up and/or re-creating partition table and raid array configuration information for use on controllers like the nForce4 raid and SI3114 raid controllers?
#6 I read a review somewhere for an intel motherboard which allows a disk to be in more than one array. Does anybody know whether that is possible on nForce raid or SI3114 raid controllers? I'll search for the article again if anybody is interested, but the tac was that this is un-usual.
#7 Where is the array config stored, I.E which spindle is the 1st, 2nd, 3rd etc in a specific stripe, what raid-level is each stripe, what parts of each disk is used by each stripe, etc. This answer is probably related to #1 above :-)
Thanx for your time!
_Hartz
As a Unix engineer working on big iron systems, I've always wished to be able to do some form of raid on my desktop, but shied away from tacky software raid. Now finally I have a system which allows me to play arround with motherboard based raid, call it hardware assisted raid or fake-raid or whatever you like. Since I started playing arround with it, I have discovered a few questions to which the answers still elude me.
#1 of these questions is Where does nForce raid and/or SI3114 raid arrays keep the partitioning information.
#2 is related to the above: Can I move a raid array, let's say a raid-0 stripe, from one motherboard to another, or even from one controller to another on the same motherboard. In particular, I have a motherboard with an SI3114 as well as nForce raid, so I may want to move an array from the one controller to the other.
#3 is there a way to add a mirror to a disk after the array was created? This is probably not possible using the default BIOS tools as these appear to destroy the simple disk partitioning information when you "commit" the array config changes made.
#4 related to #1 and #2: Is a JBOD disk usable on a non-RAID controller without losing the data and partitioning information? More accurately, if I have a disk configured on a RAID controller as a JBOD disk, can I take this disk to a non-raid S-ATA controller and still see the partition table and file systems? Conversely, if I have a controller set to non-raid mode and have a disk on it, and later want to add a mirror to the disk, which requires me to switch on the raid-controller mode to add the mirror, what happens to existing partition information on the disk?
#5 Are there any tools for backing up and/or re-creating partition table and raid array configuration information for use on controllers like the nForce4 raid and SI3114 raid controllers?
#6 I read a review somewhere for an intel motherboard which allows a disk to be in more than one array. Does anybody know whether that is possible on nForce raid or SI3114 raid controllers? I'll search for the article again if anybody is interested, but the tac was that this is un-usual.
#7 Where is the array config stored, I.E which spindle is the 1st, 2nd, 3rd etc in a specific stripe, what raid-level is each stripe, what parts of each disk is used by each stripe, etc. This answer is probably related to #1 above :-)
Thanx for your time!
_Hartz
0
Comments
#3 depends on the controller
#4 yep should work if your talking about a single drive array being moved to a non raid. Again you can't make a blanket statement that fits all controllers.....
#5 you can back up the array itself using many of the ghosting tools. You talking unix here? or Windows I should of asked. You need to recreate the array itslf in the bios before restoring.
#7 This one makes no sense to me at all. Which sorta relates to #1 also. The "what raid level is each stripe" was sorta baffling for example.
The controller creates a lets say "raid-0" array. With a 64k stripe. Meaning it alternates back and forth writing to each drive in the array every 64k of info. The controller does not partition the drive... OS does and its stored in stripes also. Everything is. Your raid-0 array is basicaly a virtual disk to the OS. It writes to it thinking its one drive. Partitioning or data, it sees it all the same and its spread across all the drives in the array in stripes. Thats why you are screwed to lose a single drive from a raid-0 array. I do this for a living and the motto is you never put anything on raid-0 you can't afford to lose.
Now if in number 7 above you wanted to know which drive in the array is written to first and second etc... Usually its the order you select the drives into the array in when its built. But again differant controllers can handle this differantly. Meaning if you picked them out in a stupid sequence they might try and correct it for you.
Tex
Thank you for your answer. Let me add a bit of background info, though this may make the question much less "general" ;-)
On a normal hard drive, the partitioning information is stored in the disks first sector (I believe - someone please correct me if I'm wrong). But this sector (wherever it is) is corrupted when the disks are placed into an array, at least when the array is a non-JBOD (I have only tested this for raid-0 and raid-5). This, at least, seems to be true for the nForce4 raid controller and the SI3114 controller from my limited experience.
Linux sees the individual drives, but does not see the partitioning information on any of the drives (at least not without the latest kernel and softraid support, which to some extend defeats the purpose). This I found out the hard way, trying with various diagnostics and repair live-CDs to gain access to the partitions, even on disks in a raid-1 config!
If I knew exactly where on each drive the important configuration data is kept (array details, partitioning details) I'd be able to use dd to extract this and put it in a file on some other media, eg a memory stick, before I start to fiddle with the config. I'd also (probably) be able to reconstruct a normal "first-sector-of-the-disk" partition table, say on a disk from a mirror set, to be able to mount the disk under non-raid controller control.
I have managed to once do this, but only because I were intimately familiar with the layout of the partitioning of the raid-1 array. I created a new partition table onto the disk, guessing the start sector, and spanning the partition to the end of the disk. I was then able to fsck and mount the ntfs file system to recover my files. But in general I may not be so lucky, and obviously with arrays which distribute the data blocks over several disks this becomes much harder to do.
Does the "virtual disk" somehow re-direct partitioning data to somewhere other than the first sector? And is details about the array (stripe), eg type and offset, get stored on the disks?
So far your answers are not giving me a lot of hope however. Is there maybe a "Standards" document or RFC detailing this type of raid configuration?
P.S. To answer your question: Primary client OS is Windows, though I dabble in Linux. This will change once Linux support for soft array become standard, right now it is just a little bit too distribution-specific for me :-)
I'm 49 and was selling SCO xenix/Unix boxs and writing customn software for them back in the early/mid 80's before spending most my time as a software developer on Unix/Aix systems in the 90's so yes I do see where you were going with this.
I have never seen a standards document such as you are looking for and I think it will be both controller and OS dependent on how/where the info you seek is stored.
I do most my work these days with high end scsi raid controllers like those from LSI/Dell/Intel. Yes they store array config info on the drives. I can move an array from one controller to another intact. It keeps a copy on the drives and in the bios/firmware and knows if there is a mis-match and trys to get you to update one or the other so they match for example.
The thing you don't like about linux software raid... "that its to distribution specific" is what you will bang your head on with the hardware or psuedo hardware raid in that every maker does it in a proprietary way. There are no real standards held between differant controller and raid implementations.
On your moving a jbod disk for example... The partitioning table is crapped out when you init the array. But you should be able to move the array off allright but again its controller specific. On my high end scsi controllers I can move a disk ONTO the controller filled with data. Create a single drive raid-0 array and not INIT the array and the data all stays just fine. Its a two step process. On many psuedo raid controllers for ide/sata you create the array in one step and the data is gone. Its all very controller specific and we are talking in broad generalities as if these answers apply across the board is the problem.
Tex
I'm more used to intelligent disk subsystems doing most of the work for me (Eg HDS 9960s, HDS USP, etc). On those systems I rarely if ever need to worry about the actual functioning of the raid underlying mechanics, so I've been spoilt. Unix just sees the LUNs presented as disks. I also have some ... superficial ... experience of external HW raid arrays used int the workgroup / midsized Windows server space, but it is from about 5 years ago. That used to be a very nice happy medium, allowing you to set up hot spares and even do some stripe re-layout online (eg change nr of columns). The controllers were dual-ported, the trays supported dual-path connections, and you had battery-backup of the in-progress transaction data. Nicely fast too.
Now I don't expect to be able to move from raid 5 to raid-1 or increase a raid stripe width on a softraid controller, but I'd think adding a mirror to an existing disk is not too far fetched!
So in any case, my questions are all related to my dabbling in PC hardware, which is a hobby. I am hoping to create a setup with two disks striped for performance, onto which to install the OS and programs, including also my MySQL database.
Backups will be to a pair of disks mirrored, includung the location for user-data and some archives. I don't have tons of data to backup, so for now that will suffice for things which don't get dumped to CD or DVD media.
I must say I get the feeling that "softraid", if I can call it that, is just so immature that nobody has yet through about setting up standards. Kinda like where the fibre channel protocol was 10 years ago.
But I'm still hoping that someone here would have some technical data about how / where the arrays store partitioning and stripe config details.
Thanx for the info so far, this forum is great!