I had almost 6 TB of unprotected data on ageing 1Tb drives. I decided I wanted to protect my data and converted an old machine into a server. Why a raid card? Motherboard based raid is not true raid, it is bios based and the error correction is poor. Dedicated pro cards have all the memory, correction and cpu rolled into one nice package. The card: Dell perc 6i The Dell perc 6i is a card made by LSI for dell, it is a volume solution used in many servers. so many servers in fact, that it is stupidly cheap on the used market. We will exploit that savings fully. This card is by far the most affordable raid 6 card. The same card from LSI (maker of 6i for dell) is appx $650. Add in that you can use LSI's excellent software and it's a deal. The icing on the cake is that the card works on "consumer" OS like windows 7. It is important to get the battery backup with this card (BBU) More on it later. Remember to order two SAS to SATA cables off ebay, and make sure you get the 32 pin and not the mini SAS cable. The Raid level: Why raid 6? Why not raid 5? Raid 5 has one drive failure allowed. The problem is on rebuild, the rebuild can fail leading to 100% data loss. Raid 6 has two drives for parity, you can lose two drives on raid 6 and still rebuild. What are the Cons? Cost. You need two drives for parity, for two 2TB drives this is appx $160 as of today. Size yield is poor until you add past the 4th drive Using 2TB drives, 4 drives yields only 4TB (really less, as 4tb is bytes -> kb) However, after 4 drives the size of the array grows the full capacity of each added drive. the "hump" is the two drive sacrifice to safety. Raid 6 can add more drives at a later date. It takes forever to do (48-72 hours) but it can be done. When 2tb drives are $50, add in three more to hit 10TB of rock solid storage. My build: Hardware: Any system with a free 8x or 16x pcie slot. I used a GA-965P-DS3 with e4300 and 2tb DDR2-667 For your psu needs, calculate your power needs. Primary drive (os) is 2x250gb sata wd I had around, in raid 0 (mirror) using the onboard raid. Os: Windows 7 64bit Note about motherboards: Some boards will drop the second 16x slot to 8x or 2x if there is a pci-e card in an adjacent slot. Example: The second 16x slot has three 1x/2x slots around it. Anything in those slots will drop the 16x slot down to 8x or worse, so double check. To overcome this, I used the "graphics slot" which is just a 16x pci-e slot. Because of this, I used a basic low power pci video card. (Ati rage appx 3-5watts) The Perc 6i: The Perc 6i can come on a metal board. This is the cheapest one to get. It is a normal pci-e card on a metal mount. Remove it from the plate. This setup has has no bracket. Using zip ties or scythe ninja wire, you can hold it steady. Inside the case easily. Some people sell home made brackets for $8. Conflict with intel chipsets: You may need to cover pins 5/6 on the heat sink side on the 6i. It's easy to find out if you need to do this, the system will hang on boot. Some properly cut electrical tape will do the trick. The install process: Put in the raid drives, and connect them to the perc 6i. Mount and connect the BBU unit. The BBu unit protects against data loss in power outage. If you have a power outage and are copying something to the drive, you will lose the data in memory being written to the array. This is the sector x drives. So 64kb x 8 drives appx. Not good. The BBU allows you to power on, and it will finish writing to the drives what is in memory. The card will give a warning halt about there being a different array, and will go to bios screen or prompt you to do so. Follow this tutorial with this configuration. 1. Select raid 6 2. Tab to the drives and add them using space bar. a.Adding each drive will show the final capacity of the volume on the right. b.You will notice adding the first two drives adds no size as they are the parity. 3. Go to advanced and select 64kb for default. Setting it larger has no benefit unless you are doing video streaming commercially (512kb) 4. Tab to advance and edit the options as so: Select read ahead only if you have a BBU. I selected adaptive read ahead, it enables it in sustained reads. Write policy you set depending on if you have a BBU. I used write through as write back can lose that data if there is no BBU or a BBU failure in a blackout. 5. Select initialize. 6. Select OK Initializing can take up to 12 hours or more. 6 x 2TB drives took appx 8 hours. When the virtual disc (the array) is complete, you can exit and reboot. Before you exit, make sure the bios enabled tab is checked.Reboot. On EFI motherboards, you can install the OS to the new volume if you want to. On older hardware with the 2tb limit, the array will have to be a second non boot drive. Enter your bios and make sure the motherboard has not changed your boot options. Exit the bios, watch the card load and initialize. When you boot to windows, the card will be installed. The default, up to date dell perc 6i drivers in windows 7 are fine. Go to disk management, and initialize the disc using GPT to show the array as one drive. It's pretty fun to see one drive with 10TB capacity. The initialization on the raid card makes the long format unneccesary, so I picked quick format. Download the LSI software and install it. It will install java as part of the package. This great software can be accessed remotely, removing the need to have the server in the same room. http://www.lsi.com/storage_home/pro.../megaraid_sas/megaraid_sas_8888elp/index.html All done (for the most part). Encryption: If you do encrypt, use truecrypt. Encrypting a huge volume can add another 12-14 hours to the 8-12 hour initialize time! Note on the 6i 32 device capability. Using an SAS expander box you can have 32 drives in your array. Example: Four 2tb drives in JBOD on each of the 8 sata lines. That would be 8TB per line, or 48TB maximum capoacity using 2tb drives! Note on expanding capacity: Adding another drive has the same risks as rebuilding. If you do it, back up your data somwhere then make a fresh array or if you are curious add more drive(s) and follow the resize steps. It is possible to resize while using the array but it will be degraded and take longer. Summary: The perc 6i is a great affordable raid 6 solution for people with some tech ability and patience. I was able to build a six drive raid 6 array for the price of an off the shelf raid 6 card (without any drives!). IMG_0040 by ASSEMblerEX, on Flickr IMG_0044 by ASSEMblerEX, on Flickr IMG_0048 by ASSEMblerEX, on Flickr IMG_0046 by ASSEMblerEX, on Flickr IMG_0052 by ASSEMblerEX, on Flickr IMG_0051 by ASSEMblerEX, on Flickr
Almost as cool as me getting 32Gb of storage for my PSP via PhotoFast CR-5400 for the price of a 16gb Pro Duo So I guess thats what you were talking about for days -_- I had figured you were just going to get a standard RAID card, didn't think you were trying to chapskate in any way... given as it is you after all I should have known better I used to use RAID3 on my desktop bfore it blew up queing for an Arena match on Warcraft... I really should learn to start using surge protectors more often, especially givn my bad experience with this particular socket in the past =/ If people are going to RAID though I do reccomend going for some parity too - I've had to perform miracles on a mirrored RAID where the primary drive failed and the backup drive only ever backed up once, several years ago. In the end I had to buld a Frankenstein to pull the data off and it only lived long enough to do the copy. Very nice though. Time to roll out the goat porn, eh?
If you care about the data at all then some parity is essential. My 2x500GB RAID1 array died recently. At first I was angry, but the surviving drive worked fine and allowed me to save the data, which is, after all, the whole point. Compared to the personal value of the data, the cost of new drives was nothing.
A very nice guide, ASSEMbler. Something I've thought about for a while. At the moment I upload all important data to my server (raid6 based) since I have basically unlimited space but in the future I'd also like my data stored locally for peace of mind.
Most server level RAID cards are PCIe x8 cards anyways, so it doesn't matter if the slot is electrically above x8 (especially if the card uses PCIe 1.x spec and the mobo uses the 2.x+ spec, in which case you could, in theory, use a physical x8 or x16 slot that is electrically at least x4 with 2.x+ spec). Also, you only have a certain amount of throughput via the PCIe slot so an expander box may not be ideal, since the other drives on the card must also share from that available throughput.
Nice work, I'm looking to do something similar but with not quite so much parity. Did you mean RAID1? RAID0 is block level striping. I'm sure you meant to say RAID1 but I'd hate for you to have actually configured as RAID0 and not have the redundancy that you think you do on the boot drive.
Actually I wouldn't be surprised if it was RAID0, I ran an OS partition on RAID0 myself for a while. You get speed benefits and if an OS partition collapses it's not the end of the world (assuming you make regular backups of bookmarks etc).
Question: Is a hardware RAID controller better than software? I noted that when I purchased a 1TB drive for someone, it was actually two 500GB drives in striped (or was it concatenated?) mode or something. When the proprietary controller inside the case died, I was screwed. And there was no way to replace the controller as it was some unknown generic deal. Would it be better to just get Linux and have it implement RAID in Software? That way the RAID implementation is open and when something fails, you can always fix it since it's not hardware dependent?
Hardware raid is crucial. If errors occur in raid with no error correction, corruption of data will occur. External devices tend to use certain chipsets, but yeah you are screwed when they die. Industry cards are proven, built to work 24/7/365 and have extensive testing and support. If the card dies, get a replacement and it will rebuild properly.
Hardware RAID is better, especially if it has its own RAID processor. The downside is that these can cost a fair amount more than a cheapo RAID card. Also, a RAID card at that level is usually used for enterprise or critical storage requirement applications, rather than just putting together a RAID box. The upside is that with a card with a RAID processor is that there isn't much in the way of CPU load due to the calculations required for the RAID process is done on the card instead. With memory on the card being used for cache, there's less memory usage as well. Low end cheapo cards are only marginally better than software RAID using an on-board chip since in almost all cases, the CPU still has to do the calculations. For hardware RAID, I'd go with an enterprise class card that meets the needs and is affordable.
You can get the card I bought for under $120 with battery, it's an industry card that costs $400plus in it's unrebranded form. Just an update: Time to initialize and check 10TB: 36 hours Time to encrypt 10TB: 72 hours
