RAID of the Paranoid

mpteach · Post by **mpteach** » Sun Jul 04, 2004 6:17 pm

I tend to have a healthy level of paranoia when it comes to data storage. It all started after i lost 2/3 of a semesters work on a bad hard drive crash

Ive used a backup-HD but i think its time for RAID. No need to worry, i have plenty of UPS power, and a hardware firewall.

I know that a RAID 1 array will increase the HD reliability exponentially against normal failure. But what about catasrophic hardware failure?

Could a PSU failure destroy both HD's in the array? What would happen if a bug flew into my PSU and the thing overheated and something shorted out and the thing blew. Would it be possible for the PSU to send a 100 volt spike through out the sytstem frying all my components simultaneously? Do they have fuse protection on the output lines?

josephclemente · Post by **josephclemente** » Sun Jul 04, 2004 6:44 pm

Paranoia is a good thing when it comes to hard drives. At work, our server RAID drives died at nearly the same time. All of them. There was no time for any recovery. We trust our data to tape backup - but need an alternative soon because our 40GB (after compression) tapes are small.

At home, I've started backing up to another hard drive on my gigabit network. I may do something similar at work, but use multiple, removable hard drives as well. So I'm experimenting to find a good solution.

It is pretty simple, but now I'm having trouble with speed. Doing a normal file copy, my tranfers are FAST. Using Windows XP's built in backup software, the backups seem to be extremely slow.

I tried the trial of Acronis True Image 7, and it seems to be a lot faster than XP's backup, when I turn off or use "low" compression. It still isn't anywhere near as fast as just copying files.

I can copy files from my main computer to the HTPC computer at about 51 megabytes per second. It is a little slower when I copy from the HTPC back to my main computer due to differences in hard drive write speeds.

So far I like True Image, and might consider the server version for the server at work. If anyone knows of anything faster, I'd be interested...

Putz · Post by **Putz** » Sun Jul 04, 2004 7:00 pm

Like josephclemente suggested, I'd recommend network-attached backup. A cheapo used Celeron or something similar with a nice, big ATA hard drive (possibly a controller to go with it) and a fast network card is all you need hardware-wise. Then pick some software that will synchronize an image on the backup server with your actual files, schedule it to run on a regular basis (nightly/weekly/whatever), and you're set. Then you'd really only need to worry about natural disasters (fire/flood/tornado/etc.) and theft, and the only real solution to those threats is removable media stored offsite and a regular backup routine.

Elite · Post by **Elite** » Sun Jul 04, 2004 7:01 pm

RAID 1 was never meant to be a backup solution, nor does it function well as one. It's purpose is to eliminate downtime in the event of a drive failure. It does not guard against viruses, corruption, accidental deletion, theft, power irregularities, or any kind of total system meltdown. Get an external drive or DVD recorder instead.

mpteach · Post by **mpteach** » Sun Jul 04, 2004 7:39 pm

I have an extra HD in an external enclosure but its a pain. I have to hook it up and set aside hours for the backup. In the event of a failure i have to remove the Hd from the enclosure, stick it back in the computer and lose all information since the backup.

My plan was to run a RAID 1 array with 2 partions. Partition#1 would be my small windows boot parition and i would schedule an image backup weekly in case of corruption or a virus etc. Partion #2 would be my Program Files, My Documents, My Shared Folder, etc..

JosephClemente, why did the raid drives at your work have a synchronous destruction? Was it heat or power related? I could deal with heat its the power im worried about.

Non hardware issues such as fire water and the boogie man dont concern me. I dont have any extra old silent computer around the house to use as a backup.

EDIT: josephclemente what do you mean by failure of the raid drives, do you mean they were physically cooked/fried and battered or just complete data corruption?

josephclemente · Post by **josephclemente** » Sun Jul 04, 2004 8:11 pm

mpteach wrote:JosephClemente, why did the raid drives at your work have a synchronous destruction? Was it heat or power related? I could deal with heat its the power im worried about.

I'm not sure the exact cause, it is still a mystery. Our system builder replaced the drives (10,000 RPM SCSI, made by IBM) under warranty. I'm not sure if they replaced anything else. I remember one drive was still spinning before we sent it back, but that one died too.

I opened the side case panel after it failed. Powered it up and saw that the rear 80mm exhaust fan wasn't moving. I physically moved the fan and it started running again. This could have been a heat issue. However, our system builder told us that their diagnostic tools indicated that the drives were exposed to excessive shock. I have never heard of a drive having a G-force detector. Regardless, I know the system was treated with extreme care.

For those who skip over long messages, here is the most important issue I'm having right now:

While I am using a second computer on the network for my home backup, I am considering using a rotation of removable IDE cartridges for the work computer. They'll have to be inexpensive since we'll be getting a number of them, but reliable. If anyone has experience with these cartridges (and their use with Windows 2000 Server) I'd be interested in knowing of any problems or recommendations if any. Importantly, we should be able to hotswap the drives without any rebooting. The drives should be able to spin down before the cartridge is pulled. Thanks in advance.

mpteach · Post by **mpteach** » Sun Jul 04, 2004 9:42 pm

Joshephclemente i think it was a HEAT issue. Heres a link where a guy had a random "excessive shock" SMART error on his IBM DEATHSTAR drive.
http://www.resellerratings.com/forum/t36147.html Those drives have serious heat issues and are being sued against. IM guessing IBM (interesting bowel movements) just gives odd eror codes for heat failures.

Or maybe the LAN party got wild and the asistant with two left feet started rack dancing and excesssively shocked all the drives at once.

mpteach · Post by **mpteach** » Sun Jul 04, 2004 9:56 pm

ON TOPIC what are the chances of non heat related total Raid failure or total sytem meltdown? Using the raid partion sheme described above i should be pretty safe.

kesv · Post by **kesv** » Mon Jul 05, 2004 12:06 am

mpteach wrote:ON TOPIC what are the chances of non heat related total Raid failure or total sytem meltdown? Using the raid partion sheme described above i should be pretty safe.

A powerspike can always kill your drives. You did mention that you are behind an UPS, so that narrows the risk down. It's also possible for a powerspike to reach your machine through any other cabling that is connected to it. Think network, modem or monitor cables.

I'd think that the chance of your psu dying and producing a spike on it's own is slim. Atleast assuming you have a high quality PSU. Still it can't be totally ruled out either.

The point is that 100% security is not possible. This is also part of the reason why RAID 1, while better than a single drive, still does not make backups unneccesary. Also remember that for best effect the backup should never be attached to the machine that was backed up, for longer than neccesary anyway.

For best cheap solution I'd go with raid 1 and a backupdrive in an usb enclosure (or firewire).

PS. According to several studies performed by companies that work in the datastorage business, most datalosses can be followed back to software error or human error.

josephclemente · Post by **josephclemente** » Mon Jul 05, 2004 12:54 am

I agree with kesv.

Really, I think a rotation of multiple backups are important. These additional backups will be disconnected for the best security.

I think RAID is a great start. But don't end it there - there is definitely room for disaster. I know I've learned the hard way...

Putz · Post by **Putz** » Mon Jul 05, 2004 3:59 am

mpteach wrote:I dont have any extra old silent computer around the house to use as a backup.

A 500-1000Mhz-class computer can be had for ~$150. Once you replace whatever hard drive is in it with your backup drive (assuming such a drive would be chosen with quietness in mind), silence is just a fan swap or two away. This machine would not need a monitor once it is set up, and keyboards are not exactly expensive these days. All you need is a little bit of space and possible some network wiring -- the rest is easy and relatively cheap.

In any case, it's a small price to pay for a set-it-and-forget-it backup solution.

Jan Kivar · Post by **Jan Kivar** » Mon Jul 05, 2004 6:06 am

josephclemente wrote:While I am using a second computer on the network for my home backup, I am considering using a rotation of removable IDE cartridges for the work computer. They'll have to be inexpensive since we'll be getting a number of them, but reliable. If anyone has experience with these cartridges (and their use with Windows 2000 Server) I'd be interested in knowing of any problems or recommendations if any. Importantly, we should be able to hotswap the drives without any rebooting. The drives should be able to spin down before the cartridge is pulled. Thanks in advance.

Well, I've been eyeing this Kingwin USB 2.0 external mobile rack. No experiences yet.

Cheers,

Jan

mpteach · Post by **mpteach** » Mon Jul 05, 2004 9:07 am

firewire is much faster when backing up a HD than usb 2.0. Also firewire uses less cpu power.SATA2.0 is supposed to have an external SATA standard which will be awesome.

All my computer components are spread across two UPS's including the cable to my cable modem so im not worried about lightning. If lighting did go through my window and fryed my computer it would probably fry a newtworked backup and/or burn my house down.

Puting windows on a seperate partion really helps prevent against human error and viruses/worms. My firewall heps with worms. I think any nasty virus that could wipe all my drives could probablly also wipe a newtwork drive also.

Convienence is an big issue. extenal disconnected backups are less convienint than raid or network drives and thus i wouldnt use them often enough.

MY real question is this: Is a RAID 1 setup up any less reliable than a network drive? Especially if the Raid is on a quality PSU and has good cooling. Would it be worth the extra expense?

Im no actuarial, im just wondering if the difference in risk is lower than the risk of gettting hit my an 18 wheeleer while crossing the street or my neighbor burning the builiding down, or even as small as wining the lottery.

I know that a psu with fuse protection on the outputs couldnt fry anything, though i dont know if that how thier made.

Ofsite monthly backup might become an additon in the future when i get more money, maybe.

greeef · Post by **greeef** » Mon Jul 05, 2004 3:35 pm

Something as simple as password access tonetwor folders is usually enough to stop viruses spreading. Personally i only allow copying from, not to, on my home network.

griff

bomba · Post by **bomba** » Wed Jul 07, 2004 6:55 am

josephclemente wrote:Paranoia is a good thing when it comes to hard drives. At work, our server RAID drives died at nearly the same time. All of them. There was no time for any recovery. We trust our data to tape backup - but need an alternative soon because our 40GB (after compression) tapes are small.

At home, I've started backing up to another hard drive on my gigabit network. I may do something similar at work, but use multiple, removable hard drives as well. So I'm experimenting to find a good solution.

It is pretty simple, but now I'm having trouble with speed. Doing a normal file copy, my tranfers are FAST. Using Windows XP's built in backup software, the backups seem to be extremely slow.

I tried the trial of Acronis True Image 7, and it seems to be a lot faster than XP's backup, when I turn off or use "low" compression. It still isn't anywhere near as fast as just copying files.

I can copy files from my main computer to the HTPC computer at about 51 megabytes per second. It is a little slower when I copy from the HTPC back to my main computer due to differences in hard drive write speeds.

So far I like True Image, and might consider the server version for the server at work. If anyone knows of anything faster, I'd be interested...

I'm in the paranoid crowd and am pretty religious about backing up a large multimedia library and irreplacable family photos. I used to use Ghost to image the system & data drives, but w/ around 80G on the data drive this took too long. I still use Ghost to image the C: drive (partition), but have been using the incremental backup feature of the built-in WinXP backup utility to backup the data drive. Since it only backs up files created or changed since the last back-up, it is fast. Also, I've done a 2nd backup of the aforementioned photos to DVD-RW's which are kept in my safe deposit box.

peerke · Post by **peerke** » Wed Jul 07, 2004 11:23 am

bomba wrote:Also, I've done a 2nd backup of the aforementioned photos to DVD-RW's which are kept in my safe deposit box.

I've read a few articles on the durability of DVD-R's with all the same outcome: Even well stored DVD-R's can start to deteriorate horribly within as little as one year!
This would mean your irreplacable photo's are not save this way unless you make new copies every six months or so. I would choose to back them up on a harddisk and keep that in your safety deposit box. O well, maybe I'm just being paranoid

.

bomba · Post by **bomba** » Wed Jul 07, 2004 11:58 am

Yeah, I did read about DVD rot too, but hey multiple backups in two geographic locations, that's pretty paranoid eh? Hopefully at lease one of three copies will be ok at all times! Remember that the HDD is not infalible either, one accidental drop and it may become a paperweight...

Putz · Post by **Putz** » Wed Jul 07, 2004 1:50 pm

mpteach wrote:I think any nasty virus that could wipe all my drives could probablly also wipe a newtwork drive also.

In my proposed solution, the backed-up drive image wouldn't even need to be visible on the network. It's a true set-and-forget solution that should work as well as anything short of an off-site backup regime.

mpteach · Post by **mpteach** » Wed Jul 07, 2004 6:20 pm

I might still get raid, but ive seen the need for some sort of isolated off site storage.

mpteach · Post by **mpteach** » Wed Jul 07, 2004 6:23 pm

I have a solution

1.I can take some of my old hard drives and put them in external enclosures

2. Put those external drives in a foam lined box in the trunk of my Car.

Not only would they be offsite, and impervious to my house burning down , but they'd be mobile.

Any comments suggestions?

I might even get to write an article about this.

markjia · Post by **markjia** » Thu Jul 08, 2004 11:46 am

Interesting idea...but considering that there are substantially more car accidents than house fires, the box would have to be very well protected against fire and maybe flood. (maybe in a water tight, fire repellant cage).

Plus, a safer place in the car might be the rear passenger seats rather than the trunk. And if you have passengers, just have them crawl in the trunk instead.

mpteach · Post by **mpteach** » Thu Jul 08, 2004 12:29 pm

An accident proof case might be a good idea. If it did get damaged in an accident id still have the primary Hd in my apartment, Likewise if my apartment got damaged id have the backup in my car..

The convience of always having mobile storage close by is a real plus.

I wonder if i could buy a case, like one used for transporting cameras and sensitive equipment, might have to modify them or make something. Any suggestions?

peteamer · Post by **peteamer** » Thu Jul 08, 2004 12:45 pm

Good idea.

Bearing in mind the G's a HD is capable of surviving... what were you intending to do to the car to make a box with foam etc. in necassary ????

Cars are designed to protect low G threshold humans... Put the naked HD in/around the centre and you'd have to go some to damage the thing.

Having said that... I'd use a 'camera' case with foam block cut out to store the HD

If you want to do back-up 'properly' it should be ... 1 back-up on site and two others in seperate off site locations. But that still wont stop Murphy's/Sodt's Law...

Pete

mpteach · Post by **mpteach** » Thu Jul 08, 2004 1:25 pm

I drive over a nasty set of speedbumps everyday, so i think faom will help the HDs in the long run.
...If only i knew how to make side impact airbags for my hd cae.J/k

1 onsite Raid1 and 1 offsite(car) backup will be fine for me. Thats 2.3 times more backup than the status quo. Backing up "properly" is when you have people out to get you especially a guy named murphy.

I hate to say this, but the more convienient backing up is, the more often i do it. Also with too many crazy backups everywhere theres the chance of one of them getting permanantly borrowed.

alock · Post by **alock** » Thu Jul 08, 2004 3:33 pm

There will most likely be people who will disagree with this but...

The most important aspect of a backup is that it is correctly backing up your data. You should therefore periodically test your backup.

If you Ghost your C partition, try formatting it and restoring your image. If you have a RAID, shutdown your machine, unplug a drive and reboot. The same applies to anything you backup.

If you are not prepared to do the above, you don't trust your backup. If this is the case you might as well save yourself some time and not bother backing up.

This method was used at a company I used to work for. We would install and fully setup a server and take the first backup. We would then reformat all of the hard disks and restore them from backup. It is not easy to format a drive you have spent several days configuring, but it has to be done to test the backup.

bomba · Post by **bomba** » Thu Jul 08, 2004 4:46 pm

alock wrote:There will most likely be people who will disagree with this but...

The most important aspect of a backup is that it is correctly backing up your data. You should therefore periodically test your backup.

If you Ghost your C partition, try formatting it and restoring your image.

I most definitely don't agree with destroying the drive you just imaged in order to test your image. What if the image IS corrupt? Besides, Ghost has a function which will verify the integrity of the image without having to resort to this Russian Roulette style backup! I do agree about having confidence in and testing the integrity of your backup, but not like this.

Putz · Post by **Putz** » Thu Jul 08, 2004 6:07 pm

This is particularly easy, yet spectacularly superfluous when talking about ordinary data (documents, multimedia, etc.). Most of us (and I think mpteach said so as well) back up only our irreplaceable data, since a day or two of downtime on one of my machines is usually only a minor annoyance, but losing my documents would be crushing. However, when the system drive is being fully backed up in a mission-critical application, I agree that one should be comfortable enough with their backup to not be nervous restoring from a backup as a test. If you're not comfortable with the idea (or worse, if you run the test and it fails), your backup is probably not good enough.

mpteach · Post by **mpteach** » Thu Jul 08, 2004 6:08 pm

I think i found my case http://pelican.com/cases/1bo_1400.html

Its virtually indestructable and hermetically sealed(waterproof). The box comes filled with a high density foam block inside, and i have to carve out individual spaces in the foam for each component. I'll probably stack the 2 HD enclosures( w/HD's inside)on top of each other with 3 cm of foam between them. And put all the cables and power adapters on the rigth side of the box.

The box is $51 and a lock is $9 if i want to get one.

If i ever drive off a bridge and sink into the water; I could use this case as a floatation device and still know that my HD's are safe:)

mpteach · Post by **mpteach** » Thu Jul 08, 2004 6:18 pm

Putz, try the Russian roulette backup test with blanks instead of risking your livelyhood.
AKA make a second backup before trying the test. Duh

alock · Post by **alock** » Fri Jul 09, 2004 12:14 am

The point I'm trying to make is only perform a backup that has value. The point Bomba made about not destroying your data is valid, not because it is too drastic, but because it is not drastic enough.

For those of you at work/college/school when reading this. When you arrive home tonight, pretend one of the following:
1) You open a document you were working on yesterday and find it is corrupt.
2) You press the power button on your computer and nothing happens.
3) You find your front door open and every electrical item in your house missing.
4) You see your road full of fire engines and a smoldering pile of rubble where your house used to be.

The above scenarios happen to many people every day. I'm not saying your backup has to cover all of them, but have a perspective on which of these you want to be able to restore your data from. I (try to) protect myself from these with the following:
1) All data is backed up to a NAS device (mini-itx pc).
2) Once a week, working set backed up to CD and hidden in the house (not perfect but not a bad idea)
3) Critical data is backed up from my NAS device via FTP to server in telehouse in London every night. My business partner then downloads this to his computer which he creates our release builds from. This creates a perfect test of the backup.

By the way, I am not going to burn my house down to test my backup

[/list]