Opinions on storage system

Got a shopping cart of parts that you want opinions on? Get advice from members on your planned or existing system (or upgrade).

Moderators: NeilBlanchard, Ralf Hutter, sthayashi, Lawrence Lee

Post Reply
protellect
Posts: 312
Joined: Tue Jul 24, 2007 3:57 pm
Location: Minnesota

Opinions on storage system

Post by protellect » Wed Jun 15, 2011 1:33 pm

Hello.

I'm looking for some opinions on how I'm upgrading from my current storage system, in particular any pros and cons for operating systems.

I'm familiar with all manner of operating systems, including Ubuntu Server, Server 2008/R2
I am also comfortable with virtualization platforms, in particular Vmware ESXi [Now, ESX.]
I have not used Windows Home Server much, and very little [dabbling, really] NAS style OS like freenas or nexentia.

I'm currently using a MS Windows Server 2008 system with a small OS hard drive, and 4x1TB WD Green drives, each drive in the system as a single drive [no RAID] and tasked for different things [media, movies, music, documents].
It's a supermicro board with 2GB memory and a dual-core atom a SM SAS expander card

I back each drive up to 3x1.5TB WD Green drives in my [Win7] media server. I also have two 2TB hitachi 7200RPM drives, and 4x1TB RE3 WD 7200RPM drives sitting around that I have up on craigslist but no buyers, as well as a 2.5" 7200 RPM 500GB drive, and another 500GB 2.5" seagate momentus XT sitting around. Also two Supermicro 4x3.5" drive in 3x5.25" hot swap caddies that work.

I have bought two ASUS E35M1-I Fusion Boards, two Antec 380W Green Power Supplies, two Antec 300 Cases, and 4x4 GB DDR3 memory. I plan on recycling exisiting HDD's, so to sum it up;

2x2TB Hitachi 7200RPM
3x1.5TB WD 5400RPM
4x1.0TB WD 5400RPM
4x1.0TB WD 7200RPM Raid Edition
2 x 2.5" Laptop Drives

So, each case will have a dual core 1.6ghz AMD each with 6x SATA3 ports, 2x4GB memory, 380w power supply and gigabit ethernet, and possibly 4x hot swap bays.

What I'm looking for opinions on is
How would you configure the HDD configuration to support file-level mirroring [probably with syncback SE or rsync, depending] between the two machines
Any opinions on recommended operating system(s), I have access to MS Window Server 03, 08, MS Home Storage Server , HSS2011, and any bsd/linux distros freely distributed.

One idea is a RAID1 OS install of server 2008 on the WD RE7200RPM drives, then move each drive into one or the other machines.

I've also contemplated installing ESXi, partitioning each individual HDD to an instance of something like FreeNAS, and then doing all file transfers over iSCSI to a host server 2008 OS, as well as my desktop and or home media server.
This seems very flexible but also overly complicated, adding several extra layers and passes through the CPU stack.

cordis
Posts: 1082
Joined: Thu Jan 15, 2009 10:56 pm
Location: San Jose

Re: Opinions on storage system

Post by cordis » Wed Jun 15, 2011 2:36 pm

Well, sounds like you know more about this stuff than most of us, not sure if I can really help. But I do use Ubuntu, and it would be fine for doing something like what you're looking at. I have an ubuntu machine with all my spare drives in it, and I've been looking into using it for a backup to my NAS. The easy way to do it would just be to use LVM to merge all the disks, and then you can rsync off of your other server (or maybe use Unison for bi-directional syncing, it's unclear what kind of use you may want out of these machines besides backup). You'll need to use samba to make the disks visible, but that shouldn't be too hard. You may want to use windows on the other machine, if you're still doing streaming or whatever from it, but you could also make it a 2nd linux box with the same kind of configuration. If one machine backs up the rest of your network, and the 2nd box rsyncs a copy from that box, that might work. Or one machine could do backups and sync to the 2nd machine. I've actually been in the process of using the same board in a system to act as a PBX, and using that machine as a backup for my main NAS backup system. I'm leaning more toward writing my own fuse filesystem, but rsync or Unison with LVM should do the job ok.

HFat
Posts: 1753
Joined: Thu Jul 03, 2008 4:27 am
Location: Switzerland

Re: Opinions on storage system

Post by HFat » Wed Jun 15, 2011 3:14 pm

protellect wrote:What I'm looking for opinions on is
How would you configure the HDD configuration to support file-level mirroring [probably with syncback SE or rsync, depending] between the two machines
Any opinions on recommended operating system(s), I have access to MS Window Server 03, 08, MS Home Storage Server , HSS2011, and any bsd/linux distros freely distributed.
My opinion: use hardlinks to keep several versions. As with everything, there are pitfalls (like performance is you have a lot of small files) so I don't know if it would be a good idea for you with the amount of information you've provided.
You could do it on any POSIX-compliant OS such as Windows. But really, if you have no compelling reason to use Windows, avoid! It's trouble and lacks flexibility. The most obvious OS options would be Debian and RHEL (or clones thereof) I think. Maybe Ubuntu LTS wouldn't be such a bad idea but avoid the non-LTS versions (as well as Fedora).
Rsync can do the hardlink magic but it's only a tool. You could roll your own and get exactly what you want or use somebody's scripts (like: http://schapiro.org/schlomo/projects/rbme.php ). There are even a few GUIs out there (like: http://backintime.le-web.org/ ).

As far as HDD configuration is concerned, use JBOD if at all possible or LVM pools (on Linux) if you can't split your data between drives and you need to consolidate capacity. You want to use something like LVM anyway even in a purely JBOD configuration.
protellect wrote:I've also contemplated installing ESXi, partitioning each individual HDD to an instance of something like FreeNAS, and then doing all file transfers over iSCSI to a host server 2008 OS, as well as my desktop and or home media server.
This seems very flexible but also overly complicated, adding several extra layers and passes through the CPU stack.
I don't understand what that would buy you. What flexibility? Maybe I don't understand what you're aiming for but it sounds overly complicated indeed... KISS!

Post Reply