Advice on low-powered file-server build

Got a shopping cart of parts that you want opinions on? Get advice from members on your planned or existing system (or upgrade).

Moderators: NeilBlanchard, Ralf Hutter, sthayashi, Lawrence Lee

Post Reply
injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Advice on low-powered file-server build

Post by injuryprone » Tue Jul 27, 2010 10:46 pm

Hi,

I recently acquired an Intel i5 661 CPU and some WD 2TB Green HDDs and was thinking of using them to build a low-powered file server (which seems to be the trend at the moment).

I have an old Antec SOHO 1080AMG case and am hoping for advice on a motherboard, RAM and power supply. Would like to fit 8 drives (2xRAID1 OS and 6xRAID 5/6/10 storage) and for the PSU to be fairly quiet as this will be sitting in the bedroom now that my kids took over my computer room. :(

Any suggestions would be greatly appreciated!

Cheers,

Tom

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Wed Jul 28, 2010 3:29 am

Are you going to get a RAID controller card? Not a cheap FakeRAID card but a real hardware RAID controller? If so most of what is bellow is irrelevant.

If no hardware RAID:

What OS are you going to use? Linux and FreeBSD have good softRAID implementations. Windows softRAID is OK for RAID 0/1/10, but not 5.

If you are using Windows and want good RAID 5 performance without an expensive RAID controller then you are going to need an H57 board, or a P55 with an external GPU. For RAID 5 Intel Matrix RAID / Rapid Storage is much better than other fakeRAIDs or Windows software RAID.

The only H57 board I can see on a quick search with 8 SATA ports is the Asus P7H57D-V EVO. Not specifically recommending it, I've never used it. You could use the 2 SATA 3.0 ports as your boot RAID 1 and then the 6 Intel RAID ports for your RAID 5. I have a similar setup in my server, but using an older P45/ICH10R 775 board.

I have a Earthwatts 380 powering my 9 drive server with no issues. You probably want something a bit quieter, but power wise something in the same range will be more than enough.

april.shower
Posts: 2
Joined: Wed Jul 28, 2010 11:47 am
Location: Germany

Post by april.shower » Wed Jul 28, 2010 12:57 pm

The 1156-platform is with its two-chip-design clearly the most advanced energy saver. Very good choice in so far. In addition Clarkdale and Lynnfield CPUs show an unmatched sophisticated integration and interaction of boosting cores (Intel Turbo Boost) and energy saving mechanisms.
Don't get me wrong. But nethertheless I wonder (par example) what exactly do You intend to do with the considerable grafic solution built in this cpu.
Ok, I understand that consumer decisions in general are not that accurate. Because there are many purposes for consumers and the by mentioning emphasized purpose ("file serving") isn't as a rule a full time job (so to say). Therefore consumers tend towards all-purpose equipment.
And bingo, i5 661 is the nearly ideal general purpose cpu. Can absolutely compete with the strongest AMD Quadcores, has the mentioned built-in grafic solution, comes with a new instruction set for lightening fast encryption, ... that's all very nice.
But You know that there is no ECC support by Intel non-server cpu's and that the 3D abilities of the i5 661 are not required for pure file serving purposes. (AMD consumer cpus support ecc but the feature is useless if the mobo bios doesn't support it.)
I personally prefer more specialized solutions only for a few narrowly defined purposes. In this sense I'm going to build up a Windows Home Server whose core application is basically nothing else than "file serving". If you're in the mood throw a glance at the beginnings of my "file server"-solution here: april-shower()spaces()live()com (feel free to leave a comment, microsoft live id necessary (the parentheses substitute points)).
Hereby I want to say that I would have recommended more server-like components if You already had not purchased the i5 cpu. Under the given conditions as well the Asus P7H57D-V EVO would not necessarily be my first choice. Asus and also Gigabyte are not known for making energy saving boards rather their boards comes usually with a jumble of features that may increase energy consumption.
And so this Evo-Board: for further info look at xbitlabs in mainboard section. Basically the same with the Asus P7H55D-M EVO: see also at xbitlabs
What about the Intel DH55TC or the MSI H57M-ED65? Or the Intel® DH57JG (Mini ITX with one PCIe 2.0 x16 slot) or Intel® Executive DQ57TM (Micro ATX with 1x PCIe 2.0 x16, 2x PCIe 2.0 x1, 1x PCI and with Trusted Platform Module, Intel vPro-Administration (Remote) and
Intel Active Management Technology 6.0 Professional) ...
Regarding power supply I would suggest You one of these brand new Seasonic X - PSU. Power requirements at start 8 EADS or EARS * 12V * 1,75A = 168 Watts + Sata Add On - Card (about) 4 Watt = 172 (DC) for storage, Sata Raid Card about 6 Watts more than a Sata Add On - Card
i.e. the fanless Seasonic X-400 or X-460 would be sufficient, unfortunately only available in August or September.
With regard to currently available devices one can say on the basis of spcr test results that the Enermax 82+ devices are unusually effizient (measured by the bronce certification) in the lower load.
Regarding RAM I would suggest DIMMs within the specification whatever the circumstances are (DDR3-1066 (CL7, 1,5V), DDR3-1333 (CL9, 1,5V))

P.S.
The Server is in the bedroom?
How solid is your sleep? How easily do you sleep if there is slight monotonous noise?
Passive cooling the i5 cpu may be possible with a big cooler. Scythe Orochi and Noctua NH-D14 good results but then still one case fan at least is required. Besides the bunch of hard drive disks need some cooling, otherwise these roast in their own juice.
By the way it would be very interesting to hear more of the behavior of these tending to fall asleep "green" hdds under raid conditions in the long term.

piglover
Posts: 134
Joined: Mon Jun 08, 2009 9:32 am
Location: California

Post by piglover » Wed Jul 28, 2010 2:46 pm

I'd go with the Intel H57 MB (BOXH57DD). Its rock-solid and doesn't have a lot of bells/whistles that you don't need wasting power. H57 uses the on-die video to keep power use down and also gives you Intel ICH10R-based on-board RAID (h55 does not have raid). Use the on-board raid to build the Raid-1 OS drive you are looking for.

Even more important, it is uses the real Intel ProSet network chip. If you look at other fileserver/NAS builds you'll find almost universal agreement that getting the Intel NIC is the best upgrade you can make for fileserver performance - and this board already has it built in instead of the Realtek junk that most low-end motherboards use.

It doesn't have enough SATA ports for you. You probably won't find a board with 8 SATA ports unless you get a high-end board that has lots of extras you don't want too - extras that consume power and generate heat - and heat is the enemy of silence.

To satisfy you thirst for SATA ports you will want either a Raid card or a 8-port non-raid HBA. If you go with hardware Raid stick with the SAS cards from Areca (ARC1680i for 8 ports) or LSI (9261-8i). Either one will set you back $500-600.

You should really consider software-based RAID or RAID-like solutions. You can get a fully functional fileserver running WHS which uses file duplication instead of Raid. Or you can run Server2008 and get software Raid-5. If you are running Windows (or a Windows derivative like WHS or Server2008) then the one you want is the SuperMicro AOC-SASLP-MV8 (~$100).

You could also try Linux software raid or ZFS.If you are going to run Linux or FreeBSD then the Marvel chip on the Supermicro card has some driver issues - instead look at the Intel SASUCI8i (~$150, but it also does RAID-0/1 as a bonus).

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Wed Jul 28, 2010 8:52 pm

Thanks for the responses! I'm no hardware guru so I really appreciate all your input.

What would the benefit be for choosing Q57 over H57? Is the i5 661 a good choice for what I have planned?


washu:

I was planning on using onboard hardward RAID if possible as I didn't want to shell out on an expensive RAID controller. Does the Marvell Sata Controller support RAID? Asus didn't list the RAID options on their website.
I was thinking RAID 10. Are you not a fan of Windows for RAID 5?

I'm experienced with Microsoft systems so I was leaning towards Server 2008 64bit. It will be used 90% of the time as a file server. Accessed via terminal server or Logmein and FTP. Never to play games, rarely to convert a video, and regularly to stream to a laptop or Xbox. So I guess HDD read/write speed and network throughput would be the biggest consideration (aside from power and noise).

I was thinking a 400ish W PSU as I had read that they reach maximum efficiency at around 40-60% load.



april.shower:

File serving and media streaming is the full-time job for this box. I don't believe I'll require a better GPU.
Do you think I would be much better off with ECC for this machine? Obviously I'll have to wait til I pick a board first before I choose (and check RAM power usage).

Would a Mini ITX or Micro ATX be much more energy enfficient than a uATX board?

I translated your website from German and it made some sense, but your graphs confused the crap out of me. :)

If the X-400 or X-460 isn't too expensive then I can wait a few weeks to purchase the PSU. How is the noise-level on the Enermax? What do you think about the Seasonic S12II?

I have newborn twins, my sleep is limited so the less noise the better!



piglover:

Something basic without the bells and whistles to reduce power consumption sounds perfect (although I'm liking the EVO) as long as it 's not at the expense of performance. If that's the case with the Intel H57 MB you listed then that could be a great option, although I'd have to purchase a RAID or HBA card.

Is that the average price I'm looking at for a good RAID card or HBA? I was hoping for $200 max if I had to buy one. Maybe I'll go the SuperMicro option as I'll be running Windows 2008. How do you rate the performance of Server 2008 software RAID as opposed to spending $500-600 on a hardware RAID card?



Cheers,

Tom

piglover
Posts: 134
Joined: Mon Jun 08, 2009 9:32 am
Location: California

Post by piglover » Thu Jul 29, 2010 9:13 am

Q57 has better virtualization support, vPro workstation management, and support for more advanced management features used in datacenters. Otherwise, not much different from H57.

The cheaper Raid cards - the low-end highpoints, low-end LSIs, Silicon Image, etc, are really just simple HBAs that implement Raid-5 through their drivers (i.e., in software on your PC). There is no advantage at all in using them if your OS will support software Raid (as most Linux and Server2008 does).

The Marvel based board (the AOC-SASLP-MV8) does not even pretend to support Raid - but you can certainly run software raid using it. I know - I do it today. Its not a high performance solution - but as a fileserver, you performance is always going to be limited by the performance of you LAN and almost any software Raid can exceed the 100 MB/s read/write performance of a perfectly optimized GigE LAN.

piglover
Posts: 134
Joined: Mon Jun 08, 2009 9:32 am
Location: California

Post by piglover » Thu Jul 29, 2010 9:36 am

Another H57 option for your specific app might be the Gigabyte ga-h57m-usb3. This board has a separate Jmicron SATA controller that gives it two more SATA ports with raid-1 capability.

For your desired config, you could run the 2xraid-1 OS drives from the Jmicron controller ports and the other 6 drives in a big raid-5 using the Intel SATA ports.

Only problem you have doing this is that Gigabyte puts one of the Intel SATA ports on the I/O panel as an eSATA port. To use all six drives in your case you'd have to get an eSATA-SATA cable and loop it back into the case. It would be a little Ghetto, but definitely easier and less expensive for you.

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Thu Jul 29, 2010 1:53 pm

injuryprone wrote: I was planning on using onboard hardward RAID if possible as I didn't want to shell out on an expensive RAID controller. Does the Marvell Sata Controller support RAID? Asus didn't list the RAID options on their website.
I was thinking RAID 10. Are you not a fan of Windows for RAID 5?
Unless you are shelling out really big money for a server board, none of the controllers on a MB are going to be hardware RAID. They are at best fakeRAIDs.

The Marvell controller doesn't support RAID at all, and it only controls two ports on that board. It is perfectly fine to use for a software RAID1 for your boot volume.

The other 6 ports are controlled by the Intel chipset. It is a FakeRAID, but by far the best one out there. It can hold it's own with several hundred dollar real Hardware RAID cards at the cost of only a bit more CPU usage. The CPU time isn't really much at all, unless you were maxing out all the time while simultaneously pounding your disks you will never notice the difference. Of course that only applies to RAID 5. RAID 0, 1 and 10 use pretty much no CPU no matter what controller you have.

The reason I don't recommend Windows for software RAID 5 is that it performs like crappy fakeRAIDs. Windows and most fakeRAIDs (not Intel) are not smart enough to do full stripe writes to avoid having to read-modify-write all the time. It makes writing to the RAID 5 very slow. Reading is fine. Linux & FreeBSD softRAID and the Intel fakeRAID are smart enough to avoid this problem whenever possible and thus perform much better.

So really all this only matters if you want the specific case of RAID5 with Windows. If you want to do that then you need an Intel chipset that supports it or you need to drop big bucks on a hardware RAID. If you want to do RAID 1 then the software RAID in Windows is more than fine making the controller irrelevant. If you want to use Linux or FreeBSD again the controller becomes irrelevant.

If you want to do RAID 10 then you need something that supports it, but almost any card that does will perform fine. Windows doesn't do RAID 10 in software, so you need some kind of controller to support it.

If you end up buying an add on HBA make sure it's PCIe. PCI just doesn't have the bandwidth to keep up with modern disks / network.

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Sun Aug 01, 2010 4:13 am

And here I was thinking how setting up a cheapish/low-powered RAID was gonna be easy! :P
Thanks again for all the info guys. I had no idea about non-RAID HBAs and fake raid.

So to keep the speed up I've decided on RAID 10. Which means hardware RAID; as Windows 2008 doesn't support it. Which kinda sucks.


piglover:

Very true about the software RAID 1 and the netwoork speeds, looks like the EVO is still on the cards.

I don't mind too much about it looking ghetto but I don't think the motherboard doesn't support RAID 10.


washu:


Good to hear the onboard Intel RAID will be decent enough.

Yeah I've read in a couple places that software RAID 5 on Windows 2008 has some speed issues.



So I guess my options are:

The EVO - Might draw more power but has Intel RAID 10.
A basic motherboard with a RAID controller - Lower power but more expensive.

Is the SASMF8I any good (hoping it's not just a HBA that uses RAID through drivers)? A little more than I wanted to spend but it's PCIe and has 8 connections.

theycallmebruce
Posts: 292
Joined: Sat Jul 14, 2007 10:11 am
Location: Perth, Western Australia

Post by theycallmebruce » Sun Aug 01, 2010 5:53 pm

I would think that the 8x 7200 RPM drives will drown out quiet CPU and PSU fans. Unless they are all spun down at night, I wouldn't want that in my bedroom.

Just curious, what's your application? The 6 disk RAID5 would be around 10TB. That's around 2000 DVD ISOs!

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Sun Aug 01, 2010 6:39 pm

The case I am using is pretty good for noise reduction, and the HDDs (for the RAID 10) are WD20EARDS; which are 5400RPM??
Haven't decided whether or not I'll be spinning them down at night as yet. Looking into pros and cons of this.

RAID 10 mate, so 6TB. Storing the usual stuff; photos, movies, etc. Already have a 2TB Mybook which is full, and sucks to work with.

theycallmebruce
Posts: 292
Joined: Sat Jul 14, 2007 10:11 am
Location: Perth, Western Australia

Post by theycallmebruce » Mon Aug 02, 2010 3:35 am

OK then, don't mind me :D

Look forward to seeing it when you're done.

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Tue Aug 03, 2010 11:25 pm

injuryprone wrote: So to keep the speed up I've decided on RAID 10. Which means hardware RAID; as Windows 2008 doesn't support it. Which kinda sucks.
Not to knock RAID 10, but what kind of speed are you looking for? A good RAID 5 will still max out gigabit even on writes. The network will be the limiting factor as long as you don't have a crappy RAID 5 implementation.
injuryprone wrote: Is the SASMF8I any good (hoping it's not just a HBA that uses RAID through drivers)? A little more than I wanted to spend but it's PCIe and has 8 connections.
The SASMF8I is a fakeraid, but probably a good one. The key words here are "Host Based".

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Sun Aug 08, 2010 6:38 pm

I was concerned because I had been reading about slow RAID 5 configurations (<400Mb/s read and <150Mb/s write) and wanted to get the most out of my setup. But these could have simply been crappy RAID 5 implementations, and can't envision myself having the same issues. RAID 10 would have at least reduced any CPU overhead.

Ahhhhh, thanks for the info mate!

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Sun Aug 08, 2010 10:20 pm

injuryprone wrote:I was concerned because I had been reading about slow RAID 5 configurations (<400Mb/s read and <150Mb/s write) and wanted to get the most out of my setup. But these could have simply been crappy RAID 5 implementations, and can't envision myself having the same issues. RAID 10 would have at least reduced any CPU overhead.

Ahhhhh, thanks for the info mate!
While you can do much better, 150MB/ write is not bad at all. Gigabit is 125MB/s theoretical and more like 80-90MB/s in reality with a good setup. Crappy fake raids have write speeds of like 10-20MB/s or lower.

As to CPU time, I doubt an i5 is going to even notice raid 5 vs raid 10. You would have to be maxing out both cores all the time for it to even matter. My backup file server has a Celeron e1200 and CPU is no issue.

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Mon Aug 09, 2010 3:11 am

<150Mb/s as in 19MB/s approx. Might stick to RAID 5 and see how speedy I can get it. At least that way I'll have a lot more storage!

Leaning towards buying the EVO now. Easier on the wallet and has all the features I require.

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Tue Sep 28, 2010 4:06 am

Due to its availability I ended up going with the GA-H57M-USB3 board with 4GB Corsair XMS3 CMX4GX3M2A1600C9 and Corsair VX-450 450W PSU. With 5 2TB drives the RAID 5 initialization took around 3 days and almost a week for the format (7.5TB).

I encountered some issues with what looked like the i5 graphics driver but I stumbled upon an Intel HD driver that worked with Server 2008 and it seemed stable, however this was short-lived.
If I open the Intel HD Graphics or Rapid Storage Technology applications the screen will flash black a couple of times and then come back with horizontal lines (very thin and coloured) and an error "Display driver igfx stopped responding and has successfully recovered").
If I do this too many times it will eventually bluescreen and reboot:
STOP: 0x00000116
igdkmd32.sys - Address 9061BD00 base at 90605000

I installed PerformanceTest 6.1 and it always crashes on the 3D test (All), but is fine when running memory, cpu and 2D tests. I noticed in speedfan that the temperature spikes from 38 to 55 degreesC as soon as I run anything that would cause the crash. Not sure if that's unusual or not though.

Would appreciate any helpful hints! :D

MikeC
Site Admin
Posts: 12285
Joined: Sun Aug 11, 2002 3:26 pm
Location: Vancouver, BC, Canada
Contact:

Post by MikeC » Tue Sep 28, 2010 8:39 am

injuryprone -- can't help you with the immediate problem... but surely you don't really plan on leaving that beast in your bedroom?!! W/ that many drives, you probably can't even hear the babe crying! :lol:

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Tue Sep 28, 2010 12:13 pm

injuryprone -- For whatever reason Intel doesn't supply proper video drivers for server 2008, as you have found out. I simply don't bother with them and let it run in VGA mode. First thing I do after the install is turn on remote desktop and use it from there, so video drivers are pretty much irrelevant.

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Thu Sep 30, 2010 1:47 am

MikeC -- Haha, secretly that doesn't sound so bad (don't tell the wife) :P
, but currently the rear fans are the loudest part so I'm working on finding silent replacements. I'll do it part by part till it's as quiet as possible. Good thing is the kids have music playing which is slightly louder so it's not painfull...........yet.

Cheers mate. Go Intel! How did you go about running it in vga mode? Remove the drivers, choose 640x480 at boot, change display settings?
As soon as I have all the main data transferred to it and streaming is perfect I'll RDP to it but at the moment it's annoying as hell when it crashes testing stuff.

washu
Posts: 571
Joined: Thu Nov 19, 2009 10:20 am
Location: Ottawa

Post by washu » Thu Sep 30, 2010 2:04 am

Just remove/uninstall them. Your video card should say "Standard VGA Graphics Adapter" in device manager.

injuryprone
Posts: 9
Joined: Tue Jul 27, 2010 10:34 pm
Location: Sydney

Post by injuryprone » Thu Sep 30, 2010 2:35 am

Thanks washu. I'll give that a shot now.

Post Reply