This is my new fileserver, named "Monster" because its the biggest and most expensive build of mine yet...

Specs:

Coolermaster Stacker Classic + 4in3 modules
Asus P5WDG2-WS
Intel 3GHz Celeron
Intel PCI-E Gig-E network
Xigmatek cpu cooler (coming soon)
4GB PC2-6400 Corsair ram
3ware 9500S-12 PCI-X RAID controller
12x1TB Samsung F1 (11x1TB raid5 + hotspare)
Seagate 80GB SATA 2.5" 7200rpm system drive
Nexus 120mm fans for the 4in3 modules
Corsair HX520 PSU

Pics: (the important bit)





















Noise:

Honestly, the noise level IS NOT bad, im impressed so far, there was some vibration...

I swapped out the Coolermaster fan with an Antec Tricool in the back, removed the restrictive fan grill, and put in Nexus 120mm on the 4in3 modules... no fan controller as of yet, except the onboard asus stuff.



Performance numbers will be posted when the box is up and running, still initializing the raid array now.

12TB? Are you trying to download the internet?

Good luck.

Looks good btw!

Not quite..



11 drives in raid5 + hotspare, so...

If you apply "P.L.O.H"* rule, then that's alot of freekin porn.

* Porno Law of Hardrive = .5 X total harddrive space = amount of porn files.

just a Q6600?

clever hiding that 2.5" all the way up top

i'm a little surprised myself that one PSU handles 13 drives + quad.

I dont really need the Quad core at all, and for some reason i cant run 64bit linux and get network connectivity with the onboard gigabit, so im gonna put the Q6600 in my ws, and put an E2140 in the fileserver, with HWraid its not an issue anyway

The only reason I got the Q6600 was to run folding@home on it aswell, but with no 64bit I cant run SMP, so no point in having it in there...
I was abit sceptical about the 2.5" at first, but it works fine, so no worries then...

2.5" drive:

.....That's......impressive......(should I say overkill?) .......




(resized)

If you apply "P.L.O.H"* rule, then that's alot of freekin porn.

* Porno Law of Hardrive = .5 X total harddrive space = amount of porn files.



.....
.......
......... .... ............

Drives pull very little power, so that's about the equivalent of the Quad plus just one high-end graphics card. Should be no problem for a high-efficiency unit.

Nice build there! But I'm happy to buy you some zip-ties for those SATA-cables.

Looking at your pictures, I'm guessing you have the same problem I've been having in my system - SATA power cables. I've also resorted to using molex->SATA adaptors for a few of my drives, but I really wish that there were a few 4-port SATA power cables for the Corsair PSUs.

Looks like a nice setup.

Where did you put the 2.5 drive? Can't find it on the pics..

I'm completely puzzled as to why the design has been changed from Molex at all. Probably just to make everybody's life a little more complicated.

sata power cables have one extra wire- surely it must have some sort of use
i think i remember something about hot-swapping sata hdds?

it took me a minute too. its all the way up top. look for a black SATA cable coming off the motherboard, and you'll see it snake all the way up, and then over to the whereabouts of the optical drive.

Is this the SPCR version find the Waldo? Instead: Find the 2,5" HDD?

Anyway, case is awesome looking and that is indeed a monster. 11 TB's sweet! I could use such space ^^. That could take my anime collection like 2,5 times ^^

The SATA power connector was actually a much needed improvement over molex, adding SATA-only capabilities (such as hot swapping), as well as improving the electrical security of the setup. Have a look here for more information.

I'm puzzled as to the shitty performance im getting with the 9500S on 12x1TB... 40MB/s tops write speed on 11 or 6 drives in raid5, and top read speed of about 365-370MB/s... the other box with 9x500GB will do 100+MB/s write easily..

PLOH.. lol..

i hate hard disk noise... 12HD must sound horrible ...

Thats why i chose the F1 drives, performance and pretty low noise... that box is very quiet for what it is... a big hunking monster...

(too bad write performance is lacking for now.. I might have to mail both samsung and AMCC/3ware about this, as its not supposed to be like this at all!)

Some lessons learned from Oberon to this box:

- Know what port is the first one, and how they're numbered...
in Oberon, drive 1 and port0 != the same, drive1 at the top = port1, drive2 = port0, etc.. big confusion, and a big headache when something breaks...
in monster, drive0 and port0 = same, and the order is kept for all the drives, cables are numbered accordingly, with drive0 at the top and drive11 at the bottom...

- Think through how the cabling is supposed to be laid, especially the power cables... Spent atleast an hour doing only the power cables, but I got some good dividends...

As a result of a few precautions and some experience, monster is alot neater built than oberon, even tho im running 13 drives and a DVDRW in monster, compared to 11 drives and no DVDRW in oberon.

have you tried running the system without the exhaust and what appears to be a top exhaust?
considering you have 3 intakes it should work fairly well- the less fans to control the better, right?

@wibla - Are you sure you've set the controller to StorSave-Performance mode? I found that I get slow write speeds when it's on Protection or Balance.

Do you use the 3DM2 to administer your settings or the command line utility?

What are you using to test the write performance? I've always just tried writing dummy files from /dev/zero using dd. You can easily test batch performance for small files or large files with that kind of setup.

Beware - don't set that box on a slippery surface or the torque from all those HD's will make the entire chassis spin in the opposite direction the HD's rotate!!

WOW!... thats huge fucking build. I dont really understand what u will use 12TB of space for thou. I have 1 750gb drive and i dont think i will be able to fill it up in quiet some time.

I'm a big fan of the KIS(S) principle for cooling, front to back cooling is what works best for servers in my experience, and with 3x120mm intake and 2x120mm (rear + psu) exhaust it should be decently balanced.



Ive tested with both bonnie++ and dd if=/dev/zero of=bigfile bs=1M count=4096 etc

There is no way to set storsave mode on the 9500S as far as I can see? This hasnt been an issue on my other fileserver.

Im using both 3DM2 and tw_cli, 3DM2 for managment, tw_cli to check status etc...

Performance numbers:

3x1TB in md raid0, exported from 3ware as single drives:

bonnie++: 37-40MB/s write, 188MB/s read
dd: 40MB/s write, 4GB file

11x1TB, raid5:

bonnie++: 37-40MB/s write, 365MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid5:

bonnie++: 37-40MB/s write, 250MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid0:

dd: 40MB/s write, 4GB file (!!!)



I'll have to test that



I already have a 3.7TB array in my old fileserver, and its full... so I had to expand... I filled it up in a year, nuff said..

And surely I'm not the only one that finds those stupid molex connectors to be the most annoying damn things to unplug? They should come with some kind of oil to make it easier

Ahh, thank you. I'm still annoyed that all SATA power connectors are daisy-chained on my Pro82+.

Just out of curiosity Wibla, what does one save on 11TB? Are you in the imaging business?

Ouch... that's painfully slow, even with the 6x1TB RAID0 setup (which should be the fastest). I'll agree - a call to 3ware is definitely in order. Are you sure that the F1's are on the supported drives list? Are you reusing the same 9500s card as was in your previous build, or is it a new card?

Under Management->Controller settings is where I find the following page:

You don't have anything similar for your card?

Yes, but I do not have the queuing and Storsave options, the controller isnt new, but one i got off fleabay as a spare for the one in my old fileserver.

I have already contacted 3ware and there's an open support ticket.

For now, I suspect the 3ware controller doesnt want to play with the motherboard PCI-X controller on the X38 based mobo, so I've ordered an open box P5WDG2-WS (same as in the old board) and I'm getting an old celeron 3GHz from a friend, I hope that will work better.

An easy way to test that is to put the card in a PCI (non-X) slot. You'll be limited to 133MB/s bandwidth, but you should still be able to write at higher speeds than what you're getting now.

I'm gonna test that this weekend

WOW. I love you're whole set up. I never thought I would need more than my 80Gb Samsung Spinpoint, but after getting into DVD ripping for a streaming media server, I'm almost filling up my first 250Gb. It's nice to see what I'll be doing in 5 years.

Just curious what OS you're using and the inevitable... why might one need so much data? (and the "because-I-can" answer is perfectly OK).