It is currently Fri Aug 01, 2014 10:34 pm

All times are UTC - 8 hours




Post new topic Reply to topic  [ 76 posts ]  Go to page 1, 2, 3  Next
Author Message
 Post subject: The Monster! (last updated: 25.02.10 - power usage numbers)
PostPosted: Sat May 24, 2008 2:17 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
This is my new fileserver, named "Monster" because its the biggest and most expensive build of mine yet...

Specs:

Coolermaster Stacker Classic + 4in3 modules
Asus P5WDG2-WS
Intel 3GHz Celeron
Intel PCI-E Gig-E network
Xigmatek cpu cooler (coming soon)
4GB PC2-6400 Corsair ram
3ware 9500S-12 PCI-X RAID controller
12x1TB Samsung F1 (11x1TB raid5 + hotspare)
Seagate 80GB SATA 2.5" 7200rpm system drive
Nexus 120mm fans for the 4in3 modules
Corsair HX520 PSU

Pics: (the important bit)

Image

Image

Image

Image

Image

Image

Image

Image

Image

Image

Noise:

Honestly, the noise level IS NOT bad, im impressed so far, there was some vibration...

I swapped out the Coolermaster fan with an Antec Tricool in the back, removed the restrictive fan grill, and put in Nexus 120mm on the 4in3 modules... no fan controller as of yet, except the onboard asus stuff.

Image

Performance numbers will be posted when the box is up and running, still initializing the raid array now.

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Last edited by Wibla on Sat Feb 27, 2010 9:02 am, edited 4 times in total.

Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 2:27 pm 
Offline

Joined: Thu Mar 22, 2007 10:28 am
Posts: 313
12TB? Are you trying to download the internet?

Good luck. :D

Looks good btw!


Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 3:08 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
Not quite..

Code:
/dev/sdb              9.1T  1.1M  9.1T   1% /mnt/storage


11 drives in raid5 + hotspare, so...

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 3:15 pm 
Offline

Joined: Mon Feb 05, 2007 2:10 pm
Posts: 1803
Location: Northern New Jersey
If you apply "P.L.O.H"* rule, then that's alot of freekin porn.

* Porno Law of Hardrive = .5 X total harddrive space = amount of porn files.


just a Q6600?

clever hiding that 2.5" all the way up top

i'm a little surprised myself that one PSU handles 13 drives + quad.

_________________
|Dual Intel Xeon E5-2620-Xigmatek SD1283 DK-II|Asus Z9PA-D8-HR-05 IFX|WD 250gb Velociraptor SATA3|16gb DDR3-1600 ECC RDIMM|WD Blue 640GB SATA2 x2|Logitech K750|Logitech Wireless Mouse|nVidia GTX660ti 2GB|Antec HCG-750|NZXT Source 210 Elite|M-Audio ProFire 2626|Art TubeOpto8 with Smooth Plate Tube Swap|Avid Artist Mix x2|
FartingBob wrote:
A 9500GT with 1GB of RAM is the most pointless thing since NASCAR.


Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 4:16 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
I dont really need the Quad core at all, and for some reason i cant run 64bit linux and get network connectivity with the onboard gigabit, so im gonna put the Q6600 in my ws, and put an E2140 in the fileserver, with HWraid its not an issue anyway :-)

The only reason I got the Q6600 was to run folding@home on it aswell, but with no 64bit I cant run SMP, so no point in having it in there...
I was abit sceptical about the 2.5" at first, but it works fine, so no worries then...

2.5" drive:
Code:
# hdparm -tT /dev/sda
/dev/sda:
 Timing cached reads:   8246 MB in  2.00 seconds = 4129.83 MB/sec
 Timing buffered disk reads:  146 MB in  3.00 seconds =  48.60 MB/sec

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 7:01 pm 
Offline

Joined: Tue Oct 23, 2007 7:52 pm
Posts: 75
Location: The Most Mexican Place in the U.S.A
:shock: .....That's......impressive......(should I say overkill?) :P .......


bonestonne wrote:
If you apply "P.L.O.H"* rule, then that's alot of freekin porn.

* Porno Law of Hardrive = .5 X total harddrive space = amount of porn files.




(resized)

If you apply "P.L.O.H"* rule, then that's alot of freekin porn.

* Porno Law of Hardrive = .5 X total harddrive space = amount of porn files.



.....
.......
......... :? ....:| ............ :mrgreen:

_________________
The best Integrated heat spreader removal guide/database...period


Top
 Profile  
 
 Post subject:
PostPosted: Sat May 24, 2008 9:50 pm 
Offline

Joined: Wed Apr 16, 2008 3:32 am
Posts: 486
Location: Poland
bonestonne wrote:
i'm a little surprised myself that one PSU handles 13 drives + quad.

Drives pull very little power, so that's about the equivalent of the Quad plus just one high-end graphics card. Should be no problem for a high-efficiency unit. :)

_________________
Can you keep it down? I'm having trouble hearing the artillery.

Current rig
Old PC


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 12:00 am 
Offline

Joined: Sun Mar 23, 2008 9:29 am
Posts: 38
Location: Molde, Norway
Nice build there! But I'm happy to buy you some zip-ties for those SATA-cables.


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 12:44 am 
Offline
SPCR Reviewer

Joined: Tue May 30, 2006 8:22 pm
Posts: 561
Location: Vancouver, BC
Looking at your pictures, I'm guessing you have the same problem I've been having in my system - SATA power cables. I've also resorted to using molex->SATA adaptors for a few of my drives, but I really wish that there were a few 4-port SATA power cables for the Corsair PSUs.

Looks like a nice setup. :)

_________________
Zxian.org || Kwikpiks.com
Workstation || Server


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 1:23 am 
Offline

Joined: Mon Jan 29, 2007 2:12 am
Posts: 146
Location: Poland
Where did you put the 2.5 drive? Can't find it on the pics..


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 2:28 am 
Offline

Joined: Fri Mar 14, 2008 5:18 am
Posts: 484
Location: London, UK
Nick Geraedts wrote:
Looking at your pictures, I'm guessing you have the same problem I've been having in my system - SATA power cables.
I'm completely puzzled as to why the design has been changed from Molex at all. Probably just to make everybody's life a little more complicated.


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 3:05 am 
Offline

Joined: Thu May 25, 2006 1:19 am
Posts: 489
Location: Australia
sata power cables have one extra wire- surely it must have some sort of use :evil:
i think i remember something about hot-swapping sata hdds?


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 6:38 am 
Offline

Joined: Mon Feb 05, 2007 2:10 pm
Posts: 1803
Location: Northern New Jersey
m^2 wrote:
Where did you put the 2.5 drive? Can't find it on the pics..


it took me a minute too. its all the way up top. look for a black SATA cable coming off the motherboard, and you'll see it snake all the way up, and then over to the whereabouts of the optical drive.

_________________
|Dual Intel Xeon E5-2620-Xigmatek SD1283 DK-II|Asus Z9PA-D8-HR-05 IFX|WD 250gb Velociraptor SATA3|16gb DDR3-1600 ECC RDIMM|WD Blue 640GB SATA2 x2|Logitech K750|Logitech Wireless Mouse|nVidia GTX660ti 2GB|Antec HCG-750|NZXT Source 210 Elite|M-Audio ProFire 2626|Art TubeOpto8 with Smooth Plate Tube Swap|Avid Artist Mix x2|
FartingBob wrote:
A 9500GT with 1GB of RAM is the most pointless thing since NASCAR.


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 7:26 am 
Offline

Joined: Mon Feb 26, 2007 9:20 am
Posts: 3141
Location: Missing in Finnish wilderness, howling to moon with wolf brethren and walking with brother bears
Is this the SPCR version find the Waldo? Instead: Find the 2,5" HDD?

Anyway, case is awesome looking and that is indeed a monster. 11 TB's sweet! I could use such space ^^. That could take my anime collection like 2,5 times ^^

_________________
If seeing is believing, how can blind person believe in anything?
Maturity is just not experience in life but also ability to make compromises.


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 11:47 am 
Offline
SPCR Reviewer

Joined: Tue May 30, 2006 8:22 pm
Posts: 561
Location: Vancouver, BC
Cistron wrote:
I'm completely puzzled as to why the design has been changed from Molex at all. Probably just to make everybody's life a little more complicated.


The SATA power connector was actually a much needed improvement over molex, adding SATA-only capabilities (such as hot swapping), as well as improving the electrical security of the setup. Have a look here for more information.

_________________
Zxian.org || Kwikpiks.com
Workstation || Server


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 3:49 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
I'm puzzled as to the shitty performance im getting with the 9500S on 12x1TB... 40MB/s tops write speed on 11 or 6 drives in raid5, and top read speed of about 365-370MB/s... the other box with 9x500GB will do 100+MB/s write easily.. :(

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 4:36 pm 
Offline

Joined: Wed Apr 28, 2004 5:16 am
Posts: 777
Location: canberra, australia
PLOH.. lol..

i hate hard disk noise... 12HD must sound horrible ... :(


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 5:30 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
Thats why i chose the F1 drives, performance and pretty low noise... that box is very quiet for what it is... a big hunking monster... :D

(too bad write performance is lacking for now.. I might have to mail both samsung and AMCC/3ware about this, as its not supposed to be like this at all!)

Some lessons learned from Oberon to this box:

- Know what port is the first one, and how they're numbered...
in Oberon, drive 1 and port0 != the same, drive1 at the top = port1, drive2 = port0, etc.. big confusion, and a big headache when something breaks...
in monster, drive0 and port0 = same, and the order is kept for all the drives, cables are numbered accordingly, with drive0 at the top and drive11 at the bottom...

- Think through how the cabling is supposed to be laid, especially the power cables... Spent atleast an hour doing only the power cables, but I got some good dividends...

As a result of a few precautions and some experience, monster is alot neater built than oberon, even tho im running 13 drives and a DVDRW in monster, compared to 11 drives and no DVDRW in oberon.

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 8:33 pm 
Offline

Joined: Thu May 25, 2006 1:19 am
Posts: 489
Location: Australia
have you tried running the system without the exhaust and what appears to be a top exhaust?
considering you have 3 intakes it should work fairly well- the less fans to control the better, right?


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 9:29 pm 
Offline
SPCR Reviewer

Joined: Tue May 30, 2006 8:22 pm
Posts: 561
Location: Vancouver, BC
@wibla - Are you sure you've set the controller to StorSave-Performance mode? I found that I get slow write speeds when it's on Protection or Balance.

Do you use the 3DM2 to administer your settings or the command line utility?

What are you using to test the write performance? I've always just tried writing dummy files from /dev/zero using dd. You can easily test batch performance for small files or large files with that kind of setup. :)

_________________
Zxian.org || Kwikpiks.com
Workstation || Server


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 9:50 pm 
Offline

Joined: Thu Jul 26, 2007 2:16 pm
Posts: 16
Location: Gainesville, FL - USA
Beware - don't set that box on a slippery surface or the torque from all those HD's will make the entire chassis spin in the opposite direction the HD's rotate!! :D :roll:

_________________
Building homebrew computers since 1978 8)


Top
 Profile  
 
 Post subject:
PostPosted: Sun May 25, 2008 11:50 pm 
Offline

Joined: Wed Apr 02, 2008 12:20 am
Posts: 148
Location: Sweden
WOW!... thats huge fucking build. I dont really understand what u will use 12TB of space for thou. I have 1 750gb drive and i dont think i will be able to fill it up in quiet some time.


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 12:05 am 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
porkchop wrote:
have you tried running the system without the exhaust and what appears to be a top exhaust?
considering you have 3 intakes it should work fairly well- the less fans to control the better, right?


I'm a big fan of the KIS(S) principle for cooling, front to back cooling is what works best for servers in my experience, and with 3x120mm intake and 2x120mm (rear + psu) exhaust it should be decently balanced.

Nick Geraedts wrote:
@wibla - Are you sure you've set the controller to StorSave-Performance mode? I found that I get slow write speeds when it's on Protection or Balance.

Do you use the 3DM2 to administer your settings or the command line utility?

What are you using to test the write performance? I've always just tried writing dummy files from /dev/zero using dd. You can easily test batch performance for small files or large files with that kind of setup. :)


Ive tested with both bonnie++ and dd if=/dev/zero of=bigfile bs=1M count=4096 etc

There is no way to set storsave mode on the 9500S as far as I can see? This hasnt been an issue on my other fileserver.

Im using both 3DM2 and tw_cli, 3DM2 for managment, tw_cli to check status etc...

Performance numbers:

3x1TB in md raid0, exported from 3ware as single drives:

bonnie++: 37-40MB/s write, 188MB/s read
dd: 40MB/s write, 4GB file

11x1TB, raid5:

bonnie++: 37-40MB/s write, 365MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid5:

bonnie++: 37-40MB/s write, 250MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid0:

dd: 40MB/s write, 4GB file (!!!)

Gator wrote:
Beware - don't set that box on a slippery surface or the torque from all those HD's will make the entire chassis spin in the opposite direction the HD's rotate!! :D :roll:


I'll have to test that :lol:

AckeDman wrote:
WOW!... thats huge fucking build. I dont really understand what u will use 12TB of space for thou. I have 1 750gb drive and i dont think i will be able to fill it up in quiet some time.


I already have a 3.7TB array in my old fileserver, and its full... so I had to expand... I filled it up in a year, nuff said..

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 2:49 am 
Offline

Joined: Sat May 17, 2008 9:13 am
Posts: 11
Location: UK
Quote:
I'm completely puzzled as to why the design has been changed from Molex at all. Probably just to make everybody's life a little more complicated.

Quote:
The SATA power connector was actually a much needed improvement over molex, adding SATA-only capabilities (such as hot swapping), as well as improving the electrical security of the setup. Have a look here for more information.


And surely I'm not the only one that finds those stupid molex connectors to be the most annoying damn things to unplug? They should come with some kind of oil to make it easier :)


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 6:24 am 
Offline

Joined: Fri Mar 14, 2008 5:18 am
Posts: 484
Location: London, UK
Nick Geraedts wrote:
Cistron wrote:
I'm completely puzzled as to why the design has been changed from Molex at all. Probably just to make everybody's life a little more complicated.


The SATA power connector was actually a much needed improvement over molex, adding SATA-only capabilities (such as hot swapping), as well as improving the electrical security of the setup. Have a look here for more information.
Ahh, thank you. I'm still annoyed that all SATA power connectors are daisy-chained on my Pro82+.

Just out of curiosity Wibla, what does one save on 11TB? Are you in the imaging business?


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 8:43 am 
Offline
SPCR Reviewer

Joined: Tue May 30, 2006 8:22 pm
Posts: 561
Location: Vancouver, BC
Wibla wrote:
Performance numbers:

3x1TB in md raid0, exported from 3ware as single drives:

bonnie++: 37-40MB/s write, 188MB/s read
dd: 40MB/s write, 4GB file

11x1TB, raid5:

bonnie++: 37-40MB/s write, 365MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid5:

bonnie++: 37-40MB/s write, 250MB/s read
dd: 40MB/s write, 4GB file

6x1TB, raid0:

dd: 40MB/s write, 4GB file (!!!)


Ouch... that's painfully slow, even with the 6x1TB RAID0 setup (which should be the fastest). I'll agree - a call to 3ware is definitely in order. Are you sure that the F1's are on the supported drives list? Are you reusing the same 9500s card as was in your previous build, or is it a new card?

Under Management->Controller settings is where I find the following page:
Image
You don't have anything similar for your card?

_________________
Zxian.org || Kwikpiks.com
Workstation || Server


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 9:06 am 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
Yes, but I do not have the queuing and Storsave options, the controller isnt new, but one i got off fleabay as a spare for the one in my old fileserver.

I have already contacted 3ware and there's an open support ticket.

For now, I suspect the 3ware controller doesnt want to play with the motherboard PCI-X controller on the X38 based mobo, so I've ordered an open box P5WDG2-WS (same as in the old board) and I'm getting an old celeron 3GHz from a friend, I hope that will work better.

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 11:16 am 
Offline
SPCR Reviewer

Joined: Tue May 30, 2006 8:22 pm
Posts: 561
Location: Vancouver, BC
Wibla wrote:
For now, I suspect the 3ware controller doesnt want to play with the motherboard PCI-X controller on the X38 based mobo, so I've ordered an open box P5WDG2-WS (same as in the old board) and I'm getting an old celeron 3GHz from a friend, I hope that will work better.

An easy way to test that is to put the card in a PCI (non-X) slot. You'll be limited to 133MB/s bandwidth, but you should still be able to write at higher speeds than what you're getting now.

_________________
Zxian.org || Kwikpiks.com
Workstation || Server


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 1:58 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
I'm gonna test that this weekend :)

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Mon May 26, 2008 2:11 pm 
Offline

Joined: Thu Mar 20, 2008 8:51 am
Posts: 213
Location: In the server room, playing Trackmania
WOW. I love you're whole set up. I never thought I would need more than my 80Gb Samsung Spinpoint, but after getting into DVD ripping for a streaming media server, I'm almost filling up my first 250Gb. It's nice to see what I'll be doing in 5 years.

Just curious what OS you're using and the inevitable... why might one need so much data? (and the "because-I-can" answer is perfectly OK).

_________________
thatLinuxguy.wordpress.com


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 76 posts ]  Go to page 1, 2, 3  Next

All times are UTC - 8 hours


Who is online

Users browsing this forum: No registered users and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group