It is currently Mon Jul 28, 2014 7:15 am

All times are UTC - 8 hours




Post new topic Reply to topic  [ 12 posts ] 
Author Message
 Post subject: my not-so-loud file server
PostPosted: Sat Sep 29, 2007 3:36 am 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
split from http://www.silentpcreview.com/forums/vi ... hp?t=40962

speaking about file servers, here's mine (CM GeminiII came later with one 1200rpm Noctua on it):

Image


it's C2D E6550 with 8GB of RAM. 8 x WD7500AAKS, some partitions in linux kernel raid. here's the example of bonnie++ on raid5 (have no raid0 to show off):



Code:
Version 1.93c       ------Sequential Output------ --Sequential Input- --Random-
Concurrency   1     -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks--
Machine        Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP  /sec %CP
titan-raid5     16G  1001  94 176941  27 100796  16  1862  98 330395  27 312.1   5
Latency             10488us     474ms     205ms   26093us   74405us     173ms
Version 1.93c       ------Sequential Create------ --------Random Create--------
titan-raid5         -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete--
              files  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP  /sec %CP
                 64   981   3 +++++ +++   967   1   943   3 +++++ +++   875   3
Latency               495ms      82us     246ms     437ms      61us     492ms


xfs used for massive caching. small files performance would be much better with reiserfs...

_________________
gentoo user


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 3:42 am 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
@matt_garman

Code:
194 Temperature_Celsius     0x0022   114   085   000    Old_age   Always       -       38
194 Temperature_Celsius     0x0022   112   071   000    Old_age   Always       -       40
194 Temperature_Celsius     0x0022   114   086   000    Old_age   Always       -       38
194 Temperature_Celsius     0x0022   113   085   000    Old_age   Always       -       39
194 Temperature_Celsius     0x0022   111   071   000    Old_age   Always       -       41
194 Temperature_Celsius     0x0022   113   086   000    Old_age   Always       -       39
194 Temperature_Celsius     0x0022   116   102   000    Old_age   Always       -       36
194 Temperature_Celsius     0x0022   115   101   000    Old_age   Always       -       37


with room temp ~25 deg C.


on the side panel in front of lower 6 hdds, I have two 92mm fans at ~1500rpm.
in the front of hdd cage in 5.25" space, I have one Noctua 120mm fan ~600rpm.

GeminiII has one Nocuta 120mm that kicks in only when CPU goes above 40 deg C. that is maybe once or twice a day. I'll post photos with GeminiII as soon as I open that box again.

_________________
gentoo user


Top
 Profile  
 
 Post subject:
PostPosted: Sat Sep 29, 2007 7:46 am 
Offline
*Lifetime Patron*

Joined: Sun Jan 04, 2004 11:35 am
Posts: 534
Location: Chicago, Ill., USA
Awesome, thanks!

_________________
my computers (updated December, 2007)


Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 10:40 am 
Offline

Joined: Tue Jul 24, 2007 3:57 pm
Posts: 312
Location: Minnesota
Thats nuts, haha.

The drive temperatures are surprisingly low. Two thumbs up.

_________________
Chris Randel
Network Administrator, Quiet Enthusiast.
Corsair 600t
Lian-Li Q08 FS
Lian-Li Q07 Pics
New - P183 Build


Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 11:57 am 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
protellect wrote:
Thats nuts, haha.

The drive temperatures are surprisingly low. Two thumbs up.


thx.

anyway, I think we should all congratulate WD for making such a Cool'N'Quiet drive. ;)
but, it's very fast, too. :D

_________________
gentoo user


Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 3:22 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
Not really that low temps, my setup runs between 28 and 35 degrees with nexus/noctua fans at below 800rpm :)

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Last edited by Wibla on Sun Sep 30, 2007 4:42 pm, edited 1 time in total.

Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 3:33 pm 
Offline

Joined: Tue Jul 24, 2007 3:57 pm
Posts: 312
Location: Minnesota
I'm used to seagate temperatures :) 35-45C, with fans on it.

_________________
Chris Randel
Network Administrator, Quiet Enthusiast.
Corsair 600t
Lian-Li Q08 FS
Lian-Li Q07 Pics
New - P183 Build


Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 3:50 pm 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
Wibla wrote:
Not really that low temps, my setup idles between 28 and 35 degrees with nexus/noctua fans at below 800rpm :)


there's no idle state for my WDs:

Code:
 01:49:57 up 16 days,  6:35,  3 users,  load average: 0.10, 0.26, 0.36

eth0      Link encap:Ethernet  HWaddr
          inet addr:
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:980847708 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1473649187 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:555029955747 (529317.8 Mb)  TX bytes:1725676289112 (1645733.1 Mb)
          Interrupt:16 Base address:0x6000

eth1      Link encap:Ethernet  HWaddr
          inet addr:
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:990827237 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1345575160 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000
          RX bytes:503080798431 (479775.2 Mb)  TX bytes:1755732249962 (1674396.7 Mb)
          Interrupt:20 Base address:0x8000



;)

_________________
gentoo user


Top
 Profile  
 
 Post subject:
PostPosted: Sun Sep 30, 2007 4:37 pm 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
Image

There is a raid verify in there somewhere, try to spot it (not during the last 6 hours) :)

This is with ambient temp around 23C...

Samsung drives dont run as hot tho, and with hardware raid there is always some activity on the arrays, even tho its not streaming hdtv or doing verifies or whatnot.

Image
o.O traffic! :shock:

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 21, 2010 7:10 am 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
I've done a small upgrade:

- mobo is Asus P5Q Premium
- 2 x Digitus PCI-E x4 SATA controllers
- Chieftec BA
- 16 x Samsung F2 1.5TB

Image

Image

Image

Image

_________________
gentoo user


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 21, 2010 9:47 am 
Offline
Friend of SPCR

Joined: Sun Jun 03, 2007 12:03 am
Posts: 777
Location: Norway
I really like your solution for the sata cables :)

What kind of fans do you have infront of the bottom drives? and are those P12 fans on the 4in3's?

I've heard mixed reviews on how the F2 1.5TB's work in raid, ranging from "dont use them, they cant handle it" from backblaze, to others having no problems at all, whats been your experience so far?
Edit: read this: http://forums.storagereview.com/index.p ... tler-cctl/

Also, I hope youre gonna use RAID6 ? :)

_________________
Workstation | HTPC | 9.1TB | 19.1TB


Top
 Profile  
 
 Post subject:
PostPosted: Sun Mar 21, 2010 11:06 am 
Offline

Joined: Wed Dec 15, 2004 5:45 pm
Posts: 145
Location: system
I'm using couple of arrays made by linux software raid1 and raid5. raid arrays are non-uniform and they cover about 60% of disk space. everything else is not protected by redundancy.

Code:
cat /proc/mdstat
Personalities : [linear] [raid1] [raid6] [raid5] [raid4]
md4 : active raid5 sdf1[0] sdo2[9] sdn2[8] sdm2[7] sdl2[6] sdk2[5] sdj2[4] sdi1[3] sdh1[2] sdg1[1]
      443658240 blocks super 1.1 level 5, 64k chunk, algorithm 2 [10/10] [UUUUUUUUUU]
      bitmap: 0/1 pages [0KB], 65536KB chunk

md2 : active raid5 sda3[0] sdi3[8] sdh3[7] sdg3[6] sdf3[5] sde2[4] sdd2[3] sdc2[2] sdb3[1]
      11301533184 blocks super 1.1 level 5, 64k chunk, algorithm 2 [9/9] [UUUUUUUUU]
      bitmap: 0/11 pages [0KB], 65536KB chunk

md3 : active raid5 sdj1[0] sdp1[6] sdo1[5] sdn1[4] sdm1[3] sdl1[2] sdk1[1]
      1988476416 blocks super 1.1 level 5, 64k chunk, algorithm 2 [7/7] [UUUUUUU]
      bitmap: 0/3 pages [0KB], 65536KB chunk

md1 : active raid1 sdb2[1] sda2[0]
      52339648 blocks [2/2] [UU]
      bitmap: 5/200 pages [20KB], 128KB chunk

md0 : active raid1 sdb1[1] sda1[0]
      96256 blocks [2/2] [UU]
      bitmap: 0/12 pages [0KB], 4KB chunk

unused devices: <none>


I'm doing complete check on all arrays every month
Code:
#!/bin/bash

echo check >> /sys/block/md0/md/sync_action
echo check >> /sys/block/md1/md/sync_action
echo check >> /sys/block/md2/md/sync_action
echo check >> /sys/block/md3/md/sync_action
echo check >> /sys/block/md4/md/sync_action

If any unreadable sector is found, raid will try to resync it resulting with sector reallocation. so far so good.

fans used inside the case are not ideal - they are S12 and they lack pressure needed at least on CPU heatsink. I already had them. but, considering low CPU load and heat dissipation, they are doing good job. core temps top at ~58C when both cores are loaded with prime95, and core temps cannot reach 50C during normal operation (pbzip, direct connect hashing, compiling...).

fans on cages are good - they are P12. :)

fans in front of lower drives are NF-B9, three of them.

about Samsung F2...
well, it's not blazing fast. nor it's fast at all.
but, it's quiet, cool and cheap, so it feets my needs: massive storage without too much need for random access and some fancy seeks.
sequential read is good - it reaches 1.2TB/s during raid check. it's not too fast considering every of 16 drives gives 75MB/s, but it's not bad eighter.
to achive that I had to put additional controllers in PCI-E slots connected to north bridge.
my monthly average disk read including complete check of all arrays is 9.5MB/s (sum for all disks).
average write is ~700KB/s.

average temperatures for Samsung are 28 to 30C.

btw, Samsung F2 is known to have terrible multitasking algorytm. I've tried to lower impact of that by making sure two most busy arrays don't share single phisical disk (md2 and md3) and by increasing use of write buffers (commit=29, commit=30, commit=31... as mount options and vm.dirty_* tunning).

_________________
gentoo user


Top
 Profile  
 
Display posts from previous:  Sort by  
Post new topic Reply to topic  [ 12 posts ] 

All times are UTC - 8 hours


Who is online

Users browsing this forum: Bing [Bot] and 3 guests


You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot post attachments in this forum

Search for:
Jump to:  
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group