ServeMi, 23TB Server
Posted: Sun Dec 11, 2011 4:26 pm
This is by far the hardest build i ever done, a lot of issues happen before i could finish it, more than 2 months all weekends working on it. Starting with Asus P8B WS mobo with restarting issues, bought ne PSU, new memory and still no fix, so i return it and got a Supermicro mobo, the first day everything was perfect, second day OS hdd started clicking and no boot, swaped the Hitachi 5k500B for intel x25m 120gb that i had left over (wasn't planing on using it but with hdd prices today....i just grabbed what i had in hand), everything was perfect until i started noticing a loud intermittent noise from the fans, seems a lot of Supermicro mobos have issues controlling PWM fans, it drops them down to 0 and then to full 1600rpm, its an endless loop and very noisy and disturbing, i tried Speedfan and no luck... so i swapped all fans to 3pins and finally everything is good.
This is mostly a storage server for my local network, i had such a good experience with WHSv1 and Acer AH-342 that i decided to build my own as the space wasnt enough for all the media storage for the home network. I had an antec 1200 left over form my past gaming pc, so decided to get some hdd cages to fit 16x hitachi 5k3000, but with the flooding i had to settle for a combination samsung 1.5tb that i had left from the previous server. Here is the list of hardware currently in use,
MOBO: SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard
MEMORY: Kingston 4GB (2 x 2GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Model KVR1333D3E9SK2/4G
CPU: Intel Xeon E3-1230 Sandy Bridge 3.2GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1155 80W Quad-Core Server Processor
CPU Cooler: Thermalright AXP-140 RT Low Profile HTPC Heatsink
CPU Fan: Noiseblocker NB-BlackSilentPro PK-1 140mm x 25mm Ultra Quiet Fan - 700 RPM - 9 dBA
Case fans: 7x Noiseblocker NB-Multiframe M12-S1 120mmx25mm Ultra Silent Fan - 750 RPM - below 6 dBA
OS drive: Intel X25-M SSDSA2MH120G2K5 2.5" 120GB SATA II MLC Internal Solid State Drive (SSD)
HDD: 8x HITACHI Deskstar 5K3000 HDS5C3020ALA632 (0F12117) 2TB 32MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive
HDD: 6x SAMSUNG EcoGreen F2 HD154UI 1.5TB 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive
HBA: 2x Intel RAID Controller Card SATA/SAS PCI-E x8 8internal ports (SASUC8I)
Sata cables: 4x 3ware CBL-SFF8087OCF-05M 1 Unit of .5M Multi-lane Internal (SFF-8087) Serial ATA Breakout Cable
Sata power: 4x Silverstone Four-in-one SATA Power Connectors with Power Stabilizing Capacitors Model CP06
PSU: Antec CP-850 850W Continuous Power CPX SLI Certified CrossFire Ready 80 PLUS Certified Modular Active PFC
CASE: Antec Twelve Hundred Black Steel ATX Full Tower Computer Case
HDD Cages: 4x COOLER MASTER STB-3T4-E3-GP 4-in-3 Device Module Hardisk Cage
OS: Windows Home Server v1
Here some pics of the build,
After adding all hdds, here is the storage pool,
This are the idle and load (prime95) temps and power consumptions (with the flash the screen appear white and no good daylight there, in case you cant see was 123W idle and 197W load)
Here is a test of transfering a large file from the server to my desktop, and a test while watching a movie over the network,
Here is HDtune of the Hitachi 5k3000 ran inside the server,
Comments,
I had a lot of fun doing the server, started planning a low powered server, but with the trouble i had with the initial parts and wanting to do transcoding on the fly, and maybe even move to a newer OS with ZFS, so i went with better cpu/mobo than i initially planned, but really been a great experience with supermicro and xeon. The SSD is an overkill for this build, specially once i added the SASUC8I, the server takes a long time to boot, 3min+, the ssd doesn't make much of a difference, but i had it laying around and my hitachi 5k500b died while building, and didn't want to spend more so i just used the x25m that i had, still you do feel it once inside windows, like on my desktop everything loads instantly, but for the server doesn't make much of a difference. Im pretty happy with the transfer rate results, im able to sustain close to 100 MB/s over my home gigabit network (really depends on other factors, some files do not transfer that that rate). With what i experience, the only change i would do is going for a server case, while the antec 1200 works and probably maintains the hdds cooler as each cage has its own fan and directly blow air through... its very hard to work with, specially the Cable management, i really tried to hide as much as i could, but once i was dealing with all the sata cables and the PSU cables, there isn't enough room on the 1200 to router all in the back, maybe with some rerouting.... but still its not ideal.
The hdds are pretty quiet, a little noisy on startup... but hey its 14 none suspended mechanical 3.5 hdds, once it boots to WHS the drives are very quiet, even transfering i can bearly notice them. Im also happy with the fans, although its a little weird, the M12 S1 are suppose to be 750rpm fans, but the mobo reads them as 850rpm, still they are very quiet, and the PK1 is also, while not silent, im very pleased with the temps and noise.
Future,
This was suppose to be an 30TB server, but with flooding and today hdds pricing, was impossible to get new drives, so i had to reuse the samsung from my old server, and didnt have 16 drives, but ended with 14, so once the hdds prices go back to normal, im planning on increasing the storage, but i might just leave it as it is, and move to another OS that supports 2tb+ hdds. Im also looking for software that can transcode on the fly for mobile stuff like ipads, iphones, etc.
If you have any questions or suggestions or comments, pls post them.
This is mostly a storage server for my local network, i had such a good experience with WHSv1 and Acer AH-342 that i decided to build my own as the space wasnt enough for all the media storage for the home network. I had an antec 1200 left over form my past gaming pc, so decided to get some hdd cages to fit 16x hitachi 5k3000, but with the flooding i had to settle for a combination samsung 1.5tb that i had left from the previous server. Here is the list of hardware currently in use,
MOBO: SUPERMICRO MBD-X9SCM-F-O LGA 1155 Intel C204 Micro ATX Intel Xeon E3 Server Motherboard
MEMORY: Kingston 4GB (2 x 2GB) 240-Pin DDR3 SDRAM ECC Unbuffered DDR3 1333 Server Memory Model KVR1333D3E9SK2/4G
CPU: Intel Xeon E3-1230 Sandy Bridge 3.2GHz 4 x 256KB L2 Cache 8MB L3 Cache LGA 1155 80W Quad-Core Server Processor
CPU Cooler: Thermalright AXP-140 RT Low Profile HTPC Heatsink
CPU Fan: Noiseblocker NB-BlackSilentPro PK-1 140mm x 25mm Ultra Quiet Fan - 700 RPM - 9 dBA
Case fans: 7x Noiseblocker NB-Multiframe M12-S1 120mmx25mm Ultra Silent Fan - 750 RPM - below 6 dBA
OS drive: Intel X25-M SSDSA2MH120G2K5 2.5" 120GB SATA II MLC Internal Solid State Drive (SSD)
HDD: 8x HITACHI Deskstar 5K3000 HDS5C3020ALA632 (0F12117) 2TB 32MB Cache SATA 6.0Gb/s 3.5" Internal Hard Drive
HDD: 6x SAMSUNG EcoGreen F2 HD154UI 1.5TB 32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive
HBA: 2x Intel RAID Controller Card SATA/SAS PCI-E x8 8internal ports (SASUC8I)
Sata cables: 4x 3ware CBL-SFF8087OCF-05M 1 Unit of .5M Multi-lane Internal (SFF-8087) Serial ATA Breakout Cable
Sata power: 4x Silverstone Four-in-one SATA Power Connectors with Power Stabilizing Capacitors Model CP06
PSU: Antec CP-850 850W Continuous Power CPX SLI Certified CrossFire Ready 80 PLUS Certified Modular Active PFC
CASE: Antec Twelve Hundred Black Steel ATX Full Tower Computer Case
HDD Cages: 4x COOLER MASTER STB-3T4-E3-GP 4-in-3 Device Module Hardisk Cage
OS: Windows Home Server v1
Here some pics of the build,
After adding all hdds, here is the storage pool,
This are the idle and load (prime95) temps and power consumptions (with the flash the screen appear white and no good daylight there, in case you cant see was 123W idle and 197W load)
Here is a test of transfering a large file from the server to my desktop, and a test while watching a movie over the network,
Here is HDtune of the Hitachi 5k3000 ran inside the server,
Comments,
I had a lot of fun doing the server, started planning a low powered server, but with the trouble i had with the initial parts and wanting to do transcoding on the fly, and maybe even move to a newer OS with ZFS, so i went with better cpu/mobo than i initially planned, but really been a great experience with supermicro and xeon. The SSD is an overkill for this build, specially once i added the SASUC8I, the server takes a long time to boot, 3min+, the ssd doesn't make much of a difference, but i had it laying around and my hitachi 5k500b died while building, and didn't want to spend more so i just used the x25m that i had, still you do feel it once inside windows, like on my desktop everything loads instantly, but for the server doesn't make much of a difference. Im pretty happy with the transfer rate results, im able to sustain close to 100 MB/s over my home gigabit network (really depends on other factors, some files do not transfer that that rate). With what i experience, the only change i would do is going for a server case, while the antec 1200 works and probably maintains the hdds cooler as each cage has its own fan and directly blow air through... its very hard to work with, specially the Cable management, i really tried to hide as much as i could, but once i was dealing with all the sata cables and the PSU cables, there isn't enough room on the 1200 to router all in the back, maybe with some rerouting.... but still its not ideal.
The hdds are pretty quiet, a little noisy on startup... but hey its 14 none suspended mechanical 3.5 hdds, once it boots to WHS the drives are very quiet, even transfering i can bearly notice them. Im also happy with the fans, although its a little weird, the M12 S1 are suppose to be 750rpm fans, but the mobo reads them as 850rpm, still they are very quiet, and the PK1 is also, while not silent, im very pleased with the temps and noise.
Future,
This was suppose to be an 30TB server, but with flooding and today hdds pricing, was impossible to get new drives, so i had to reuse the samsung from my old server, and didnt have 16 drives, but ended with 14, so once the hdds prices go back to normal, im planning on increasing the storage, but i might just leave it as it is, and move to another OS that supports 2tb+ hdds. Im also looking for software that can transcode on the fly for mobile stuff like ipads, iphones, etc.
If you have any questions or suggestions or comments, pls post them.