Advice on new Sandy Bridge low power server

[mdchaser]

Hi, it is finally time to retire my old 4850e server as it is just too darn slow. It looks like the new sandy bridge chips are quite good when it comes to idle power (where it will spend 98% of its life) so I think a quad core might be in order. Here is a short blurb on what it will be used for:

Server 2008R2
Exchange 2010
IIS
DNS/DHCP, other misc network software
File/media/backup (a bunch of large drives set to spin down when not in use, most are wd green drives)
Hyper-v with two VMs, one Win7 and one Server 2008r2

The Win7 vm will be running the following:
dynamic dns client
utorrent
calibre book server
jungledisk
secondcopy
dropbox
tversity
microsoft live mesh


Even with the 4850e chip I am currently running the system stays at idle most of the time so I expect the new system to do the same. The current machine idles at right about 55W with all drives but the mirrored boot drives spun down.

Here is a list of the hardware I'm looking at and have already purchased:

Do not have:
CPU: Either an i5-2400S or an i5-2500S, this is where I need some help!
Motherboard: http://www.newegg.com/Product/Product.a ... 6813131689


Already purchased:
Antec mini P180
2X scythe 120x11mm fans (can't remember the model) for the front of the case
Scythe 1200RPM s-flex for rear of the case
Seasonix X-400 fanless power supply
16GB 1333Mhz RAM
160GB Intel SSD for boot drive
2X Samsung spinpoint F4s mirrored for data
4X WD green drives for media storage (should remain spun down most of the time)
Scythe mugen 2 heatsink/fan
WD 2TB usb 3 external drive for system backup

So my conundrum is whether to wait for availability of the 2500S or purchase a 2400S or 2500 right now??? They all seem to idle at the same wattage so peak power/clock are the only two differentiating metrics (even the price is pretty close). A 65W cpu (the S variants) might help to keep the overall power usage down while I'm actively using the system which would be great but I'm not sure when the 2500S will be available, anyone know where I can order one? I'm afraid if I go with a 2400S I will be disappointed in a week or two (hopefully) when the 2500S comes out, money is not an issue as the price should be very close. Any thoughts?

I've also more or less settled on the Asus P8H67-M EVO board as it has 7 SATA ports in total (will loop the esata back into the case) and I have 7 drives in total. Any thoughts on this one would be nice, there are no reviews out there at the moment that I can find.

Thanks!

[HFat]

Why do you want a fast chip for such a server? I say buy the cheapest one which has the feature you want and see if underclocking helps with temps and power consumption. The i5-2400S is massively faster than your old chip as well as the requirements of the software you listed (unless you have many users).
The chip worth waiting for would be the 35W i5 I think. I doubt the actual power savings will be as large as it looks based on the TDP however.

The one thing you may end up regretting is the G2 SSD which is kinda pricey considering the G3 should be around the corner.

The RAM looks overkill if you're going to have 2 VMs. If you start with 8G filling only half your banks you can upgrade later at no extra cost. With a fancy SSD there's less need for cache and the guests don't need to cache as much as standalone systems.

I'd put a larger share of the budget into backups.

I've no idea about which SB mobos are good.
You usually want free ports but you could always move some of your drives out of that server, starting with the data mirror which would be better off mirrored to a different server or simply backed-up properly. eSATA is better than USB3 so you might want to free that port some day.

[mdchaser]

Why do you want a fast chip for such a server? I say buy the cheapest one which has the feature you want and see if underclocking helps with temps and power consumption. The i5-2400S is massively faster than your old chip as well as the requirements of the software you listed (unless you have many users).
The chip worth waiting for would be the 35W i5 I think. I doubt the actual power savings will be as large as it looks based on the TDP however.

While the system will just sit most of the time I use it frequently enough that speed is an issue. That's why I don't mind using a higher clocked quad core, I'm fed up with the whole click and wait game . I considered waiting for the dual cores but I don't know that the power savings will be worth the loss of two cores (5 watts maybe???). I don't know for certain so if you have any data please share!

The one thing you may end up regretting is the G2 SSD which is kinda pricey considering the G3 should be around the corner.

I've had it for a while so I'm stuck with it. If I had to buy today I would definitely choose something newer.

The RAM looks overkill if you're going to have 2 VMs. If you start with 8G filling only half your banks you can upgrade later at no extra cost. With a fancy SSD there's less need for cache and the guests don't need to cache as much as standalone systems.

I picked it up on a great sale on black friday so it was a no brainer. Not only will there be two vms but only one will be running most of the time, yes it will be waaaay more memory than I need .

I'd put a larger share of the budget into backups.

I have an external WD drive to handle the backups as well as using an online service as well as syncing to a machine at work. I'm not too worried about losing data though if I ever fill up the data drive the external drive will only handle a single backup at a time which is not ideal.

I've no idea about which SB mobos are good.
You usually want free ports but you could always move some of your drives out of that server, starting with the data mirror which would be better off mirrored to a different server or simply backed-up properly. eSATA is better than USB3 so you might want to free that port some day.

I actually have a 2 port sata card in case I need to add more drives. My goal is to upgrade the current media drives instead of adding (they are all 1.5TB) more drives. By the time I run out of space the 3TB drives ought to be cheap enough.

Thanks for your input!

[HFat]

I have no data on real-world power consumption.
I just figure a quad-core won't help you much unless you have many users. The software won't require much computing power so my guess is that you'll usually be waiting on the hard drives or the network. But the quad-core i5s are available now while the dual-core isn't, right?
There's a 45W quad if you don't want a dual-core. But the i5-2400S is cheaper and I doubt there'll be 20W of difference between the two.
You're going to have three case fans (you're not planning to use the 200mm, right?) and 6 hard drives anyway so getting the CPU TDP lower than 65W might be pointless. Maybe you could drop the fan that would cool the spun-down low-power drives by the way.

[mdchaser]

I have no data on real-world power consumption.
I just figure a quad-core won't help you much unless you have many users. The software won't require much computing power so my guess is that you'll usually be waiting on the hard drives or the network. But the quad-core i5s are available now while the dual-core isn't, right?
There's a 45W quad if you don't want a dual-core. But the i5-2400S is cheaper and I doubt there'll be 20W of difference between the two.
You're going to have three case fans (you're not planning to use the 200mm, right?) and 6 hard drives anyway so getting the CPU TDP lower than 65W might be pointless. Maybe you could drop the fan that would cool the spun-down low-power drives by the way.

First thing I did was pull the stock case fans (don't care what antec says, they are damn loud!) and covered up the top grill with some sound deadening foam. I'm either going to let the onboard fan controller slow the case fans down or I have a four port fan controller, either way they should all be silent. Removing the fan that cools the spun down drives might be a good idea, I'll play with it once I get everything running. When it is all said and done on a 24hr basis the only things that will be running are the SSD, two drives (samsung spinpoint F4s) and the four fans (or maybe three).

I would agree that a quad is overkill for the software that will be in use. I do tend to create and destroy vm's (or groups of them) for testing on a regular basis and I'm planning on trying to compress some hi def video on this thing (assuming it's more efficient than my core2 quad) so I'd rather be overkill than underkill .

[CA_Steve]

Not quite off-topic....can anyone confirm that the H67 mobo's cannot be undervolted? Because, if they can, then the OP could buy the cheapest i5. Have 2 bios settings where one is stock and used for video encoding and the other is undervolted for server use. <shrugs>

[mdchaser]

Not quite off-topic....can anyone confirm that the H67 mobo's cannot be undervolted? Because, if they can, then the OP could buy the cheapest i5. Have 2 bios settings where one is stock and used for video encoding and the other is undervolted for server use. <shrugs>

As soon as I get things up I'll play with the voltage section and see if it will work. I do remember seeing a review of the bios and it looks like asus does include the proper controls so I do have high hopes. I'm also going to try their epu hardware/software power saving system.

[CA_Steve]

As soon as I get things up I'll play with the voltage section and see if it will work. I do remember seeing a review of the bios and it looks like asus does include the proper controls so I do have high hopes. I'm also going to try their epu hardware/software power saving system.

Cool beans. When you test the epu hw/sw stuff be sure to run a cpu intensive benchmark like Super Pi to see if ASUS mucks with CPU performance in order to reduce power (that's what Gigabyte did with DES on their P45 platform).

[mdchaser]

As soon as I get things up I'll play with the voltage section and see if it will work. I do remember seeing a review of the bios and it looks like asus does include the proper controls so I do have high hopes. I'm also going to try their epu hardware/software power saving system.

Cool beans. When you test the epu hw/sw stuff be sure to run a cpu intensive benchmark like Super Pi to see if ASUS mucks with CPU performance in order to reduce power (that's what Gigabyte did with DES on their P45 platform).

Excellent idea, I've heard the software based epu-6 from past boards could cause some issues but I think this is a new version that works in hardware. I'll benchmark before/after flipping the switch to see if things slow down. It will also be interesting to see if i can drop the voltage while using the epu feature or if that introduces instability.

Here are some reviews with power figures for the i5-2500/h67 combo. Looks like all of these chips run at 1.6Ghz at idle so power draw should be identical until you push the system:

http://www.techspot.com/review/353-inte ... age15.html
http://www.legionhardware.com/articles_ ... ge,14.html
http://www.guru3d.com/article/core-i5-2 ... k-review/7
http://www.semiaccurate.com/2011/01/02/ ... 0k-review/

[twitch]

The above is the reason why I went with the i5-2500k for my htpc+nas. The idle power draw would be the same where the system would spend the majority of the time. Microcenter was running a great deal on the 2500k ($179) which is even cheaper than the low power i5's. It is a waste of the K (unlocked) as there is no need (nor can you overclock the cpu ratio multiplier). Plus you lose the vt-d (for directed i/o) though you still have vt-x. However you gain hd3000 vs hd2000 which makes some mild difference in quicksync if that is important to you. I went with msi h67 ED55 board which some reviews seem to indicate will have the lowest power draw, though this board isn't as cheap as it should be.

[mdchaser]

The above is the reason why I went with the i5-2500k for my htpc+nas. The idle power draw would be the same where the system would spend the majority of the time. Microcenter was running a great deal on the 2500k ($179) which is even cheaper than the low power i5's. It is a waste of the K (unlocked) as there is no need (nor can you overclock the cpu ratio multiplier). Plus you lose the vt-d (for directed i/o) though you still have vt-x. However you gain hd3000 vs hd2000 which makes some mild difference in quicksync if that is important to you. I went with msi h67 ED55 board which some reviews seem to indicate will have the lowest power draw, though this board isn't as cheap as it should be.

I agree, all of the reviews show the same power draw at idle no matter which one you buy. I ended up going with the i5-2400 that was on sale from newegg yesterday. It should be here tomorrow so I hope to be able to at least do some basic power measurements if not get the whole thing up and running. It looks like ASUS's epu function won't affect idle power but does make a difference under load so I'll run some benchmarks and see if epu on/off changes the responsiveness. I have the ASUS board they reviewed here on SPCR, I figure after I add a SATA controller to the intel (or any other board with only 6 sata ports) the idle power would be pretty darn close.

[twitch]

I agree, all of the reviews show the same power draw at idle no matter which one you buy. I ended up going with the i5-2400 that was on sale from newegg yesterday. It should be here tomorrow so I hope to be able to at least do some basic power measurements if not get the whole thing up and running. It looks like ASUS's epu function won't affect idle power but does make a difference under load so I'll run some benchmarks and see if epu on/off changes the responsiveness. I have the ASUS board they reviewed here on SPCR, I figure after I add a SATA controller to the intel (or any other board with only 6 sata ports) the idle power would be pretty darn close.

Which sata controller (HBA?) were you thinking of going with? I am looking for a good deal on a host bus adapter (for jbod, don't need raid) that will support 6-8 sataII hdd.

[mdchaser]

I agree, all of the reviews show the same power draw at idle no matter which one you buy. I ended up going with the i5-2400 that was on sale from newegg yesterday. It should be here tomorrow so I hope to be able to at least do some basic power measurements if not get the whole thing up and running. It looks like ASUS's epu function won't affect idle power but does make a difference under load so I'll run some benchmarks and see if epu on/off changes the responsiveness. I have the ASUS board they reviewed here on SPCR, I figure after I add a SATA controller to the intel (or any other board with only 6 sata ports) the idle power would be pretty darn close.

Which sata controller (HBA?) were you thinking of going with? I am looking for a good deal on a host bus adapter (for jbod, don't need raid) that will support 6-8 sataII hdd.

I purchased one of these on sale a while back:
http://www.newegg.com/Product/Product.a ... 6816150028

In order to get the power down I've decided to skip any sort of raid set up for individual drives that can be spun down when not needed. I happen to have 7 drives so I won't need to use the controller (evo board has 7 if you count the esata port) which is a good thing. If you need a controller that can handle 8 drives all on its own the price is going to be quite high. If you just need to connect a bunch of drives it might be worth buying a few cheap cards with just two ports each (I think I paid $20 on sale). If you can afford it I'm a big fan of HP's controllers, you can generally find them refurbished pretty cheap and can get them with a battery backed cache that helps speed things up.

[mdchaser]

System is up and preliminary numbers are in on the power consumption. I've done all testing with EPU set to auto mode so keep that in mind. The power numbers are from a kill-a-watt meter so I don't know how accurate they are.

At idle with just the SSD:
33-34W

At idle with the SSD and the two drive array:
40W

At idle with all seven drives spun up:
~60-65W

At load (prime95):
114-117W


Some notes:

One of the smoothest hardware installs I've done, not a single issue even though it's v1 hardware and BIOS, this is a great board!

The i5-2400 was a good choice, it's much quicker than I thought it would be and seems to top out about 3.2?Ghz under full load with turbo enabled. If I were to do it again I would go with an i5-2500 simply because it's faster and supports all of the virtualization tech (which btw you need 2008R2 SP1 to use hyper-v, good thing it was leaked the day before I needed it).

I have all power saving features turned on and it doesn't act 'funky' or hiccup, just runs like I would expect it to.

You can use the ASUS fan software to tweak the 4pin fan headers and it works very well. The 3pin header is fixed so I put a fan controller inline to slow things down. The CPU fan ticks along at about 350-400rpm using the silent profile.

Scythe mugen 2 cooler is probably overkill but it was cheap so I would buy it again. The cpu idles at less than 24 degrees, the asus software actually throws an error because it seems to misread it below a certain temp! I've actually had it display -8. I can't remember load values but I think they were in the 30's or low 40's, nothing to worry about. The x-400 fanless powersupply also never warms up.

The Marvell controller works well and allows the drive to spin down.

The ASmedia USB3 chip (I think it's an asus original) is "better" than the NEC chips I've used in the past. The install went smoother and bandwidth seems higher.

You can't use hibernate with hyper-v installed and there is no built in method to trigger a script... if anyone knows how to detect batter power so I can tell the thing to shut down let me know!



Thats about it, I'm very impressed with the board and the processor, believe it or not it makes my i7-920 (oc'd) feel clunky, the whole thing just runs smoother.

[HFat]

If I were to do it again I would go with an i5-2500 simply because it's faster and supports all of the virtualization tech

Intel's ARK says it's got the same features as the i5-2400.

[mdchaser]

If I were to do it again I would go with an i5-2500 simply because it's faster and supports all of the virtualization tech

Intel's ARK says it's got the same features as the i5-2400.

Yah, and it's faster . Just did some video transcoding tests in a virtual machine limited to 95% cpu. It encoded about 25% faster than my core2 quad @ 3ghz... I'm going to do some power tests later to see if its more efficient but I think it will be.

[CA_Steve]

nice. thanks for the update!

[mdchaser]

Quick update, all four of my sata ii ports just failed... I've worked around it until the new boards come out but I have a feeling intel was blowing smoke when they said 5-15% affected after 3 years.

[MikeC]

System is up and preliminary numbers are in on the power consumption. I've done all testing with EPU set to auto mode so keep that in mind. The power numbers are from a kill-a-watt meter so I don't know how accurate they are.

Accuracy can vary. I have one that reads 8~10W low through most of the power scale (from 20W to 300W) -- in comparison to a $600 AC power meter used as a reference.

Here's one simple way to calibrate it:

1) Collect a bunch of incandescent light bulbs, ideally 20W to maybe 150W.
2) Measure AC power via a standard table/desk light fixture w/ standard AC cord/plug.
3) Record all your data.
4) Double check results that don't match the bulb rated power -- ie, if your 60W bulb reads 55W, try other 60W bulbs.
5) You can use anything that has a known, well-defined, accurate AC power rating.
6) Keep an eye on AC voltage, which can vary w/ time of day... or even load on that particular AC socket/line. If it's high or low, the power reading will change.

After all the above, you'll have a pretty good idea of accuracy... & compensation factors for different power levels -- ie, your meter might read 3W low at 60W, dead on at 120W but 5W high at 200W.

btw, yours is the first report of a really malfunctioning SATA port on P/H67 boards I've heard about. Not that I have been looking for it.

[ces]

I considered waiting for the dual cores but I don't know that the power savings will be worth the loss of two cores (5 watts maybe???)

The 2400 idles at about 4 watts:
http://www.xbitlabs.com/articles/cpu/di ... html#sect0

There isn't a lot of watts left to save with the 2 core chips.

[goatsandmonkeys]

I considered waiting for the dual cores but I don't know that the power savings will be worth the loss of two cores (5 watts maybe???)

The 2400 idles at about 4 watts:
http://www.xbitlabs.com/articles/cpu/di ... html#sect0

There isn't a lot of watts left to save with the 2 core chips.

And you can wait a lot longer before upgrading. the 2 core chips come out next week so you can wait a bit to see the benchmarks.