http://www.physorg.com/news105800312.html



air cooling for such massive conglomerations of computers is clearly very inefficient. you would think some kind of water or freon-based system would be better.

Well the air conditioning is freon based!

As I see it, one big reason for air cooling is reliability. If a fan breaks in one of the servers, no problem. The fans are quite often placed two in series for redundance. An alarm is triggered and someone can hotswap the fan without downtime even.

Say a hose breaks, be it water or other cooling medium, you are in trouble. The machine might have time enough to shut down before any damage, but you still have downtime. Changing the hose, refilling system, cleaning up if any coolant gets on the floor, etc etc.

Also the cool climate in the server room allows for other components to do their work in a comfortably chilly environment. I'm talking about hard drives, switches, media converters and so on. All the things you would never care cool in some elaborate way with water or phase change.


That beeing said, I agree they are resource hogs.

You would still have to cool the water, and using water cooling on such a scale would likely be ultimately more costly and potentially damaging if you spring a leak. Refrigerants as Freon are banned in many/most places and the possibility of a leak would be a serious concern.

Those objections aside I'm not sure how you could implement either on a practical level to cool rack after rack of servers of different configurations and designs, not to mention shelved systems in tower cases.

HVAC is always a tradeoff of cost and efficiency. some of the best solutions I've seen are the new ducted cooling cabinet system from APC. Not cheap though.

The form of cooling used in data centers is traditional, and dates back to (at least) the 60's. Nearly all mainframe machine rooms built since the early 60s (or even late 50s) have used raised flooring to serve both as a positive pressure cool air plenum, and for cabling. The cool air is provided by large (typically 20-ton) evaporator-blowers, and the heat is sent to roof-mounted condensor-blowers.

This form of cooling has several advantages: reliability (these A/C units have really long lifetimes), redundancy (lots of A/C units), scalability (add more units as needed without rebuilding anything), reconfigurability (add/replace computer equipment as needed by moving around or replacing tiles in the floor without changing the cooling system), tidiness (all cables are under the floor, which is therefore easy to clean), continuous operation (pulling a few tiles during maintenance does not compromise cooling), etc.

[I spent many years in such rooms, doing all of the above. One thing they are not is quiet! The typical noise level is 85-90 dB.]

More recent data centers, built from racked servers instead of big mainframe boxes, simply adopted the existing model, for all the reasons above, or even more simply because they were originally mainframe machine rooms.

I work in a campus datacenter.

I have to say if people want less power used in a data-center STOP MAKING INSANE DATA DEMANDS!

People want their 4-gigabyte mailbox and their campus ITunes server and their this and that........ All in a redundant load-balanced configuration because GOD FORBID anything should ever go down at 2AM heads would roll in my department. You need a development system of same type as the production systems to test software on extensively before production rollout, which is usually powered up all the time.

There are consequences to these expectations. Many people think that by putting their data needs off in a Google center somewhere they are being clever. They should look at the power consumption of a Google setup with all it's redundancy and it's throwaway PC thinking. They leave even dead systems powered up in the racks because it's too much trouble.

The problem isnt always insane data demands, but the morons speccing new systems with 20x146GB 15krpm SCSI drives instead of using lower-cost/higher capacity SATA drives.

20 x 146 is 2.9TB raw storage
8x 500 is 4TB raw storage...

a little googling shows the 146Gb drives using the same amount of power as a Samsung 500GB, with less than 1/3 of the storage... obviously the SCSI drive is much faster (lower latency times etc), but is that really needed for, say, a mail server with less than 10k users? nah.

An in-house built fileserver using high-end commodity hardware, 500GB SATA drives and Solaris/ZFS for storage will use drastically less power than a SCSI based setup with HWraid for a very small difference in real life performance. Obviously SCSI drives are rated for higher temps and longer MTBF, but with Raid5/6 (Raidz1/2 equiv.) its not much of a problem.

Hah, my friend does this stuff. They use massive amount of 15k RPM scsi drives off of fiber channel, and the power useage in the data center he works in is measured in kilowatts.

Haha, I bet.

I've got 3.2TB of storage in raid5 now, probably using less than 400W, thats with folding running on the 3.2 Celeron (Prescott)...

Speaking as someone who has 8+ years of experience with large-capacity datacenters and server farms, the problem with ATA drives (parallel or serial) in the past have been:

1) Reliability
2) Speed
3) Throughput

SAS drives in a 2.5" format are revolutionizing data storage, but the biggest problem is that in order to have a performant disk array, you need to have a lot of spindles, preferably in a RAID 0+1 or RAID 1+0 configuration. Raid 5 / 6 just doesn't have the I/O throughput to keep up with real-world load (especially in regards to web traffic and highly transitive SQL DBs). If anything, the biggest problem (at least for our servers) is that for SQL servers, there is too much disk space as the drive sizes have increased but we still need a high number of spindles for I/O performance purposes, so there are situations where we have highly transitive DBs that are relatively small (e.g. under 100GB) that are sitting on 1/2TB arrays. Unfortunately, all that space goes to waste. Until disk I/O performance improves to the point where you can get good DB performance from a relatively small number of spindles (e.g. 4-6), you'll be stuck with these large, wasteful datacenters.

On the other hand, some server companies (Rackable and Verari Systems, for example) are building plenum-style integrated rack solutions that are positively coupled to the (raised) floor and (suspended) ceiling that allows just the servers to be cooled and not the surrounding workspace. The servers themselves have fans, but also rely quite a bit on natural convection and 'chimney effect'. Unfortunately, HP, Dell and IBM have not gotten on the bandwagon yet.

-D

Maybe it's time to revive fixed-head disk or drum memory.

SDS Sigma Rapid Access Data

Cost per-GB would be horrendous, but it sure would be fast.

Good find.