I'm experiencing a piercing high pitch noise from my system. As I quiet fans and hard drives better, it becomes more and more annoying.

Possible culprits:
CPU - loading the CPU with Prime95 or IntelBurnTest has no effect
GPU - replaced a 6950 with a 670 and the noise remained exactly the same. I do get some electrical noise from the GPU under heavy load, but it's of a different character and tolerable considering it only happens under specific conditions, usually when I'm gaming and thus distracted.

This leaves me with PSU, which I'm beginning to strongly suspect.

PSU is a Seasonic X-660, and I'm sure I've heard of coil whine with the x-series.

The noise is always present, system load seems to have no effect.

What do you guys think?

I would listen to the system with the case open and easily accessible in order to try and pinpoint where the noise is coming from. You might want to use a cardboard tube (such as from an empty paper towel roll) up to your ear to help determine the source of the noise.

Motherboards also have coils that can whine. A UPS can also buzz or whine.

Thanks for the suggestion. I'll try that next time I get to the end of a paper towel roll. I guess the other test would be to swap in another PSU. I have an X-560 in my HTPC that would slide right in (same modular cabling).

If I determine that it is in fact the PSU, does anyone know whether will Seasonic replace it through RMA? It's a great unit otherwise.

Actually a paper towel roll works even better when not completely empty, but not sure I would try that if you are married.

It would be a valid reason to RMA as far as I know, but the case may be the same as with my Seasonic S12-II: the noise could be a feature, and make a return with the replacement unit. In that case it's just best to buy another model.

PSUs often whine but pretty much every component (except ram i think) has the ability to start whining.

I have fixed a lot of whining/broken PSUs and its usually a cap that has blown or degraded (if you open the case you can often see this!) letting noise into an inductor/transformer.
The caps can be replaced but usuallyl the dielectric in the transformer will need replacing (nail polish etc works if you can get at the coils, but usually are sealed).

I would just look inside the PSU for a bulged cap.

Unlikely to find a bulged cap (or any other defective part) on a Seasonic X-660. These come with a 5-year warranty and have not been on the market that long.

For older motherboards that had coil whine, putting some 100% Silicone caulk in/on the coil usually fixed the problem. Might work for the PSU also, but will void the warranty when the PSU is opened up for any reason. Make sure you use 100% silicone.