After months of tinkering, I finally came to something acceptable, noisewise. I'd like to present what I've done. First the general setup:

AMD Athlon XP 2400+ @ 780MHz, fan @5V
Antec Sonata case with Truepower 380 PSU, swapped&undervolted fan
Casefan @ 2.5V
Samsung SP80, 80Gig single platter HD, suspended
Older TNT2 graphics card, fanless
No floppy drive

CPU: I regret getting an Athlon XP 2400+, it turned out I don't need this much processing power. I'd buy a fanless C3 nowadays. Anyways: It turned out that I underclocked the processor! The processor is "locked" and so I'm only able to adjust the Front Side Bus. I cranked it all the way down to 52MHz (normal is 133MHz) and a resulting 781MHz (instead of 2GHz). Surprisingly this didn't result in much difference in the temperature of the processor. Before it was around 65-70° and now, after massiv underclocking it stays at 50°. I expected a bigger drop in temperature, but it's still enough to run with fans on minimum. I know this isn't much performance, but I work mostly in a terminal anyways, and I can always go to the bios and turn it up if needed.

FANS: The processor has a cheap but quiet Spire Whisperrock IV heatsink and fan @ 5. I'm sure the Thermaltake/Zalman heatsinks are superior, but this will do for now.
The case fan is the stock fan found in a Sonata. One Fan Mate wasn't enough for this fan and so I used two (!) Fan Mates, attached one after another, to bring the voltage down to 2.5. This made the case fan very silent (although still audible with your ear next to it).


PSU: The PSU brings bad memories: it's the nastiest part of the computer. I first hesitated to touch it, hearing all the stories of how dangerous it can be. They were right: I got zapped twice (!) and a couple times the power fuse (whatever it's called) went out, and the house was dark. Always be careful when working on PSUs! (Wear a glove on one hand, be patient when working, don't close the cover when the power plug is in the PSU, etc).

So i eventually managed to swap the fans and put a silent Papst fan in, the one rated at 17CFM. Big improvement! After awhile though, I noticed this Papst fan was still the loudest while sleeping... So, being brave, I opened the PSU, and tryed putting in a Fan Mate. The results can be seen on the picture.

My only concern with this setup is heat in the PSU. Here's some math I did, just skip over if not interested:
The PSU fan has 17CFM, while the stock had 34CFM. the PSU can do 380W. i tried to calculate how many Watts I was using:
25W CPU, 10W Harddrive, 4W fans (?), Mainboard (?), 4W graphics card, CDROM don't know: 3W. Total roughly: 50Watts.
The efficiency at 90W load is 65%. I'm using 50Watts -> 82.5 Watts used by the PSU and 32.5 Watts heat. Not much heat being generated -> the 17CFM fan @ 5V suffices. The fan is audible, so in the future (if I have the money), I'll consider a fanless PSU.


Harddrive: First it was in the rubber slots found in a Sonata, but then I realized it would be more silent (less case vibration!) if suspended. I simply took a long string and put it through the holes of the case (where the CDROM normally goes). Then put it on some foam so it rests nicely. -> Small improvement in seek noise.
Before this sweet Samsung, I didn't notice how loud HDs can be!! Before I had a Western Digital 60G, and it causes so much whine, that it hurst your ears! Luckily I noticed this and it's gone for good.

Now it's very silent (little humming). My friend couldn't believe it: I asked him: do you think it's running or not? He was amazed. I am too. I have this computer running 24/7 and now I can sleep well. I'd like to thank SPRC for your wonderful help! Keep on silencing.

Meanwhile the PSU is getting HOT. Either I turn the fan up (loud) or it seems too dangerous...

I decided to try something new out, and put the PSU ontop of the case instead of inside.

-> the PSU gets cooler air
-> it has a bigger surface area for letting air blow over
-> it doesn't suck in the cpu heat
-> cables are now outside (better airflow)

Sounds like you need a PSU airduct.

I'm curious at what temperatures you are getting for the inside of the case. I recently bought a Whisperrock IV as you have in your setup.

streety,
yea a PSU airduct is probably a better solution. i'll consider it.
needquiet,
case temps are at 34° (according to the bios). also: the case fan makes a big difference in temps.

OK, I've been tinkering some more and here are the results:

I got temperature monitoring to work under linux. So I don't have to reboot all the time to check the temps in the BIOS. Furthermore I scrapped up some money and invested in a Zalman AlCu processor cooler. Interesting temps:

m/b...motherboard
fsb...front side bus
all temps are under full load (folding)


Spire Whisperrock IV Cooler with Fanmate @ 5Volt (=1000 RPM):
@70 FSB
cpu: 56°
m/b: 35°

@85 FSB
cpu: 59°
m/b: 36°

@100 FSB
cpu: 61°
m/b: 36°

note: the power supply feels hotter at higher cpu speeds, due to the processor drawing more power!

Zalman AlCu with Fanmate at 5Volt (=1350 RPM):
@133 FSB ('normal' speed)
cpu: 61°
m/b: 39°

conclusion: the zalman cools much better than the whisperrock. i recommend this cooler if you're currenctly using a cheaper solution. -> don't save on the heatsink guys

note: all the readings were from lm_sensors. the BIOS readings are always 3-4 degrees C higher than the software readings. i wonder which one if more correct... from my understanding the readings are from the bottom of the headsink and the core of the processor is roughly 10° hotter. the processor is rated to work up to 85°, so I'm assuming my readings shouldn't exceed 75°.

also: the case fan @ 2.5Volt makes a big difference in hard drive temperatures. to monitor hd temps under linux i use "hddtemp". hard drive temps are between 37-39°.

note: I changed my mind about leaving the PSU outside of the case, like MikeC, and put the PSU back into the case... I think it's better this way. I noticed the PSU itself procuded a fair amount of noise, even when run fanless! It was a very high pitched noise, like from noisy harddrives. I figure it will emit less noise inside the case, even though the case temps increase.

future plans: the main noise right now come from the harddrive, general vibrations in the case, and from airflow noise (air moving through the case making a "swoosh" sound). it's pretty silent now, i can sleep well with it running, but still not perfect. Sigh I should just leave the box alone...

If anything I'll invest in case dampening (maybe acoustipack, although expensive). Maybe a silent notebook drive (even though i already got this harddrive extra ). Maybe a fanless PSU... either "normal" kind, or the external PSU's used with microATX boards (the kind you find with a notebook/laptop)...

Sigh........ Silencing is fun but addictive!!