After months of tinkering, I finally came to something acceptable, noisewise. I'd like to present what I've done. First the general setup:
AMD Athlon XP 2400+ @ 780MHz, fan @5V
Antec Sonata case with Truepower 380 PSU, swapped&undervolted fan
Casefan @ 2.5V
Samsung SP80, 80Gig single platter HD, suspended
Older TNT2 graphics card, fanless
No floppy drive
CPU: I regret getting an Athlon XP 2400+, it turned out I don't need this much processing power. I'd buy a fanless C3 nowadays. Anyways: It turned out that I underclocked the processor! The processor is "locked" and so I'm only able to adjust the Front Side Bus. I cranked it all the way down to 52MHz (normal is 133MHz) and a resulting 781MHz (instead of 2GHz).
Surprisingly this didn't result in much difference in the temperature of the processor. Before it was around 65-70° and now, after massiv underclocking it stays at 50°. I expected a bigger drop in temperature, but it's still enough to run with fans on minimum. I know this isn't much performance, but I work mostly in a terminal anyways, and I can always go to the bios and turn it up if needed.
FANS: The processor has a cheap but quiet Spire Whisperrock IV heatsink and fan @ 5. I'm sure the Thermaltake/Zalman heatsinks are superior, but this will do for now.
The case fan is the stock fan found in a Sonata. One Fan Mate wasn't enough for this fan and so I used two (!) Fan Mates, attached one after another, to bring the voltage down to 2.5. This made the case fan very silent (although still audible with your ear next to it).
PSU: The PSU brings bad memories: it's the nastiest part of the computer.
I first hesitated to touch it, hearing all the stories of how dangerous it can be. They were right: I got zapped twice (!) and a couple times the power fuse (whatever it's called) went out, and the house was dark. Always be careful when working on PSUs! (Wear a glove on one hand, be patient when working, don't close the cover when the power plug is in the PSU, etc).
So i eventually managed to swap the fans and put a silent Papst fan in, the one rated at 17CFM. Big improvement! After awhile though, I noticed this Papst fan was still the loudest while sleeping... So, being brave, I opened the PSU, and tryed putting in a Fan Mate. The results can be seen on the picture.
My only concern with this setup is heat in the PSU. Here's some math I did, just skip over if not interested:
The PSU fan has 17CFM, while the stock had 34CFM. the PSU can do 380W. i tried to calculate how many Watts I was using:
25W CPU, 10W Harddrive, 4W fans (?), Mainboard (?), 4W graphics card, CDROM don't know: 3W. Total roughly: 50Watts.
The efficiency at 90W load is 65%. I'm using 50Watts -> 82.5 Watts used by the PSU and 32.5 Watts heat. Not much heat being generated -> the 17CFM fan @ 5V suffices. The fan is audible, so in the future (if I have the money), I'll consider a fanless PSU.
Harddrive: First it was in the rubber slots found in a Sonata, but then I realized it would be more silent (less case vibration!) if suspended. I simply took a long string and put it through the holes of the case (where the CDROM normally goes). Then put it on some foam so it rests nicely. -> Small improvement in seek noise.
Before this sweet Samsung, I didn't notice how loud HDs can be!! Before I had a Western Digital 60G, and it causes so much whine, that it hurst your ears! Luckily I noticed this and it's gone for good.
Now it's very silent (little humming). My friend couldn't believe it: I asked him: do you think it's running or not? He was amazed. I am too. I have this computer running 24/7 and now I can sleep well. I'd like to thank SPRC for your wonderful help! Keep on silencing.