My question is, is it safe to remove the fan inside the powersupply? It features a small fan and I think it's generating some unwanted noise.
Generally speaking, it's unsafe and very likely it would lead to a premature PSU failure.
But perhaps the bad news is that, as far as I know, there isn't any SFX PSU which is really quiet, so I think you're stuck with it (or so about).
Well, first of all I would check where the noise come from: I would stop/disconnect the front fan, hear the noise level, and then (just for a few seconds) stop even the CPU fan (you may use a toothpick), in order to hear any noise difference. Just as a very personal bet, I think the front fan may be louder than the SFX PSU.
If you find out that the SFX PSU is the major noise source, I would check the PSU fan, if it's a 2 or 3 wires one: please take note that this inspection void the warranty.
Then, if it's a 3-wires, I would route the header to the mobo and slow down it by BIOS or SpeedFan (Windows). If it's possible.
While, if it's a 2-wires, you have to swap it with an about 1000rpm with fluid or ball bearing 3-wires one (a 4-wires PWM fan could also work in some cases), and then again reroute it on the mobo header.
You might route even to the original PSU header, but you may incurr in some startup issues (the fan might not spin at all, a tricky/risky scenario).
If your motherboard does not have any suitable fan header, you have to buy a rheobus (but I don't know where you can mount it on a SG05).
About the HDD, from a noise perspective, the Barracuda Green is generally outclassed by pretty much every low power desktop drive on the market, and it is about on par with some quiet 7200rpm desktop drives, as some 7K1000.
So a Samsung F4, an Hitachi 5K3000 or 5K1000.B, and the quoted WD Green should be noticeably preferable. You might also thin to susped it under the ODD.
Quieter than these, you have the SSD, but a 256Gb SSD is still a costly choice, not to mention the relatively low capacity.
Last option (but just for low-to-mid capacity) is a notebook drive enclosed in a SQD: double or triple the space with reference to the SSD, while the price is more than halved. But I don't know if it's really noticeably quieter than a WD Green, maybe suspended (while surely a notebook drive is less performing than it, according to benchmarks).