Actually, the bad cap problem is/was quite widespread, not just limited to PSUs, motherboards and other electronics.
Years ago, I bought a cheap (~$100) modest drill press for my workshop from Canadian Tire, one of our longer-lived commercial institutions. The drill, obviously cheaply made in China, worked well enough for a while, then a couple years later, slowly started to lose power... till one day it would not even start. Just buzzed when the power was applied. I checked all the mechanicals, the belt, etc., without success. I went back to Canadian Tire for help, and of course they'd stopped carrying the thing long ago & could only suggest buying a new one. I abandoned it for a while, resorted to slow careful hand-drilling when necessary. But the drill press bugged me, and in the end, I tore the thing apart -- to discover that there was a large externally mounted electrolytic capacitor covered by a metal "bubble cover" on the back of the unit. This capacitor -- you guessed it -- was bulging like a pregnant thing.
A bad, leaky cap. I found a Chinese replacement with slightly higher capacity & voltage rating on eBay for something like $5 delivered. Replaced it when it arrived a couple weeks later, and magically, the drill press began working 100% again, maybe with even more gusto than before!
That was about 2 years ago. Still working fine.
Spot & swap out the bad caps. If the PSU has been sitting for a while, they're probably totally discharged & safe to handle. If not, then leave it unplugged with the switch on "on" -- for a few days if you want to be totally safe.