WD-40 also works great, just squirt alittle onto a cloth and rub gently. Then flip the cloth over to a dry spot and buff the residue off.
According to Arctic Silver's Instructions
"Never use any oil or petroleum based cleaners (WD-40, citrus based cleaners and many automotive degreasers) on the base of a heatsink. The oil, which is engineered to not evaporate, will fill in the microscopic valleys in the metal and significantly reduce the effectiveness of any subsequently applied thermal compound."
I use regular acetone (an easy to find source is non-scented regualr nail polish remover) to remove the thermal compound on my CPUs and heatsinks. I put some on a rag and sort of soak the compound to soften it first, then apply more and wipe it clean. Acetone evaporates quickly and leaves no oily residue.