I've had my 180 mini since it first came out and it's been good to me. I don't use any front fans and the back and top are set to the lowest speed. It's quiet but there's still a bit of noise.

My new build is going to consist of:

3770k with thermalright hr-02 and TY-140 fan
Asus Gene V matx
Asus 670GTX
8GB ram

Since i'll no longer have an external exhaust videocard more heat will be dumped into the case.

Would moving to a define r4 be worth it? Would i get better cooling with less noise and if so what would be the ideal fan setup?

Sound wise, the R4 probably has better insulation. Airflow wise I don't think you'd have issues in the P180 Mini in idle, if you can fit the GTX 670 in there. The R4 has the side panel hole available, but that's just another hole for noise to escape through if you pop it open. I'm not convinced the bottom intake in the R4 is any more beneficial than opening the P180 Mini's compartment hatch.

Obvious improvements before swapping cases would be getting quieter (more CFM/dB) fans, removing/isolating HDDs and declining the 670 in favour of a cooler-running card. The i7-3770K makes me think you're over-investing here.