I would do as Steve posted, but just as a comment, sometimes, when i install drivers, usually Intel RST, some versions make the hdds ejectable like usb sticks, so you could eject it, but you will need to do every time you boot or every time you need to take it off.
Personally my recommendation is to go with external USB3 disk to do backups, usually its recommened that backups are not in the same box as this would procect you against thiefs, a fire, flooding, etc. The newer 2.5hdd with large capacity have very good performance, almost no penalty with usb3, i usually transfer movies from my server to my desktop pc and then to its usb3 hdd to take them on the road with me, and the hdds from my server usually saturate the gigbit lan, and there is no penalty doing it directly to usb3, practically im restricted by the speed of the 2.5, where its still above the gigabit lan. An external hdd that its using the new samsung 2tb is the Seagate Expansion 2TB USB 3.0 Portable External Hard Drive (STBX2000401)
, imo a good deal for $85, considering you end up with usb3 and no need for external power, ideal to me for a backup drive.