The best system probably is done using an external hard drive. With one of those and some backup software you can make backups to it and then disconnect it and even remove it to a different physical location for safe storage. You can bring it back for restoring from the backup, or for updating the backup set with newer data. So, look for an external hard drive and backup software. Yo may buy them together as a package, or separately.
To get an external drive you can buy a complete unit ready to use, and some of them even include some backup software - may not be the best available, but the price is right. Alternatively you can buy separately a case or enclosure and mount in it yourself a hard drive you buy. That way you get exactly the HDD and case combination you want, usually cheaper by a bit than buying the finished unit. Assembly is simple, but it depends on what you are comfortable with.
In choosing an external HDD, consider carefully the interface between it and your computer. USB2 is almost EVERYWHERE, so you are virtually guaranteed to be able to use it on any machine. However, other systems have faster data transfer speeds. IF you have a eSATA port on your machine they are very good. Some external HDD's and separate cases that use eSATA also will provide you with an adapter plate if you don't already have an eSATA port. Basically it mounts in the back of your machine in place of one PCI card slot and provides the eSATA connector. You plug its cable internally into an unused SATA port on your mobo. The other fast system is IEEE1394a, aka Firewire 400. It is a little less common, but still frequently available for video data transfers. IEEE 1394b, aka Firewire 800, is much faster but even less common. Many external HDD units have at least TWO interfaces available - mine has both USB2 and eSATA.
IF you buy separately an enclosure and HDD, make sure you also pay attention to the INTERNAL interface. This is completely separate from the case-to-computer connection. The case must be able to accept the type of HDD you buy - usually the only choices are IDE or SATA II. Just make sure they match. Of course, if you buy a complete finished unit, this does not matter at all unless you plan to change it later.
Virtually ALL desktop-size external HDD units are based on 3½" standard internal HDD's, and they ALL will have their own power supply systems. It will be either a "wall wart" plug, or a simple cord to the unit. Among portable external units based on smaller HDD's you will find some that can be powered entirely from the USB port, but that is because they use less power and are much smaller so they have lower storage capacity. The portables usually also are more expensive.
Make up your own mind whether you need one with a cooling fan system. Many like them because they keep the HDD unit inside cooler to prolong the life. However, some think that is not necessary and that the fan itself will wear out. One with a fan is usually more expensive.