In Windows delete, the filename changes and is flagged to allow the sector(s) to be over-written by new data, but stores the new name in the MBR. At that point it can be retrieved through Windows recycle bin, or through other, similar software. It will stay there and be available for restoring until other data overwrites the sector(s), you restore it, or when other deletions reach your recycle bin capacity.
To ensure the data is really deleted, you can use various secure shredder software, e.g., Spybot S&D has a utility for deleting files that overwrites the files several times, which makes it unrecoverable, there are other utilities that also do this.
To securely erase an entire HDD, you can use the mfr's. utility software to write 0's and 1's to every sector of the HDD.