What kind of hard drive is it? Sata? IDE (PATA)?
The PATA connectors can look like they are plugged in all the way, and are really unseated on one side. Make sure the data cable connector is pushed in all the way (and the power connector).
Also, if you've added or taken away a second hard drive, you might need to change the jumper setting.