I suspect what happens is that, when you attach the HDD, the BIOS changes its default setting and tries to use the HDD as the boot device, and that does not work.
The solution is to go into BIOS Setup and specifically set what devices can be used for booting. Start by installing the troublesome HDD so it can be detected by your BIOS. Usually to enter BIOS Setup you must push (hold down, actually) a particular key as the machine starts to boot. VERY often it is the "Del" key, but WATCH your screen for a message about how to Enter Setup in case it is another key. Just a push or tap may not do it. I hold the key down right after turning on power until the opening Screen of BIOS Setup appears.
Normally you have a choice of tabs or main screes to make, and then proceed through sub-menus. Read your mobo manual if you have it. You need to find the place where you specify the Boot Sequence Priority. Sometimes it just allows you to scroll through a list of devices (on-screen prompts for which keys to use) and then you choose one. SOMEtimes it is organized as a two-level selection process: first you choose between optical drives, HDD's, USB stick, etc., and then within each type of device you can choose a specific unit.
You actually are setting up a priority sequence, not just one device (although only one is an option). I would normally set mine to use one of the optical drives first, then go to the HDD that has my OS, and NO other options after that. Set up that way, my machine will boot from a bootable CD in the optical drive IF there is one (useful for using special diagnostic disks for troubleshooting). But if the optical unit does not contain a bootable CD it jumps immediately to the second choice, the HDD I specified, and boots from there.
In your case you could do exactly that, EXCEPT your second choice should be your SSD where your OS is installed. Do NOT have your HDD unit as a choice anywhere. Or, of you don't want the optical drive involved as a possibility, skip that and just set the SSD as the only choice. When you have made your settings, remember to SAVE and EXIT (look for on-screen prompts) to save these changes, and it will reboot properly from now on.