Ok, if you only drain it once and recharge as soon as the battery indicator showing the battery is flat, I am fairly confident that the battery is not drained to a point where it gets damaged.
The battery could take up to that long to recharge. Charge rate depends on the battery capacity and the output current of your charger. If you want faster complete recharge you could choose battery with smaller capacity (which is stupid to do) or get a rapid charger.
You can estimate the charge time by dividing the battery capacity to the charger's output and add times the time by 1.2 to account for inefficiency.
so if your battery is 2500mAh and your charger is 500mAh, you get 2500/500*1.2=6 hours
You don't need to fully drain Ni-MH battery before recharge because it does not have memory effect.
I don't think removing the battery from the camera makes much of a difference, the battery could be drained by the camera's clock but it will be very low usage. The batteries will naturally lose charges. But if you are putting the camera away for a long time, you probably want to remove the battery.