I would, at least to make sure everything works first.
As for being able to turn the computer off, you can just flip the power switch or pull the power cord from the back of the power supply. I wouldn't make it a habit, but it should be fine for the first couple of times.
I cheat with mine. I have an extra power switch from an old computer case I plug in to the front panel connector. Makes it a whole lot harder to accidentally short stuff out.
Casey