I have an Intel E6700 in an Acer Aspire Z3731-UR21P running Windows 7 Home Premium 64-bit. The drive is 1TB with 4GB DDR3 memory. I want to overclock it a bit and still keep the stock cooler. About 4GHZ is fine...I don't want to fry the CPU, I just got the system. Any suggestions will be appreciated.
You have somewhat unrealistic expectations of your processor. Also, there are two E6700s. The newer one is clocked at 3.2 GHz, while the older one is at 2.66 GHz.
If you've got the 3.2 GHz version, you can probably get 4 GHz out of it, but no way are you going to get there with the stock cooler. Intel's stock coolers are pretty awful.
If you've got the 2.66 GHz version (which I'm guessing you do), I'm sorry but you're not going to get that chip to 4 GHz. I have this chip, and it's only stable up to around 3.6 GHz using a good aftermarket cooler. I wouldn't expect much over 3 GHz with the stock cooler.
My recommendation would be to just try it and see how far you can push it. Keep an eye on your temperatures, they're going to be your biggest problem with the stock cooler. Keep your voltage within the limits Intel specifies (different for the two models of 6700), and stress test for 8 hours with Prime95 once you think you're stable.