An overclock to 4.0 GHz is a difference, what you will notice really depends on what you are running.
As far as voltages, it depends on your motherboard, your power supply, and your cooling solution. If you are running a stock cooler, I would recommend a basic multiplier increase, leaving everything else at standard(or auto if you don't like putting hard numbers in) values and testing your temps at 3.8 (x19) before you bump it up to 4.0(x20).
If you are running an aftermarket air or closed water block, and your motherboard can handle it, you can try to go higher and tweak some voltages, or start tweaking the bus speed.
Just make sure you turn off all the power saving options in your bios before you start doing anything.
And for reference, I also have a FX4100, with an Asus M5A97 and a Antec 920 closed loop. I can overclock to 4.6 just from increasing the multiplier and keeping fairly standard voltages and run Prime 48 hours stable. It does decrease boot time dramatically. So the chip will overclock, just make sure you keep it cool, and only change one variable at a time, while making sure not to take too big of steps.