Yeah it'll definitely do the trick.
I OCed a build with a 2500k and MSI P67A-G43 yesterday actually, the G45 should work in the same way... Just push delete on startup to get into the BIOS, click 'Overclocking' (I think thats what the button is called), and play with your settings. Basically, you will change your CPU Multiplier (33x is stock, 33x100mhz = 3.3ghz) to whatever you'd like (40x100mhz for 4.0ghz, or 45 for 4.5ghz for example), and then add a little bit more voltage to your CPU. It's farther down on the same page in the BIOS. You'll probably want between 1.3 and 1.35v for an overclock from 4-4.5ghz.
There is a bit more to it than that, but that'll get you started. There are TONS of guides to overclocking Sandy Bridge chips out there as well, google a bit if you'd like to learn a bit more.