overclocking cpu is safe (by safe I mean it is possible, no system with overclock is safe actually) with a decent aftermarket cooler (typically a cheap but effective and good one like CoolerMaster Hyper212Evo)
overclocking gpu is kinda not that safe (especially if your gpu has stock single cooler, dont go for it) but possible with slight frequency upgrades
overclocking memories is safe (just like cpu overclocking, it is possible but do it on your own risk) there is two way to do it, you either shorten the ram timings or you raise the frequency (together memories are usually less stable)
for overclocking your system... there are thousands of guides, just search the one for your cpu type...
(but if you are going to do it, just get the Hyper212Evo cooler first of all)
good luck.