Cosmin, if everyone who came here knew all about these subjects, there wouldn't be any questions! You might get some rude replies if you ask questions that have been answered many times, but not often. These are common questions and you can learn a lot by exploring other threads here in this section of the forum. Also by reading some articles. Here's a few I recommend:
http://www.pcityourself.com/choosing/memory.php
http://en.wikipedia.org/wiki/CAS_latency
http://www.tomshardware.com/reviews/phenom-overclock-athlon,2161.html
http://www.tomshardware.com/reviews/memory-scaling-i7,2325.html this is about i7 but covers the slight difference users will experience with different RAM speeds. Skip to the Conclusions page.
You mention going with BOTH the 2GB and 4GB kits, is that correct? Most would choose the 4GB kit and either another one or to leave the slot open for later expansion when memory prices drop a little. Few users need more than 4GBs of RAM at this time anyway - with 4GBs you'd be in the top 5% of current computer users (not the people here, but normal users!) If you have Win XP or other 32-bit versions of Windows, you can't address more than about 3GBs of RAM anyway.
Overclocking refers to pushing beyond the specifications guaranteed by the maker of a product. OC might be increasing the speed of the CPU, or the bus or the RAM. Putting in 1.8V RAM and setting your BIOS to that 1.8V is not overclocking. But reducing the CL or increasing the speed (to 1800 perhaps) is OC. Same with increasing the voltage to 1.9 or such to make it work.
When increasing CPU numbers, many folks get bigger "steel radiator things" to aid in cooling. They're called CPU coolers/fans/heatsinks. A cooler is a heatsink with tubes in it to draw the heat out better. All of them have fans to blow air over the heatsink or cooler and make the heat come off faster. Very few people add special cooling for their RAM - the fins you see sticking off of many RAM modules help to take the heat off and the air circulation inside your case takes it out of the system.
Maximim voltages and maximum anything is what you find out when you overclock. Your system will report the current temp of many items, usually including the CPU and the case, sometimes the RAM and hard drives. A program like CPUid Hardware Monitor will show you those numbers. Another program called CPUz is helpful in seeing how your system is running. Both available at www.cpuid.com in free versions. There are LOTS of programs related to OC and monitoring, here's a very long list of them.
http://forums.tweaktown.com/f69/latest-overclocking-programs-system-info-benchmarking-stability-tools-30530/
Sacrificing Bandwidth vs Latency. Most all agree that both are worried about too much, but Latency is more important. Always choose the best latency at a given bandwidth (speed). Actually bandwidth and RAM speed aren't the same thing, but we'll assume they are! So you choose 1600 @ 8 over 1600 @ 9 given all else equal. And many recommend choosing 1333 @ 7 over 1600 @ 8 - because the lower latency will result in better performance even giving up the speed. But all of these differences are very minor.
That should give you enough to work with for a while - good luck!