You don't specify what motherboard you have, but in general the more DIMMs the system has to drive, the lower the maximum clock speed they can be driven (or overdriven). You will always be safest with DIMMs that have been selected to be used as pairs or quads. Mixing random similar sized DIMMs can cause problems because the timings might be different.
For reference I have a http://www.asus.com/Motherboards/P8P67/.
Would I be safe to have two 4gb x 2 kits of the same RAM?