It is never recommended to mix RAM, especially DDR3 RAM as it often will not work at all or without issues. If you're feeling lucky you can give it a shot but it's always best to buy ONE matched, tested RAM kit of the full quantity that you desire, to be certain that it will work.
FWIW, most people only need 8 GB. of RAM so a proper 2x 4GB. RAM kit approved for your CPU should be all you need unless you are doing CAD, video editing or using many apps concurrently. More RAM won't increase the speed of your PC however.
I might just try it and hope it works. Worst thing that can happen is it doesn't work and I'll use the 2x4 kit to make 8gb (I don't need 12gb but I guess maybe I would in the future?)