No, you don't have to overclock your CPU to run 1866MHz memory.
The2600k/3770k would supports ram upto 2133MHz maybe even 2600MHz!.
So, if you'd install a 1866MHz memory, It'll be listed as 1333mhz. You'd have to go into your bios & set/Overclock your memory speed to 1866MHz.
The 2600k and 3770k both support memory IO bus speeds of 667Mhz (1333 MT/s) and 800Mhz (1600 MT/s) respectively. While memory speeds higher than this are certainly possible and are easily attained on most
chips they do require that the memory controller be overclocked and are thus neither supported nor guaranteed.
So yes, a part of the CPU does need to be overclocked. It's just the memory controller and not the IA32 cores.