Because It has 2 physical cores that do more work than one (even though a P4 can handle 2 threads it splits those along 1 core). So if you think...a 2ghz Dual Core can take 2 threads, both threads of information are being processed at the clock speed of 2ghz, at the same time and they dont interfere with eachother.
In a P4 they are crammed into 1 core, splitting the processing power between the 2, and cramming it out the other end. Putting it that way makes the P4 seem clumsy, (to be fair, its amazing technology) its not, but its just not as efficient as 2 physicals.
Also the architecture is much revised, the Core2s were genious at the time, Superior to anything else. My CPU is running on the original Core2Quad architecture from way back in 2k6/7 (kentsfeild) and ive kept mine benchmarking higher than most Core i5s, some i7s and almost all Phenom IIs ive seen.