There are a few things going on here. Even in CPUs there are differences between single core performance and core count, and 1GHz on an Intel core won't run the same as the same on an AMD core. Also there are changes generation to generation, which really boils it down to particular chip design.
TO start with, you're comparing AMD to Nvidia GPUs. The core designs are even more different than Intel vs AMD CPUs, and one Nvidia core can't be compared to an AMD GPU core. Modern GPUs by nature are far more parallelized than CPUs, which is to say they run with a huge number of relatively simple cores compared to the CPU with it's smaller number of powerful cores. Thus, core count matters a lot. There are also other elements in the design of the whole unit which affect how a card plays. It's not as simple as MHz, so really judge cards by their performance in games.