There may not be a perfect answer to this query.. I guess any aspect that cannot be altered is important.. For instance, the clock speeds can be altered (mostly) so they do not count as the outright factor.. Rest all specs such as memory type and size, shades units etc cannot be changed thus become a more important factor..
There is a site that lets you graph EVERY spec side by side... slicker than cow dung. I guess they took the time to put it all into a database. Look for a tool called the "Graphinator"... the people at this site use GPU hardware specs to estimate card performance. Its pretty neat.
Memory - at least 1GB, anything lower 512 is fine. On Namelessonez's note: a combination of larger memory size, higher frequency and wider interface is always better.
Core Clock speed / Units / shaders = I think this depends on the whole architecture of the card. Take the nVidia GTX 465 and gtx 460 for intance. The 465 has more of everything faster core; but the 460 has better resource management so it can produce the same performance at lower temps and less energy.
A good place to start is to look at Tom's charts as a guideline to figure out what you need.
^^Yes they do. For ATI the first one normally denotes generation of the cards. Like HD5670 are the latest DX11 cards. HD4670 is the previous generation DX10 card. The next numbers determine the performance segment it falls into. General would be higher the number, higher the performance.