I just want to point out that the die size do not directly affect the performance of a CPU or GPU.
At the most basic level, a die shrink allows for lower cost of production per unit because more GPU cores can fit on a 300mm silicon wafer. The more that can be manufactured at a single time, the lower the cost. It also allows for lower power consumption (lower voltage) which should also mean less heat.
If the GTX 580 was shrunk down from 40nm to 20nm die process, the two versions would provide the exact same performance. However, die shrinks can allow CPUs / GPUs to be clocked higher for better performance since as stated above, the die shrink should allow for power consumption and heat assuming the same clock speed.
The major fact in CPU and GPU performance is the architecture itself. If nVidia should somehow totally mess up Maxwell to the point where it performance like a GeForce 2 card from yester-year, then even if it was possible to shrink the die process to 1nm it would still perform like a P.O.S.