The $160 1060, even the 3 GB is faster overall than the 970 ... The 580 is only about 15% faster than what you have and a reference card won't bring much on the resale market so hard to justify the cost for such a small increase in performance
The 580 edges the 1060 6GB by a little bit but that advantage is quickly erased and then some by the 1060's much greater OC headroom.