For the most part a jump of two tiers is not that significant.
If you are upgrading a graphics card, make it a big jump of more than two or three tiers.
If you don't, you may be disappointed if you don't see a big improvement.
But, in the one example you gave of a GTX670 to a GTX690, it is a massive jump.
A GTX690 is about the equivalent of GTX680 sli.
For most tiers the performance difference is approximately 10%. The difference between 670 and 690 is close to 100%, depending on how well your games support SLI and how well it scales. As said above, you pay through the nose for dual GPU cards. Frankly the 690 is overkill if you aren't planning on gaming across multiple displays or on a 2560x1600 monitor.
Well, what card do you have now? If you have a 670, then the only cost effective way to really boost performance is to go SLI.
Personally, I think of tiers as a rough guide. When I have an idea of the card I want, I will compare it in benchmarks to my current card. (I know benchmarks are not perfect but it works). For example; I have a 560ti and I can get a 7970 Ghz Edition for $469 AU. The performance gap is huge and justifies the price to some extent. If I had a 670, I would keep that for another generation or 2 as it would not be worth upgrading.