They perform about the same. It all matters on how important having sm 3.0 is to you. BTW, you can get the 7600GS for cheaper on Newegg. Anywhere from $125-140. Don't spend extra on the 512MB version. 512MB is only worth it on the higher end cards.
Personally I think Sm3.0 means very little in cards of this caliber. it's not like they have the power to really take advantage of HDR nor do they have the power to last further into future games. So what's the advantage?
Anyway, stock speeds, an X800GTO is above a 7600GS. Take a look at the Digit-life charts below. they are PCI-e, but clocked the same they perform the same. Just ignore the 6800GS and 6800XT as those are beastly clocked compared to the AGP ones for sale.
Anyway, the X800GTO manages to beat the 7600GS in every single game at max playable settings. It's very close in a few games, but others are a huge victory for the GTO. IMO stock speeds and priced the same, the X800GTO is better than the 7600GS. At this level card, performance ranks above SM3.0 in my book.
When you get up to the 7600GT vs X850XT and 7800GS vs X850XTpe, then performance is so close and trading blows depending on the game, that the SM3.0 cards would both get my vote if priced the same as the equal X850. These cards and above also start to have the power to take advanatge of their features.