the CPU model in title does not match an i5; the 3770 is an i7 model.
Answering your question, both cards work fine with an i5 CPU, provided that your motherboard and power supply can support the card.
As for what card is better, I suggest you look at some benchmarks of the games you intend to play and see which card performs better. I personally prefer the r9 280x because it has DirectX 11.2 not just 11.0 and it has 3GB VRAM instead of 2GB which will be better as games are likely to start using more than 2GB. Both cards are pretty similar and the GTX 770 has things like Shadowplay, G-Sync and CUDA support so if those things interest you then you should look at the GTX 770.
Mantle is pretty new but it is already showing improvement...