Both can support high resolution gaming. But PCI-Express is the more modern format. Very few, if any, motherboards and GPU's released today use AGP.
PCI-E offers higher bandwidth than does AGP which at the highest resolutions can offer better performance.
Click below if you would like more information on the two formats.
http://en.wikipedia.org/wiki/PCI_Express
http://en.wikipedia.org/wiki/Accelerated_Graphics_Port