Will this be a good rig for my first ever budget PC?

You need a better card, also i'd get a b75 motherboard, the h61 mobos needs a bios update in order to use the cpu.

My suggestion :

PCPartPicker part list / Price breakdown by merchant / Benchmarks

CPU: Intel Core i5-3470 3.2GHz Quad-Core Processor (£139.80 @ Aria PC)
Motherboard: MSI B75MA-E33 Micro ATX LGA1155 Motherboard (£42.98 @ Novatech)
Memory: Corsair XMS3 8GB (2 x 4GB) DDR3-1600 Memory (£32.99 @ Amazon UK)
Storage: Seagate Constellation ES 1TB 3.5" 7200RPM Internal Hard Drive (£43.82 @ Aria PC)
Video Card: EVGA GeForce GTX 660 2GB Video Card (£161.98 @ Dabs)
Case: BitFenix Merc Alpha (Black) ATX Mid Tower Case (£29.99 @ Aria PC)
Power Supply: XFX ProSeries 450W 80 PLUS Bronze Certified ATX12V / EPS12V Power Supply (£36.98 @ Novatech)
Optical Drive: Lite-On iHAS124-04 DVD/CD Writer (£12.98 @ Scan.co.uk)
Total: £501.52
(Prices include shipping, taxes, and discounts when available.)

With b75 motherboard, a good quality 450W psu(it's plenty for the setup) and a much better card, the 660 GTX.

It should. But I'd go with 1600mhz ram, unless there's a big price difference. Also, if your mostly gaming, why not trade out for a core I3 and a 7850, its probably the way to go

That is an interesting opinion. 1600 MHz ram is out of my budget, for now. I will get better RAM later. As for the I3 core, I am surprised to hear that, because most people have told me to get I5. I checked out the 7850 and it is a very nice card. I might get it later. But is this a good foundation for my first PC?

It is all good and well but you forgot to include OS. Can you include the OS in your list and then downgrade on some hardware to come to a total of £500.

Why does everyone that I have seen so far that do the partpicker list forget to add CPU cooler? Is it really not necessary?

The cpu comes with one in the box

It's not necessary, the stock cooler is fine, here's an updated list, it goes ~22£ over, but it's worth it, keeping the i5 and the 7850 :

PCPartPicker part list / Price breakdown by merchant / Benchmarks

CPU: Intel Core i5-3350P 3.1GHz Quad-Core Processor (£130.98 @ Scan.co.uk)
Motherboard: MSI B75MA-E33 Micro ATX LGA1155 Motherboard (£42.98 @ Novatech)
Memory: Corsair XMS3 8GB (2 x 4GB) DDR3-1600 Memory (£32.99 @ Amazon UK)
Storage: Seagate Barracuda 500GB 3.5" 7200RPM Internal Hard Drive (£39.98 @ Novatech)
Video Card: XFX Radeon HD 7850 1GB Video Card (£126.94 @ Scan.co.uk)
Case: BitFenix Merc Alpha (Black) ATX Mid Tower Case (£29.99 @ Aria PC)
Power Supply: XFX ProSeries 450W 80 PLUS Bronze Certified ATX12V / EPS12V Power Supply (£36.98 @ Novatech)
Optical Drive: Lite-On iHAS124-04 DVD/CD Writer (£12.98 @ Scan.co.uk)
Operating System: Microsoft Windows 7 Home Premium SP1 (OEM) (64-bit) (£67.99 @ Aria PC)
Total: £521.81
(Prices include shipping, taxes, and discounts when available.)

Thank you.