This is probably the cheapest build I'd do today. There is no part on this list I would not buy myself for quality reasons:
LITE-ON 24X DVD Writer - Bulk - Black SATA Model iHAS224-06 LightScribe Support - OEM
One optical drive is much like another. This one has Lightscribe. You can save $3 for one without it.
ASRock H61M/U3S3 LGA 1155 Intel H61 HDMI SATA 6Gb/s USB 3.0 Micro ATX Intel Motherboard
$74.99 -$5.00 Instant $69.99
With USB3.0 and SATA 6Gb/s, this board is a little more future-resistant than the ~$55-$56 boards, and I think it's worth the $15 difference. This one has some better quality components too, like solid caps.
Rosewill R102-P-BK 120mm Fan MicroATX Mid Tower Computer Case
I've built in this case. It's small, but reasonably sturdy for a cheap case, and the fan is quiet but effective.
HIS H667F1GD Radeon HD 6670 1GB 128-bit GDDR5 PCI Express 2.1 x16 HDCP Ready Video Card with Eyefinity
Mail in Rebate Card
$97.99 -$8.00 Instant $89.99
A lot better than an APU if gaming is one of your intended purposes.
Western Digital Caviar Blue WD3200AAKX 320GB 7200 RPM SATA 6.0Gb/s 3.5" Internal Hard Drive -Bare Drive
$84.99 -$10.00 Instant $74.99
You can get 500GB for only another $10, but in a cheap build, this ought to do. You can always add a larger data drive later.
Antec EarthWatts Green EA-380D Green 380W Continuous power ATX12V v2.3 / EPS12V 80 PLUS BRONZE Certified Active PFC Power ...
Model #:EA-380D Green
$59.99 -$15.00 Instant $44.99
Has been my goto PSU for years in basic builds; solid, stable, efficient, and quiet. All this rig needs, even if you were to upgrade the GPU to a HD6850 or HD7770. All the Earthwatts PSUs have reduced prices right now though, on up to the 500W model for $60. If you're planning to keep this rig for a while and upgrade it to a stronger GPU later, you might want to stretch your budget for the larger PSU now, so you don't need to upgrade it later.
Intel Pentium G630 Sandy Bridge 2.7GHz LGA 1155 65W Dual-Core Desktop Processor Intel HD Graphics BX80623G630
Although it beats the APUs and Athlons in games, this is probably the first thing I'd upgrade if the funds are available, to an i3.
G.SKILL NS 4GB (2 x 2GB) 240-Pin DDR3 SDRAM DDR3 1333 (PC3 10600) Desktop Memory Model F3-10600CL9D-4GBNS
I've never had a problem with G.Skill RAM, it just works. 8GB has a lower cost/GB, but 4GB is enough, especially on a tight budget.
Microsoft Windows 7 Home Premium SP1 64-bit - OEM
If you don't need this, there's $100 off the price right there.
You could skimp a little more on the mobo and DVD drive, but still not get down to $500 or less, at least not and retain reasonable quality. I won't recommend anything I wouldn't buy myself. Each of the next two or three increments of $50-$75 would really improve this build. If it means saving up for another month or two, it would be worth it.
Depends what you want to be able to do with it - selections listed above aren't bad but component selection could be different if it's just for day to day web browsing vs. attempting to play games or do graphic design.