Ubuntu's gotten pretty good at hardware support, but WiFi and printer support is still pretty awful. You might benefit from a browse through
http://ubuntuforums.org/forumdisplay.php?f=332
Since you're not gaming and you care about speed at under $800, you'd probably do best with an AMD system. AMDs do very well on Linux (equal to or better than Intels at a much lower price point) except on some very specific scientific computing applications:
http://www.phoronix.com/scan.php?page=article&item=amd_fx8350_visherabdver2&num=1
If you don't need 8 cores, the 5800K does very well. If you do, the 8350 is top of the line, but the 8150 still does very well for non-gaming purposes. The difference is a substantial chunk of your budget, so you'll need to decide what you need.
My experience, and I think a lot of people more-or-less agree, is that Nvidia cards work better with Linux. I just think their (proprietary) drivers are better. You don't need anything crazy, but a modern GTX card is probably going to make your Ubuntu experience nicer. They've been out long enough now that they seem to have the kinks worked out. The 650Ti I'm suggesting is a nice low-midgrade card with dual DVI that would be adequate for casual gaming and more than adequate for anything else (uness you intend to do GPU computing, but then I'd expect to see a bigger budget).
The motherboard I'm going to recommend is slightly more expensive than you really need, but it has good support for Piledriver if you want the 8350, a great BIOS, and nice overclocking utilities. The difference between it and the cheaper models isn't huge.
I'm suggesting a 256GB SSD, but if you need to store large amounts of data, maybe go for the 128GB of the same model and spend the rest on HDD storage. I find I don't fill up my Linux boot very quickly myself, but your mileage may vary depending on what you're developing.
Case is up to you, but I think the 300R is amazing for the price. (Don't do the 200R, it's not up to Corsair's usual quality standards. But the 300R is solid midgrade quality - they just cut the bells and whistles from their midgrade cases.)
Aftermarket coolers are highly recommended, and essentially required for overclocking. The Evo and Plus are some of the best-known budget coolers, but others will do the job. Note that the Plus won't work on the Socket FM2 board.
8150 build:
PCPartPicker part list /
Price breakdown by merchant /
Benchmarks
CPU: AMD FX-8150 3.6GHz 8-Core Processor ($159.99 @ Amazon)
CPU Cooler: Cooler Master Hyper 212 Plus 76.8 CFM Sleeve Bearing CPU Cooler ($24.98 @ Outlet PC)
Motherboard: Asus M5A99X EVO R2.0 ATX AM3+ Motherboard ($119.99 @ Newegg)
Memory: Corsair Vengeance 8GB (2 x 4GB) DDR3-1600 Memory ($50.39 @ Amazon)
Storage: Samsung 840 Pro Series 256GB 2.5" Solid State Disk ($224.99 @ Microcenter)
Video Card: MSI GeForce GTX 650 Ti 1GB Video Card ($137.98 @ SuperBiiz)
Case: Corsair 300R ATX Mid Tower Case ($64.99 @ Microcenter)
Power Supply: Corsair Builder 430W 80 PLUS Certified ATX12V Power Supply ($24.99 @ Newegg)
Total: $808.30
(Prices include shipping, taxes, and discounts when available.)
(Generated by PCPartPicker 2013-03-01 14:33 EST-0500)
5800K build:
PCPartPicker part list /
Price breakdown by merchant /
Benchmarks
CPU: AMD A10-5800K 3.8GHz Quad-Core Processor ($130.98 @ Newegg)
CPU Cooler: Cooler Master Hyper 212 EVO 82.9 CFM Sleeve Bearing CPU Cooler ($32.98 @ Amazon)
Motherboard: ASRock FM2A85X Extreme6 ATX FM2 Motherboard ($112.86 @ Newegg)
Memory: Corsair Vengeance 8GB (2 x 4GB) DDR3-1600 Memory ($50.39 @ Amazon)
Storage: Samsung 840 Pro Series 256GB 2.5" Solid State Disk ($224.99 @ Microcenter)
Video Card: MSI GeForce GTX 650 Ti 1GB Video Card ($137.98 @ SuperBiiz)
Case: Corsair 300R ATX Mid Tower Case ($64.99 @ Microcenter)
Power Supply: Corsair Builder 430W 80 PLUS Certified ATX12V Power Supply ($24.99 @ Newegg)
Total: $760.16
(Prices include shipping, taxes, and discounts when available.)
(Generated by PCPartPicker 2013-03-01 14:38 EST-0500)