The main things to consider for noise are:
- Power Consumption
- CPU Heatsink/Fan
- GPU Heatsink/Fan
- PSU Fan
- Hard Drives/Vibrations
- Dampening
Higher power consumption = more heat. More heat = more noise because the fans have to work harder to cool. Keep the power consumption as low as you can and it is MUCH easier to create a quiet system.
The CPU is by far the easiest one to make quiet, just slap on a decent heatsink with a PWM fan that has a minimum speed of less than around 700RPM and you're fine. The Coolermaster Hyper 212 Evo or anything from Noctua is good.
The GPU is probably the hardest one because high end GPU's create a crapload of heat and don't have all that much space for a decent heatsink or large fans. Not only that, but GPU manufacturers almost never give the rated RPM of the fans they use. If you're comfortable with an aftermarket cooler, that's the best way to go. If not, I find ASUS cards are generally pretty quiet. MSI's higher end cards are usually quiet too. Once you get any card, download something like MSI afterburner and lower the fan speed.
The PSU shouldn't be too difficult because they are most likely to give some kind of noise rating. The best way would be to get a PSU with a hybrid fan like the Seasonic X series. That means the fan only spins when it needs to, so it is passive most of the time but has a fan just in case. More efficient PSU's are generally more quiet because they usually have less heat to contend with, look for the 80 PLUS gold rating if possible.
Try to avoid a HDD if possible, if you need mass storage though I would buy a slower spinning 5400RPM hard drive. You shouldn't notice the speed drop from a storage drive since your OS/programs will be loading from your SSD. 2.5'' (laptop) hard drives are quieter as well, they are slower and more expensive but again, the speed different shouldn't be that noticeable on a mass storage drive. In terms of vibrations, make sure to use rubber/silicone mounts for mounting hard drives and fans and use hard drives and fans with lower RPM's for lower noise. A nice solid, closed style case that doesn't have too many vents helps too.
You can use sound dampening foam on the sides of the case as well, that might heat it up a bit but tends to kill some of the more annoying high frequency noise.
Here are some suitable components for a quiet system:
Intel Xeon E3-1230v2 (similar performance to a quad core i7 but is cheaper with a TDP of only 69W. You can't overclock on it and you will need a GPU though.)
Coolermaster Hyper 212 Evo (Cheap and effective cooler with an extremely quiet fan. It goes down to a minimum of 600RPM and speeds up automatically if the CPU gets hot.)
ASUS GTX 660Ti (ASUS's cards tend to be pretty quiet and the GTX 660Ti is 150W which is pretty good for a decent graphics card.
Seasonic X-560 or any X series PSU (80 Plus gold, fully modular and the hybrid fan that only spins when it needs to.)
Western Digital Green Power WD20NPVT 2TB 2.5in HDD (Any slow spinning, 2.5'' hard drive will be very quiet but this one is sold by quietpc as well so that's always a good sign.)
Fractal Design Define R4 Case (Decent sized case with adequate cooling and lots of anti noise/vibration features.)