Even if you don't overclock you should look for a different heatsink with a quiet fan. Since you might want to overclock in the future I'd suggest a Cooler Master Evo. It comes with a Cooler Master Blade Master 120mm fan. I think it's very quiet when not gaming and I barely notice it from close. http://pcpartpicker.com/part/cooler-master-cpu-cooler-r...
Just about any aftermarket cooler will out-class the acoustics of your stock cooler.
If quiet is the name of the game, go for the Arctic Cooling i30 Freezer. It's on sale for $30, and does a pretty good job. Even when the fan is at full tilt, the sound is pretty graceful. Thermalright also makes really quiet options, but the options you'd want are out of your price range.
Xigmatek Gaia is another good choice; can't go wrong for $25.
The Cooler Master Hyper 212 EVO is usually a good stand-by that's hard to go wrong with, but I see that Newegg put the price back up to $40.
Is the Hyper 212 EVO the quietest in that price range? I'm not too obsessed with performance, given that I don't OC. Just want silence.
I might, just might, OC later on. I have a pretty basic Mobo --- a msi g41 mate, but I think I can do a simple modest OC.
hard question to answer. If you aren't overclocking the CM Evo then it will run very quietly cuz it isn't being pushed. You could also get a CM 212+, not CM 212 Evo. It's cheaper and almost the same fan.
Like envy14type said, it is a hard question to answer. Honestly, when left to their own devices, most fans probably won't go above 1100-1200 RPM when at full load with stock speeds. You would have to have a discerning ear to really tell the difference, although fans to carry their own tone and character. (And, although it costs more, switching fans is always an option! Markwp mentioned this earlier.)