Any amount of RAM over 4GB will give better performance for most users, but you should always use all of your memory channels if possible. I recommend 6GB minimum for triple-channel systems and 8GB minimum for dual-channel systems.
Having more channels isn't pointless, it is simply expensive. We're actually getting some extreme-performance desktop CPUs introduced in six months or so with quad-channel memory controllers. When other components catch up, having more memory channels will become more beneficial. We're starting to get to that point already.