Well.. if microseconds matter than much...
Seriously, though - I strongly doubt this -
{increased latency due to large memory footprint to search} - is anything that could possibly be noticed by a human being. Though if the idea of it sits in your head and festers
(...happens to me all the time), what you can do is monitor your memory usage with the current (6GB) of RAM you have.
For example, in Win 7 you can use the Resource Monitor - Open it to the memory tab and leave it where you can see it. {A dual monitor setup rules for this} The Grey (system), Green (in Use), and Orange (Modified) areas are of concern, since this represents the memory that's actively being used and would have to be paged or written to disc before it could be made available for something else. The Blue (standby/cached) and Light Blue (Free) areas are available to applications at all times, so they're of no concern. Very simple - Load up your comp with whatever you do that represents the highest levels of stress, then maybe open a few more programs just for grins. Then check the resource monitor - Do you see a lot of Grey/Green/Orange? If yes, then you may want to consider adding more ram. If no, then you don't need any more than you already have.
Hope that helps.
{edit} Warmon's words about caching are also on point. The benefit of being able to cache more objects in active memory outweighs the latency cost of having a large amount of RAM by several orders of magnitude.