If you use WPA/WPA2 security and a high quality key/password (long, random, and based on a large character set, such as the ones available at http://grc.com/password), no one is going to get into your wireless network. No how, no way, not by brute force anyway.
IOW, the key (no pun intended) is to never use a key that someone can guess! But obviously if you use a weak key/password (some ppl do for the convenience), then obviously you’re far more vulnerable. You could add MAC filtering as a secondary protection, but MAC filtering is easily circumvented through ARP poisoning (at least by anyone worth worrying about).
Bottomline, use WPA/WPA2 w/ a high-quality key/password and you won’t have to worry about it.