It's not ideal. As you stated, you want to have matched pairs (or quads) to insure dual channel and all that.
But really, if you're getting your proper 667/800/1066MHz speed, and you're trying to stay on a super tight budget, then you're fine. Dual channel vs single only gives you 7-10% performance bump in real world experience (if you're lucky).
If you have the money, do it right. But if you're on a super tight budget, it's working stable for you, don't fret. It's fine.
Complete tangent; make sure you're using a 64bit OS.