How does a internet phone affect online gaming?

Internet phones (voip) is starting to get very popular.
Skype, Vonage and all of the cable companies are vying to get your phone service dollars to change over.

My understanding of Internet phones (voip) is that it uses Udp for this service.
I have yet to see a test defining how much your gaming bandwidth may suffer if gaming was going on at the same time as a voip call.

I am always aware of programs that may interfere with my latency and throughput while gaming. Since Internet phones uses Udp (my understanding) as do most if not all of the online games, how much does one affect the other.

Is it worth the trouble to change carriers?
Is the interference negligible?

Or should I not go the way of Voip because you can be sure that when a great frag is in my crosshairs and the kill is mine the voip phone will ring and for some reason I find myself respawning and picking myself up while silently cursing at the idiot that chose to call me at such a crucial moment.

And to think that only friends have my number