Mumble affects internet connection

[swiftfinch]

I've been trying to use Mumble for about a month now, and it's been sporadic at best. It works fine, everyone can hear me and I can hear them, but sometimes running Mumble messes with my internet connection, bringing it to a screeching halt. It's mainly a problem because I'm trying to use it with my competitive TF2 team, but because it makes me lag so much I can't. It's weird though, because it only happens sometimes. Last night during a match it happened, but earlier in the day it was working fine and my connection was great.

I've tried tweaking the settings a lot to decrease my sound quality, turned off the overlay, and set the priority in task manager to low, but nothing seems to do the trick. Is there anything else I can do? I'm running version 1.2.2 on a Windows XP partition on a Macbook Pro.

[swiftfinch]

I think I've determined that the more users are in the server, that's when I lag. It seems to start at around 5 or more users. With my team there will be 9 players in the server during the game, so are there any solutions to this?