large latency during long lasting sessions

[tufuwe]

Hi,

we experience after different amounts of time (30 minutes sometimes after 1.5 hours) very long delays/latency (for example 5 seconds) between the members of the group.

Is this problem known?

The network is fine and after a disconnect and reconnect everything is fine again.

This happens between all the server and client version number combinations we tested, so 1.2.4 and 1.2.3.

Any ideas?

Sound like a minor problem, but if you experienced this once and don't know of it, this 5 seconds delay can really cause nasty conversations and people get really angry with each other....

A disconnect would be much better, than this delay!

Thanks

[kissaki]

No, this is not a known issue.

I’d definitely blame it on some part of networking - as it is for several people, on the servers (or other resource limits on that end).

Once you reach your delay state, is it a 5s delay once, and then it works again? Or you only hear something like a 100 ms sound from the others every 5 s?

Does this problem persist if you do not reconnect?

Does the problem occur for everyone at the same time?

Does the problem go away for everyone once one person reconnects? Or only for the person reconnecting? If three people have the issue, and two reconnect, does the third still have the problem?

[tufuwe]

Hello,

once the delay is there, it doesn't vanish anymore. It is persistent for the whole session and really (up to) 5 seconds delay, but perfectly clean sound, no distortions.

When we reconnect, everything is fine again.

Yes, it is one poor guy in the group who has the delay and the others are fine!

If the one culprit reconnects everything is fine again for everyone, no need that the others have to reconnect.

We never tried to keep the delayed person in the session and let the good ones reconnect....

I think that we reduced (or even solved) the issue by using "voice activity" instead of "continuous".

If continuous is selected, there has to be an algorithm in mumble which prevents a queuing, like the comma characters on a 10b8b-serial line. If this is not implemented then this could be an explanation of the observed behaviour.

Thanks!

[kissaki]

If it’s just an issue with one person it should definitely be a users system / client issue.

 

If continuous is selected, there has to be an algorithm in mumble which prevents a queuing, like the comma characters on a 10b8b-serial line. If this is not implemented then this could be an explanation of the observed behaviour.

Not sure what you mean there.

[tufuwe]

Hello,

the issue is not user(-machine) and client version dependent. It happens with all contacts, also when only two persons are talking for long periods in "continuous mode".

Concerning the 10b8b-serial line:

The thing I wanted to mention is: If a data stream is sent in real time for a long time to a receiver, there will be always a difference in the system clock of the receiver and sender. If for example the sender-clock is faster, then it is sending more packets to the receiver than this can handle, as it unpacks them (and the case of mumble plays them on the sound-card) with a slower clock. So, over some time more and more packets will accumulate in the receive buffer.

Therefore, all serial transmission lines implement a method to insert from time to time some gaps (comma characters) which assure that the receiver doesn't drown in data over time. Serial links with no buffers will otherwise just lose data. With buffers the delay will increase over time...

I was just brainstorming that this method has to be implemented when you use the "continuous mode" and talk over hours. I assume that you did this. It just came to my mind, as everything is fine if we switch to "voice activity" mode.

Thanks!

[hacst]

We don't perform any audio skew compensation in our processing chain afaik so it would be possible for packets to queue up in continuous transmission mode. At some point the jitter buffer should discard packages which are too old though. I'd have to look at that code to verify the specific behavior.

My guess is it just didn't cause any noticeable issues before and hence wasn't worth the effort. What kind of audio device is on the other end to cause such a drift over such a short period of time? At 1h30m a delay of 5s would mean a drift of nearly 1ms per second. That sounds quite high to me.

In any case. Resolving audio skew issues without hearable artifacts in playback isn't all that straight-forward unfortunately. You'd have to analyse queue depth in the jitter buffer and do dynamic resampling of the incoming data based on that to prevent under/overflows caused by skew.