Develop a BOT to record sessions - protocol documentation

Ice/DBus, Web-Interfaces, Management tools

Develop a BOT to record sessions - protocol documentation

Postby roberthendrickx » Tue May 17, 2011 8:01 am

Hello,

I'm interested in a BOT that could record automatically the conversations. The "record" function on the client is not sufficient because I want to record all teams at once (basically, recording a specific channel that ears everyone).

It should be possible with a dedicated client, but it's hard to manage (one additional computer is needed as it's not easy to start 2 clients on the same machine)

I could try to develop it myself, but I can't find any documentation on the actual protocol between the client and the server... The protobuf "mumble.proto" is not documented at all, and I have no idea of the process of authentication, how flows are received, how the negotiations occurs, ...

I found 2 BOT that do part of that (EVE and smoak-mmb) that do part of it, but reverse engineering of their own reverse engineering is not the best way to do some good code...

Thanks
roberthendrickx
 
Posts: 15
Joined: Thu Mar 10, 2011 1:07 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby hacst » Tue May 17, 2011 9:12 am

We have some limited documentation for the protocol in our repository. If you want to develop a bot libmumbleclient might be your best bet. There are also some protocol implementations in python in various stages of completion (rather less complete the last time I looked ;-) ).
hacst
Team member
Team member
 
Posts: 339
Joined: Wed Sep 23, 2009 4:28 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby roberthendrickx » Tue May 17, 2011 11:47 am

I hoped to do it in python, but I could also wrap libmumbleclient in a cython extension... thanks for the hint

It could be interesting to put some links in the "3rd party" part of the wiki... I only found 2 pythons projects (smoak-mmb seems the most advanced protocol implementation, but only for the parts they need)...

If you have some other links, I'm interested :D

Anyway, thanks
roberthendrickx
 
Posts: 15
Joined: Thu Mar 10, 2011 1:07 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby hacst » Tue May 17, 2011 12:45 pm

Interesting. I didn't even know of mmb before ;-)

I don't really have any links I could give you from the top of my head (there's mumbo in our mumble-scripts repository but that doesn't do audio yet).

It would be great to have a relatively pure python implementation (I guess noone wants to do protobuf and encoding/decoding in pure python ;-) ) to play with.

Anyways. Be sure to report back if you find anything interesting :lol:
hacst
Team member
Team member
 
Posts: 339
Joined: Wed Sep 23, 2009 4:28 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby roberthendrickx » Tue May 31, 2011 9:32 pm

OK, It begin to work... I'm able to connect on the server and receive/decode sound on TCP... It's not yet a generic library, but's that already something...

I still havea questions : How does the "sequence" field works in the protocol ?

I try to figure out how to put together the different user's streams in a single sound file, but for that I need to keep the different streams synchronized, and as there is no timestamp with the audio frames, I hope to be able to use the sequence to check when there is a continuous emission and when there is a "gap"

Thanks
roberthendrickx
 
Posts: 15
Joined: Thu Mar 10, 2011 1:07 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby roberthendrickx » Wed Jun 01, 2011 6:22 am

Another question... How does the codec negotiation work ?

When authenticating, I have to send the list of the CELT bitstreams versions I support, but how is it "negociated"?

I receive a "CodecVersion" message, that include all the time the bitstreams versions for CELT 0.7 and CELT 0.11 even if I only support the 0.11 in my authenticate message, only the "prefer alpha" bool changes...

Basically, how do I know I must use Speex (which is mandatory I understand) CELT alpha (0.7 ?) or CELT beta (0.11) for outgoing audio ?

(by the way, this alpha-beta thing does not seems very extensible to support new CELT versions... why not a list of supported codecs that would only include the ones that are supported by all the connected clients, like in the authenticate message with for each a preference...)

thanks
roberthendrickx
 
Posts: 15
Joined: Thu Mar 10, 2011 1:07 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby pcgod » Wed Jun 01, 2011 1:49 pm

iirc the sequence number in an audio packet is the number of the first frame in that packet (i.e. each frame is 10ms and it will increase by 2 for each packet with audio per packet set to 20ms) and the counter resets after 500 "silent frames" (i.e. if you're not talking for 5 seconds).

about the codec negotiation: the client sends all supported versions in the authenticate message. The server will tell you which codec you should use using the preferAlpha field. The client always sends audio data encoded with the version depending on the value of preferAlpha, if it's true it will use the version in alpha, if false it it uses the version in beta. CELT 0.7 is mandatory, Speex and CELT 0.11 are optional (for now) and any client which supports only CELT 0.7 will force all other clients to use CELT 0.7.
User avatar
pcgod
Team member
Team member
 
Posts: 68
Joined: Tue Nov 24, 2009 12:24 am

Re: Develop a BOT to record sessions - protocol documentation

Postby roberthendrickx » Wed Jun 01, 2011 3:38 pm

Thanks

pcgod wrote:iirc the sequence number in an audio packet is the number of the first frame in that packet (i.e. each frame is 10ms and it will increase by 2 for each packet with audio per packet set to 20ms) and the counter resets after 500 "silent frames" (i.e. if you're not talking for 5 seconds).


You mean that the counter is increasing by 1 for each 10ms even if no frames are sent ? I understand better what I saw during my debuggings. This also means you cannot know if a packet was lost...
I also suppose this counter is specific to a session and not shared...

pcgod wrote:about the codec negotiation: the client sends all supported versions in the authenticate message. The server will tell you which codec you should use using the preferAlpha field. The client always sends audio data encoded with the version depending on the value of preferAlpha, if it's true it will use the version in alpha, if false it it uses the version in beta. CELT 0.7 is mandatory, Speex and CELT 0.11 are optional (for now) and any client which supports only CELT 0.7 will force all other clients to use CELT 0.7.


OK, it's clear now

Thanks for the answers.
roberthendrickx
 
Posts: 15
Joined: Thu Mar 10, 2011 1:07 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby hacst » Wed Jun 01, 2011 6:58 pm

The counter is set by the client and distributed to all listeners. It is specific to that client and, as pcgod says, will reset after a short time to keep it small.
hacst
Team member
Team member
 
Posts: 339
Joined: Wed Sep 23, 2009 4:28 pm

Re: Develop a BOT to record sessions - protocol documentation

Postby pcgod » Thu Jun 02, 2011 6:28 pm

roberthendrickx wrote:You mean that the counter is increasing by 1 for each 10ms even if no frames are sent ? I understand better what I saw during my debuggings. This also means you cannot know if a packet was lost...
I also suppose this counter is specific to a session and not shared...

The counter is per user and is increased for every audio frame sent to the server. It won't increase if you're not talking. A lost packet should result in missing numbers, i.e. someone with audio per packet 60ms should send packets with the sequence numbers 6, 12, 18, 24 etc. (audio per packet 20ms -> 2, 4, 6, 8 etc.) and there will be a gap if the server didn't receive one of those packets.
User avatar
pcgod
Team member
Team member
 
Posts: 68
Joined: Tue Nov 24, 2009 12:24 am

Next

Return to Scripting

Who is online

Users browsing this forum: No registered users and 1 guest

cron