Mumble Link Positional Audio Help

[SpaceCroissant]

Hi there,

I'm implementing positional audio into my game using Mumble Link. I've added the init and update code from the wiki, each client has a unique ID and shares the same context ID, and the players positional data is being passed into the update.

The clients all have "Link to Game and Transmit Position" enabled in the Mumble settings.

On connection to the game server, the clients get "TestLink linked" in the mumble log, so the game and Mumble client are definitely seeing each other.

Despite this, the audio is at full volume all the time.

Does anyone know what I might be doing wrong?

Thanks in advance.

[hacst]

Hi. I assume PA works for you with other link enabled games (e.g. counterstrike or similar)? In that case there might be a problem with the data being received. We have a small tool called PAHelper which is meant to help diagnose problems with PA plugins. It can load PA plugins like mumble does and displays the data it receives from them.

You can either compile it from source yourself https://github.com/mumble-voip/mumble-pahelper/ or use my binary version from http://d0t.dbclan.de/dump/MumblePAHelper2013.exe .

If you place the binary into your mumble's plugin directory and run it it should automatically notice once your game connects and display you the data the link plugin receives.

[SpaceCroissant]

Thanks for your quick reply hacst.

I've placed MumblePAHelper.exe in the same directory as the .dll however it's not linking when running the game (I'm using CryEngine, so it's in Bin64 - the same place as CryGameSDK.dll - if you conveniently to have any experience with that!).

This is confusing because the Mumble client does link.

I assume I run MumblePAHelper instead of the client (I have tried with and without; neither worked)?

[hacst]

You can run it alongside or instead of the client. Both should be fine. Note that you have to place PAHelper into the _mumble_ plugin directory (e.g. C:\Program Files (x86)\Mumble\plugins) not in the games directory. It should display all the plugin dll's in the folder in it's "available plugins" list. The PA plugin used by your game will be link.dll (it handles link enabled games) and should link to your game after a short time. Should look something like this:

[SpaceCroissant]

Ahh that makes sense. Will give that a try after work tomorrow. Thanks very much!

[SpaceCroissant]

I've placed MumblePAHelper2013.exe (the one taken from your link to the already compiled version) in C:\Program Files (x86)\Mumble\plugins, however it doesn't pick up any of the plugins (including link.dll) when run. The 'Rescan' button doesn't appear to do anything.

I have tried reinstalling Mumble, but there was no change.

Any ideas?

[fwaggle]

It does the same thing for me. @hacst is this PAHelper built with the correct Visual Studio?

[hacst]

Built this with vs2013. Should work with the current snapshots (tested with 1.3.0-467 32bit locally).

[SpaceCroissant]

Any idea why it doesn't work for fwaggle or me?

[hacst]

I'll try to get hold of fwaggle in IRC and figure it out.

[SpaceCroissant]

That would be fantastic. Thanks!

[fwaggle]

OK I feel rather silly, but mine wasn't the exact same problem - mine was a crash in PAHelper2013. It can be fixed by not pointing it at plugins that are built with debug. If I build with nmake release, then put PAHelper in release/plugins, it seems to work correctly.

[hacst]

Or in other words: Please try with a recent x86 snapshot. It should work.

[SpaceCroissant]

Thanks! It worked with 1.3.0~467~ga897437~snapshot. I was using 1.2.8 before.

Here are the details in PA Helper.

http://i.imgur.com/cUZlrZd.jpg

I cheated a little for now by setting the up vector as (0,1,0). Viewing this here I've also noticed that the forward vector Z value is always 0 for some reason. I'm also using world position instead of local (which I believe is correct?).

Anything stand out as a big glaring mistake?

I'll have to look into it further when I'm back at my main PC after Christmas.

Thank you for your help so far. I really appreciate it!

[fwaggle]

I cheated a little for now by setting the up vector as (0,1,0).

 

I don't think that's really cheating, unless you can tilt your head in that game?

[SpaceCroissant]

I cheated a little for now by setting the up vector as (0,1,0).

 

I don't think that's really cheating, unless you can tilt your head in that game?

 

That's very true!

[hacst]

The front value seems to be out of whack. Avatar top and front by their nature are supposed to be always perpendicular to each other. So if top is pointing straight up there shouldn't be a Y component to the front vector. Maybe your internal coordinate system doesn't match the one mumble uses (see here)?

This should only affect the direction of the sound though. Attenuation should still work. Make sure that you scaled your coordinate units in pos to (approximately) one meter per unit and also make sure both clients trying to communicate are in the same coordinate space. Otherwise the attenuation effect might never kick in as it levels off after a certain distance or might not be noticeable as the changes are to small.

If that is all in order and contexts match, the plugin links and both sides have positional audio enabled in their output options it ought to work.

[SpaceCroissant]

Thanks for your help hacst.

Stupidly I hadn't checked that the coordinate systems were the same, and as it turns out, they're not. The game uses a system like this:

 

 z-axis                                  
  ^                               
  |                               
  |  y-axis                   
  |  /                         
  | /                           
  |/                             
  +---------------->   x-axis

 

I've changed it so that now for the position and front I'm giving the game's z coordinate to mumble's y, and the y coordinate to mumble's z.

This seems to almost work, but I think something is still slightly wrong. If two players facing opposite directions stand next to each other, the audio position is correct, however if they stand next to each other facing the same direction the position is reversed.

[hacst]

Ops. Sorry for the delay. I saw your message but couldn't respond right away and forgot to mark it unread again :oops:

Use PAHelper to make sure that if your front vector is displayed as (1,0,0) you actually increase your x position when moving in that direction. Same for (0,0,1) and y and so on as well as the negative vector directions. Maybe there's some mix-up in there.

Also turning clockwise on the spot should give you a repeating progression of (0,0,1), (1,0,0), (0,0,-1) and (-1,0,0) in the front vector (which one you start with ofc depends on the initial rotation). If the vector isn't a unit vector at any time that would be an issue too.

[SpaceCroissant]

Turning clockwise gives the correct progression of vectors, assuming when you say it should be a unit vector that it can be anywhere between 0 and 1 (and not 0 and 1 exclusively).

There does seem to be some issue with both X and Z increasing when at those 90 degree intervals however.

At (1, 0, 0) and (-1, 0, 0) X moves faster than Z, and at (0, 0, 1) and (0, 0, -1) Z moves faster than X. The slow one of each pair moves at 0.1 units for every 1.0 unit of the other.

What does this suggest?

[hacst]

Unit vector means the vector is of length one. So sqrt(x*x+y*y+z*z)==1 should hold true no matter your orientation. Top and Front are expected to be unit vectors.

In the picture below the green arrow would be the front vector with v_x and v_z being the corresponding x and z values of the vector (we are assuming y=0 here so only the character rotation matters):

http://d0t.dbclan.de/dump/mpics/vectors.png

So if you turn clockwise (without looking up or down) you swipe along that dotted unit circle with the coordinates changing accordingly.

As for the position: If x=1 and z=0 walking forward should make only your x position increase (assuming the ground is flat). x=-1 should decrease it. z=1 should make your z position increase and so on. If you have a mix (say x=0.5, z=0.86) walking forward should increase your z by 2*0.86=1.732 for every 1 x. x=y=0.7 (meaning the front vector would point smack in the middle between z and x) would see them increase evenly.

Hope that helps. Not sure I understood what you tried to say this time.

[SpaceCroissant]

Essentially, I can't find a point at all at which only X or only Z increases/decreases.

There also seems to be a variation of +/- 0.1 for the opposite value at each X=1/X=-1/Z=1/Z=-1. For example if X = 1, if I slowly rotate Z can equal all the way up to 0.1 or down to -0.1 before the X value changes.

This means, for example, that it's possible for X = -1.0 and Z = -0.6, which ends up with the vector as 0.88. No idea why this is happening - perhaps it's CryEngine or the CryEngine sample being weird.

I apologise if I'm not being very clear!

[hacst]

Sounds strange. Is there some public documentation on the data you are retrieving or the api you are using you could link me to? Also seeing the part of the code you use for integration (retrieval from the engine and filling of the vectors) would be helpful (hopefully shouldn't be more than a few lines ;) ).

[SpaceCroissant]

In the player update I'm simply doing:

Vec3 position = GetEntity()->GetWorldPos();
Vec3 forward = GetEntity()->GetForwardDir();
mumbleLink.updateMumble(position, forward);

http://docs.cryengine.com/display/CPPAPI/IEntity

As far as I can see these are valid functions to be using, and are used in other places throughout the player class.

When passed into updateMumble it's simply assigned to the various Mumble arrays (as I mentioned before, due to coordinate diferences I've had to swap Y and Z).

[hacst]

Definitely looks like the right functions to use. The documentation doesn't say so explicitly but at least this discussion suggests it should also be a unit vector (Everything else would be strange anyways). Swapping the coordinates on those vec3's is all you should have to do in that case.

With

I can't find a point at all at which only X or only Z increases/decreases.

you meant position x and z? Because when you rotate, front x and z will always change together. For the position you should definitely be able to find a direction (approx. (1,0,0), (0,0,1),... ) where walking forward pretty much (unlikely to point 100% precisely) only changes one of them.

As for

For example if X = 1, if I slowly rotate Z can equal all the way up to 0.1 or down to -0.1 before the X value changes. This means, for example, that it's possible for X = -1.0 and Z = -0.6, which ends up with the vector as 0.88.

. That's quite strange. These values shouldn't be possible for a unit vector. Not sure what you mean with 0.88. With those values (assuming y=0) the vector has a length of sqrt((-1)^2+(-0.6)^2)~=1.166.

If you directly display/output the vector in the game does it match up with what you see in PAHelper?