Positional Audio for Borderlands.

[rawnar]

Borderlands is a first person shooter with RPG elements that can be played with 4 people in co-op. I think it would be nice to have a positional audio plugin for this game. The game used the FMOD sound system as the game directory contains a fmodex.dll file.

I started with reading the following two part of the mumble site.

http://mumble.sourceforge.net/Pluginguide

http://mumble.sourceforge.net/HackPositionalAudio

For Borderlands one can use the following to aid his search for the memory addresses that hold the position and heading of the avatar and the camera.

a) Switching between window mode and full screen mode.

Will in the game press the following key combination ALT+ENTER.

b) Getting the in game position.

For this one has to edit a configuration files found in:

Documents\My Games\Borderlands\WillowGame\Config\

Open the file WillowInput.ini and add the following line after the existing bindings in the section [WillowGame.WillowPlayerInput]

Bindings=(Name="F2",Command="dbm_ToggleShowDebug",Control=False,Shift=False,Alt=False)

If there are on binding in this section you need to go into the game in change one of the keyboard input settings and exit the game. Otherwise you will not be able to move in the game. Pressing "F2" will show the debug information which also contains the position of your avatar. The inventory needs to be opened at least once before this command works.

c) switch between first and third person view.

This is handy to distinguish between the avatar and the camera. In the same file and section as in the previous point add the lines

Bindings=(Name="F3",Command="Camera ThirdPerson",Control=False,Shift=False,Alt=False,LeftTrigger=False,RightTrigger=False,bIgnoreCtrl=False,bIgnoreShift=False,bIgnoreAlt=False)
Bindings=(Name="F4",Command="Camera FirstPerson",Control=False,Shift=False,Alt=False,LeftTrigger=False,RightTrigger=False,bIgnoreCtrl=False,bIgnoreShift=False,bIgnoreAlt=False)

Pressing "F3" will put you in third person view and pressing "F4" will put you in first person view.

Using Cheat engine I found a part of the memory that holds the camera position and heading. I also found a vector holding the avatars position, but without heading information. For the state value I have problems finding a good memory address, as Cheat Engine needs to go through the 1GB of allocated memory byte by byte. The coordinate system used seems to be left handed and the X-axis is point to the North and the Y-axis is point to the East, resulting in a Z-axis pointing up. The units, as it is based on the Unreal Engine also know as Unreal Units, seem to be in centimetres so the results need to be divided by 100.0f. Unfortunately these addresses are not static so a pointer search had to be performed. After doing several re-scans of the pointers, while jumping to different maps and restarting the game I found three consistent pointers for the camera.For the avatars position the results was a large amount of pointers, but some of them only needed two offsets.

I have some question how to proceed with finalizing the plugin.

Any idea how to find a good state value?

How do I pick the correct pointer? The pointer with the smallest amount of offsets, or are there any other handy criteria I can work with?

For the camera pointers, the second and the third offset of these pointers are the same, so the three different pointers of the first level point to the same structure, right?

Why is the heading of the avatar needed? In other words, how is mumble using the information of the avatars heading to change the positional audio experience? Will my voice sound louder to my team made when I look at him in the game?

The game is a FPS in essence but there are moments, like when you are driving a car, that the game switches to third person view. Would you advice me to use the in-game position of the avatar or the camera position for the avatar_pos, avatar_front, avatar_top vectors in the plugin?

[hacst]

First things first: Welcome to our forums.

1) Any idea how to find a good state value?

State values are always a bit tricky. It takes quite a number of tries to find a value that works reliably. Especially since you don't know what values to expect so all you can do in the beginning are scans for changes. My recommendation would be to limit the scans to a certain memory region (module) that you consider promising, that speeds up scanning quite a bit.

2) How do I pick the correct pointer? The pointer with the smallest amount of offsets, or are there any other handy criteria I can work with?

Well. For pointers it is really just picking one that works. Usually the ones with the least number of levels will be the more reliable ones (and are also easier to work with).

3) For the camera pointers, the second and the third offset of these pointers are the same, so the three different pointers of the first level point to the same structure, right?

For a first person shooter we simply set the avatar position to the camera position. The differentiation between them is meant to be used in 3rd person game where your camera (read ears) aren't where your avatar (read voice) is. So if you have the necessary information about the camera there's not really anything to gain from knowing the avatar values.

3) Why is the heading of the avatar needed? In other words, how is mumble using the information of the avatars heading to change the positional audio experience? Will my voice sound louder to my team made when I look at him in the game?

As far as I know, I might be wrong, this additional information is not yet used by Mumble. Afaik only the 3d orientation of your ears and the position of the audio source is considered. This might change in the future though.

4) The game is a FPS in essence but there are moments, like when you are driving a car, that the game switches to third person view. Would you advice me to use the in-game position of the avatar or the camera position for the avatar_pos, avatar_front, avatar_top vectors in the plugin?

Hm. Considering there will might be more players in that car talking to each other it might actually be counter productive to have avatar != camera in that case. But that's really up to you. Do what you think suits the game.

I hope this helps. If you have any additional questions either post them here or visit us in IRC (#mumble on freenode). Look out for Snares. He did most of the current plugins.

[rawnar]

Thanks for the reply.

For simplicity I will stick with the camera only option.

For the state value I have found a static address that goes from 0 to 1 when entering a map. Unfortunately, I was doing the memory searches in off-line mode, as I expected memory allocation not to change when I would go on-line. But checking it yesterday I found that the pointer to the camera structure does not hold and the state value already goes to one before a map is loaded. So I am back to square one.

If I have finished the plugin, were is the best place to post it? As attachments (src+dll) to one of the post?

[kruegerfreddy]

@rawnar thank you for the contribution to the comunity. I guess the best way to post your source would be new ticket in the patches section if the sourceforge page of the mumble project. Just use the Add New Link.

I will help you to update theSupported Games page in the Wiki if it gets accepted.

@dd0t: would you be ok if i try to add some of your answers to the (hack postional Audio) page of the wiki?

[kissaki]

@dd0t: would you be ok if i try to add some of your answers to the (hack postional Audio) page of the wiki?

yes, go for it

[hacst]

@dd0t: would you be ok if i try to add some of your answers to the (hack postional Audio) page of the wiki?

Sure. The more ppl. can gather from the wiki the less I have to repeat myself ;-)

@rawnar: As kruegerfreddy said the patches section would be the place to put it. What would be important for getting it into the official distribution is whether the game still receives updates and, if it is still updated, whether you are willing to provide updates in the future (not in all eternity but there's no use for us to distribute a plugin that is outdated a week after publishing it with noone available to update it ;-) ). Thanks for your effort.

[rawnar]

Finished finding the pointers to the position,front vector and top vector of the camera. Using the advised level of three during a pointer search never gave me pointers that would stick after restarting the game. Using a level four search solved that problem. Now I have a set of 16 pointers that point to the right address even after reboot. I am collaborating with a friend to find out if these pointers also work on his rig.

The state value is still giving me problems. A static address that was jumping from 0 to 1 when entering a map, does not do that any more when a second player is on the server. I will have a look at the memory region around the static address used by the UT3 plug-in. Both games are based on the UE3 and maybe the static addresses are similar. Another option will be not the check the state value and just go for it.

I have also found an dynamic address that holds the public ip:port combination of the server you are playing on. This could be used for the context string. In all the other plug-ins I see that a length of 128 is used. Is there a standard format for the context string and does it need to be a fix length? Maybe someone can add some info about the context and identity strings on the wiki page.

[hacst]

Nice to see you are making progress.

No, there's no limitation to the length of the context string. I guess the code you are referring to is reading a chunk of memory from the game in which case you have to settle for the amount to read. If you are reading strings from another process you'll usually have to guess a sane maximum size (for example an IP:PORT, HOST:PORT combination is very unlikely to be longer than 128 characters).

[rawnar]

I have tried my plugin with a friend, but it was not working. For debugging, it would be nice to see some information that is send to the murmur server. Is there a dialogue I can open in Mumble, so I can see the optional information that is send to the murmur server?

If such a funtion is not implemented in mumble, maybe I can use Cheat Engine for monitoring Mumble. What memory region is Mumble using to store the positional audio information? A pointer to the position data will also do.

[hacst]

Unfortunately we do not have any plugin debug facilities in place at this point in time. So the easiest thing to do is to add some debug output to your plugin in places you want to analyse. Not very elegant but it works ;-)

[rawnar]

You mean adding some lines to the plugin code. Which will look something like:

cout << "BP position:" << position[0] << position[1] << position[2] << endl;

And run Mumble from the command line.

Why didn't I think about that? Probably because I don't see MS windows as a console orientated OS.;-) Most of the time I use Linux as my OS. MS windows is used for gaming and gaming only.:-) Within Linux I use the console regularly, but not in MS windows.

update:

The simple iostream example I gave above doesn't seem to work. I have added add the beginning of the plugin code two lines

#include <iostream>
using namespace std;

I changed the if statement add the beginning of the trylock function to

	DWORD pid=getProcess(L"Borderlands.exe");
if (!pid) {
	cout << "BP: no process found" << endl;
	return false;
}
cout << "BP: process found" << endl;

Then I started mumble by going into the command line with 'cmd' and execute mumble.exe. I logged into a empty server and waited for some time, but nothing happened. Also starting Borderlands didn't give me any output to the command line from mumble. So there are some possibilities here that can lead to this result.

1 - I have made an error in writing the plug-in.

2 - Mumble is started in the background and catches all the output to the command line.

3 - Mumble is smart and will only start probing with the plug-ins when at least 2 persons are in one channel.

If it is the first one, I will check, re-check and triple check my code. If it is the second one, how can I make a simple QT dialog were I can write some strings to? If it is the last one, I will start two instances of mumble on my computer and login to the same server. I have noticed you can start mumble multiple times by adding the flag -m to the command. By the way, how often are the trylock function of the plugins called?

P.S.: I have posted more or less the same question add the irc channel, but was unable to wait for the answer. Sorry for that.

[hacst]

Ah. Sorry about that. Should've warned you: Stdout doesn't work in the plugin (just gets sinked /dev/null style).

I'd recommend either using a plain old text file to pipe your debug output to or use Windows debug functions like OutputDebugString (http://msdn.microsoft.com/en-us/library/aa363362(VS.85).aspx) and attach a viewer for them like DebugView.

[rawnar]

Okay I can now monitor the debug information. Be aware that you need a wstring to go into the outputDebugString. A lot of examples on the web use a standard string, but using standard strings gave me some error outputs during compilation. I'm running MS Windows 7 64bit, so maybe this is causing the problem.

Unfortunately now that I have the debug output I see that the pointers I am using are not working. Can I just copy past the pointer offset from CE to the source code of the plugin, or do I need to perform some algebra on it before implementing it?

update:

Found the solution. From the plugin guide page on the mumble site I suspected that 'Borderlands.exe+01BF3918' means that the first address of the pointer is 0x01BF3918. Unfortunately the exacutable also has an offset that is not zero. The executable has a offset of 00400000 giving me the static address that holds the first pointer to be 0x01FF3918. Now lets hope the the executable offset is the same for everyone.

second update:

Used the function that finds the address of a module, to find the address of the main executable. So, now the code should also work on computers were the executable has a different offset then 00400000.

[rawnar]

Okay so the part of the position and direction works for the plugin. In principle you would get directional audio with Borderlands when using this plug-in.

Unfortunately I have not found a static address that is 100% working as a state value. I could remove the state value checking, but I am not sure if that is sensible.

The context string also has some problems. For ones, only when you are logged in the memory that is holding the string is allocated. For LAN games you don't need to login so no string can be found. Meaning when using this string as a context string it will not work on LAN games. And I don't know if the string I am getting is correct. Trying to show it with outputDebugString gives me row of question marks.

Still I have to check the working of the plugin on other computers. So if anyone has Borderlands are you willing to check the plugin I have written? I can provide the source code or the dynamic library.

[rawnar]

Me again.

The pointer series that gives me the location of the position vector in the memory is not working robustly. Is there a setting in pointer search of CE 5.6 that can help me find a good one? At the moment I search for the pointers with the default settings and a level of depth of 4. Then I play around in the game and do a pointer rescan. I repeat this until I am left with a dozen of pointers. Pull these pointers into the main window of CE and restart the game with different settings. This will show again some pointers as unusable. As a final check I restart the computer and check the pointers again.

The found pointer seem to be quite consistent, but now and then they become completely useless. Come to think of it the moment that the pointers became useless is around the time my windows applies an update. Also having multiple programs running before starting Borderlands also seems to effect the pointers. If someone has an idea how I can find better pointers, please post.

I have been reading into assembly code, so glues about how to find the memory that holds the position vector by looking at the assembly code in CE will also help.

[rawnar]

Yesterday, I finally had the option to test my plugin with a friend. It worked, but the probe interval of the position is quite large. This could be due to the fact that we were playing with a version of the plugin that was outputting information to debugview. Is there a setting in Mumble that will change the memory probing interval?

I will do a final search for a more stable state value after which I will put the final plugin in the patch section. I will also include the debug version of the plugin. Could be useful for other plugin developers.

[rawnar]

Plugin has been added to the patch section. Have fun with it.

[hacst]

Great news. Thanks for your contribution.

(Sorry for not responding to your posts. Somehow my notifications for this whole forum got switched off :evil: )

[rawnar]

I check the borderlands plugin this morning to see if it worked and if it would be downloaded automatically. It was downloaded together with some other plugins that were updated. Only when I started playing the game Mumble crashed. The crash seems to happen when mumble gives the 'borderlands linked' output to the console. This output is printed at the moment mumble get true returned from the trylock function. This only happens when the fetch function returns true. I have done the following to find the bug:

Compiled the git version of the plugin on my own computer. -> mumble crashes

Compiled the my own version of the plugin, this one worked last week. -> mumble crashes

Compiled the debug version of my own plugin, this one also worked last week. -> mumble crashes

Disabling all the other plugins from the config dialog. -> mumble crashes

As you can see I have not found the problem. Could the problem be in the other plugins that were updated? I can not remove the other plugins from the AppData/Roaming/Mumble/Plugin dir as they will be automatic download on mumble start. They can also not be removed after mumble is started as mumble locks the files. Can I switch of automatic updating of the plugins or do I just need to disconnect my network?

There is also a core dump in the mumble roaming dir. Is there a program with which I can examine the core dump? Is there also a way to connect a debugger to the Mumble program, so I can backtrace the code when mumble crashes? Or do I need to recompile mumble to get any debug information?

I will also check this evening if other games now have the same problem.

[hacst]

We have a server with debug symbols for all ours releases so if you can provide us with the crashdump (or you could submit a bugreport with our reporter tool and tell what you wrote in there so we can find yours) we can check out where it crashed.

If you want to examine the dump yourself you can do so with , for example, the Visual Studio debugger.

[rawnar]

Send a crash report. I forgot the exact text but it started with something like: "It crashed while mumble was linking to borderlands."

But I found from the core dump that it had to do with some string handling. I pinpointed it the the following line of the plugin.

 

context = std::string(strstr(ccontext,"bderlandspc"));

It seems to be that the string I am monitoring can have some strange characters (I think they are cr+lf). This causes the strstr function to give a faulty string to the constructor (). When I was testing the plugin last week these strange characters were not occurring, because I was already in a online game. Hopefully I will have a solid patch ready by this evening.

BTW UT3 plugin just worked fine.

[rawnar]

Patch has been added to the tracker. Hopefully it will be pulled quickly into the source tree. That way less people will be annoyed by my mistake.

So, until the patch is pushed and the new plugin is available through the auto update, uncheck the Borderlands plugin when playing borderlands or first get into an online game of Borderlands and then start mumble.

Update: I just saw on the tracker that the patch has been pushed. Thing are looking good.

Update 2: Check the autodownloaded plugin and now is says "Borderlands linked." in the console. :D Happy looting on Pandora.

[rawnar]

Here we go again. The pointers I was using to get to the memory location that holds the camera information was not always stable on my system. The last few times I played Borderlands it even got worse. So, I have started CE again in search for a new memory location that holds the camera information.

The old memory location was also in a strange place, surrounded by string that were overwritten by other information. I found some other options that I had ignored the previous time, because the postion was not closely followed by the front,top vectors. From the assemble code of the fmodex.dll (sound system standardly used by UE3) I found the offsets between the position, front and top vector. Using this info I found the stucture that fmodex.dll uses. I also know the offset of the position vector to the start of the structure, this is 0x9200.(Normally CE doesn't search for such big offsets.) So I stated that the last offset should be 9200 and I ended up with only one pointer that had only one offset being 0x9200, Bingo. :D I have tested my new pointers succesfully and submitted a patch.

For other programmers of plugins for games that uses the FMOD sound system here are the structure offset for all the vectors.

position vector: 0x9200 front vector: 0x9248 top vector: 0x9230

The vectors themselfs are also directly followed by a copy of it. Probably used for checking if the camera has changed after previous call to the function. Happy hunting. ;)

[hacst]

Sounds great :P

[kruegerfreddy]

Hi,

I bought the borderlands game to try this, but the plugin dosent even connect to the game.

Seams the plugin dosnt reconize the .exe file in my memory.

Im using mumble 2010-07-18-1111-7208fc (1.2.3 Developer Snapshot) on a vista system.

connecting to a 1.2.2 linux mummur.

I use the 1.3 German censored version of borderlands.

Can you tell what version of Borderlands is been used to test the plugin?