Unity Spatial Audio Endeavors

Hello! This was my research project for Engineering III class. We had complete freedom to explore a topic of our choosing, and so I chose to experiment a little with Spatial Audio in Unity.

My motivation came largely from trying to simulate real-life acoustics into a virtual space. Right now, acoustics in games are hand-crafted by sound designers who know what a space is supposed to sound like. For the average indie game-dev however, hiring a sound designer can be expensive and crafting the sound yourself can be difficult if you don't already have a good handle of sound design.

What I chose to hone in on here is a material-based reverb calculation. Tools like WWise already does this through their Acoustic Textures, but I wanted to figure out how it worked a little better.

So I attempted to write a material-based reverb plugin from scratch.

For this, I used the Unity Native Audio SDK to write a Unity mixer plugin. Here, I'll document the steps I took so you can try it out yourself! Download the actual build at the bottom of the page!

Here's my Git repo for source files: https://github.com/mikosama326/unity-audio-endeavors

Unity Native Audio SDK

To set up my plugin, I followed the steps on the official Unity guide. I used the demo project they provided to get me started on a basic reverb plugin that I could feed my material data into.

The plugin itself is a C++ project that compiles to a .dll file that can then be loaded into Unity.

The Reverb

The wet reverb was calculated by using the Reverberation Time. This is based on the Sabine Equation:

Where 'V' is the volume of the room and A is calculated as:

Here 'S' represents the Coefficient of Absorption of the material of each surface. It is a value ranging from 0 to 1. This is one of the parameters we feed into the mixer plugin.

The Plugin

The plugin itself was written to be highly configurable. The data for the geometry/materials of the room are fed into the plugin as 3 parameters: Volume, SA (the 'A' in the Sabine Equation) and the Number of Walls in the room.

I guess I haven't explained the Number of Walls parameter yet. It contributes to calculating the number of bounces the sound waves goes through in the room. This determines the complexity of the reverberation.

The Controller

To demonstrate all this, I wrote a simple controller using Unity Monobehaviors. It basically feeds in the correct values into the reverb plugin based on the state of the objects in the scene.

For the demo in this download, I set up a mapping of Unity Materials with Absorption Coefficients and allowed users to swap out the material of the walls of the room. For now, Number of Walls is fixed at 4 (but it's an exposed variable. Easy to change.)

I also assumed the room was a cube and considered room size as the side of the cube. The goal of the demo is to test materials anyway.

I also hooked up some basic UI so that we could control these values. Pretty simple.

Lastly, and most importantly, was to set up an AudioSource routed to the mixer track with my Material Spatializer Reverb plugin on it.

And voila!

Notes about Spatialization

I made use of the existing demo's calculation of HRTF and Spatialization for my reverb. 

Make sure to set up your project to use the Spatializer plugin in the Audio tab in Project Settings.

Issues I ran Into

The singular biggest issue I ran into was finding the equations to actually make this happen. I had to scour the internet to figure out to use Sabine's Equation for calculating delay.

Another issue I ran into was building this application for WebGL. It seemed to build fine and even worked fine, but it's finicky since we need to build the plugins for HTML5 as well. I finally switched over to building a 64-bit Windows exe instead to make this more deterministic.

More Resources

For more info on Unity Audio Spatialization, here.

The Sabine Equation, a whitepaper.

Images and equations from here.

How to do all this on MacOS, here.