Game Developers Diary 2: The world is alive with the sound of OpenAL

In the last chapter we learned a bit of basic OpenGL functionality. While not enough, to be actually useful in a game, it got our appetite wet and gives a good base, from which to investigate further.

Today, we will mainly leave OpenGL behind and focus on the other second-most important user interface in games: Sound.

More to the point: Sound, using OpenAL. That choice is convenient for a few very simple reasons:

• It’s available in nearly any possible target system
• It’s open and rock solid
• It’s very simple to understand
• It has most of the basic features, you could want from a game sound interface

What we want to end up with

As last time, we will attempt to build a small scene, which shows the features of our effort.

Since our effort will be 3d sounds and music playback, we want something, that has some kind of traveling object, making noises, in an attempt to hear the sound travel along the axis of our speakers.

Therefor we will extend the scene from the first diary. Instead of a single spinning cube, we will have a spinning cube with a little second cube, that circles the first one as a kind of satellite. Being a satellite it will emit some kind of weird space noise.

To show the mixing of sound and game music, we will also have a little looping music track in the background.

Also, to emphasize the 3d aspect of the sound even more, we will have the camera travel along some kind of path, while always facing the main cube.

For the audio resources, I used some beeping noise from Freesound. If you go hunting for one yourself, think about either getting a sound that is a loopable mono-wav file already or convert it later, using an Audio-editing tool, such as Audacity. Remember: You want to have a 44k mono wave file, that sound nice, when played in a loop.

For the background music I played around with apple’s garageband and the factory-delivered apple-loops, until I got something decent. But you may find something on Freesound for that as well. With the music, you are again looking for a sound that is 44k and sounds nice, when played in a loop; only this time, you want a stereo file.

So what’s with all the mono and Stereo? I hear you cry.

Well, OpenAL, being a 3d sound engine and all, needs sound sources to be mono, to be able to distribute them along a virtual space. But it does play stereo sources as well. Only, it does so without bothering to position the stereo sound in the virtual space.

Therefor all stereo sources will appear to be played normally, as they wood from any other player, while mono sources can be distributed along a virtual environment. Which as it turns out, is exactly what we want.

Some Housekeeping

Since our scene will get a bit more complex, we will first need to cleanup the code a bit, so that we don’t have to repeat ourselves.

A brand new Cube class

Remember that whole chunk of drawing code for the cube, we did put directly into our draw function? Yeah, that needs to be gone. In fact, since we are now drawing a second cube as well, we will create a separate Cube class.

For starters, call up the New File dialog (⌘N) and select Objective-C class from the Cocoa submenu. Call your new class Cube.

Our Cube class will know it’s scale, rotation angle and center position (x,y,z) and will be able to draw itself. It will also have a convenience method, to set the position with x, y and z in a single setter. Therefor, update your header file, to look like so:

#import <Foundation/Foundation.h>

@interface Cube : NSObject

@property (assign) float rotationAngle;
@property (assign) float scale;
@property (assign) float positionX;
@property (assign) float positionY;
@property (assign) float positionZ;

- (void) setPositionX:(float) x Y:(float) y Z:(float) z;
- (void) draw;

@end

In the implementation, between the @implementation and @end directive, you want to start with synthesizing the properties like so:

@synthesize rotationAngle;
@synthesize scale;
@synthesize positionX;
@synthesize positionY;
@synthesize positionZ;

Synthesizing is a neat little feature of ObjC 2.0, which spares you the misery of creating getters and setters for your class properties, while still giving you control over which properties you want getters and setters be created for.

Next, let’s implement our convenience method, to set all three parameters of the center position at once.

- (void)setPositionX:(float)x Y:(float)y Z:(float)z{
    self.positionX = x;
    self.positionY = y;
    self.positionZ = z;
}

If you’re new to ObjC, that may seem a bit odd to you. Basically, ObjC likes to interweave the name of a method with it’s parameter, in an attempt to be nicer to read. Therefor the full name of this method would be setPositionX:Y:Z: and you would call it, by writing the different arguments after the colons.

Before we start implementing the draw function, we have to remember, that OpenGL is basically a matrix-transformation framework. Which means two things we have to keep in mind, to be able to draw a rotated cube independently from any other scene object, that might rotate or otherwise transform the MODELVIEW matrix:

1. We need to take care, that we end our drawing code, with the same unaltered matrix active in OpenGL as when we started drawing.
2. We want to first transform the matrix to suit our needs and then draw the cube like normally.

With that in mind, let’s prepare the scene:

-(void)draw{
    glPushMatrix();

    //TODO: scale and draw the cube
    
    glTranslatef(positionX, positionY, positionZ);
    glRotatef([self rotationAngle], 1, 1, 1);
    glPopMatrix();
}

The above code makes use of the OpenGL matrix stack, to guarantee that we do leave the drawing function with the same matrix active, as when we entered it.

The OpenGL matrix stack is a simple stack on the graphic card, that can hold a number of matrixes. You can either push the currently active matrix to the stack, which creates a copy on the stack, leaving the active matrix in place; or choose to pop the top-most matrix from the stack and set it as the active one, which overwrites any matrix-state on the active matrix with the one from the stack.

So, after pushing the current matrix to the stack, we can safely rotate and transform it, to fit the properties of our cube.

Now that we have a correct matrix and have taken care, that we don’t effect the drawing of any other objects, we can put in the actual drawing code for our cube, where the TODO comment was:

    float s = scale*.5;
    
    glBegin(GL_QUADS); 
    {
        glColor3f(0, 0, 1);
        glVertex3f(-1*s,  1*s, -1*s); //F T L
        glColor3f(1, .75, 0);
        glVertex3f( 1*s,  1*s, -1*s); //F T R
        glColor3f(0, 1, 0);
        glVertex3f( 1*s, -1*s, -1*s); //F B R
        glColor3f(1, 0, 0);
        glVertex3f(-1*s, -1*s, -1*s); //F B L
        
        glColor3f(1, 0, 0);
        glVertex3f(-1*s, -1*s, -1*s); //F B L   
        glColor3f(0, 1, 0);
        glVertex3f( 1*s, -1*s, -1*s); //F B R
        glColor3f(0, .5, 0);
        glVertex3f( 1*s, -1*s,  1*s); //B B R
        glColor3f(.5, 0, 0);
        glVertex3f(-1*s, -1*s,  1*s); //B B L
        
        glColor3f(0, 0, .5);
        glVertex3f(-1*s,  1*s,  1*s); //B T L
        glColor3f(0, 1, 1);
        glVertex3f( 1*s,  1*s,  1*s); //B T R
        glColor3f(0, .5, 0);
        glVertex3f( 1*s, -1*s,  1*s); //B B R
        glColor3f(.5, 0, 0);
        glVertex3f(-1*s, -1*s,  1*s); //B B L
        
        glColor3f(0, 0, .5);
        glVertex3f(-1*s,  1*s,  1*s); //B T L
        glColor3f(0, 0, 1);
        glVertex3f(-1*s,  1*s, -1*s); //F T L
        glColor3f(1, 0, 0);
        glVertex3f(-1*s, -1*s, -1*s); //F B L   
        glColor3f(.5, 0, 0);
        glVertex3f(-1*s, -1*s,  1*s); //B B L
        
        glColor3f(0, 0, .5);
        glVertex3f(-1*s,  1*s,  1*s); //B T L
        glColor3f(0, 1, 1);
        glVertex3f( 1*s,  1*s,  1*s); //B T R
        glColor3f(1, .75, 0);
        glVertex3f( 1*s,  1*s, -1*s); //F T R
        glColor3f(0, 0, 1);
        glVertex3f(-1*s,  1*s, -1*s); //F T L
        
        glColor3f(1, .75, 0);
        glVertex3f( 1*s,  1*s, -1*s); //F T R
        glColor3f(0, 1, 1);
        glVertex3f( 1*s,  1*s,  1*s); //B T R
        glColor3f(0, .5, 0);
        glVertex3f( 1*s, -1*s,  1*s); //B B R
        glColor3f(0, 1, 0);
        glVertex3f( 1*s, -1*s, -1*s); //F B R
        
    }
    glEnd();

That’s it. Now we have a fully functional cube class.

Refactor the MyOpenGLView

Now, let’s start putting our glittering new Cube to work and kill some code in the MyOpenGLView class.

We will define the main cube from the last scene and also it’s little satellite in one step. To do that, our view needs two new class-fields. The code highlighted below should be added to the header file, to accomplish that. Also we can remove the variable for the cubeRotationAngle from here now, because that information is now handled inside the cube class.

@interface MyOpenGLView : NSOpenGLView {
    long lastTicks;
}

@property (retain) Cube* mainCube;
@property (retain) Cube* flyingCube;

@end

Also we would want to synthesize our two new cupe properties in the class implementation file:

@synthesize mainCube;
@synthesize flyingCube;

Let’s initialize our cube properties to some meaningful values.

Anyone who played around a bit with the code from the last article, will have noticed that there is a small bug, that didn’t cause much damage in our old code, but will get vital now, that we actually do some real initialization. The old init was actually never called, when the view was loaded. Instead, Cocoa initializes a view it loads from the Nib-file via a function called awakeFromNib.

So to successfully perform our initialization, let’s delete the old init function and put our new initialization into the new function:

- (void)awakeFromNib{
    lastTicks = clock();
    self.mainCube = [[Cube alloc] init];
    self.flyingCube = [[Cube alloc] init];
    [mainCube setScale:1];
    [mainCube setPositionX:0 Y:0 Z:0];
    [flyingCube setScale:.2];
    [flyingCube setPositionX:5 Y:0 Z:0];
    
    [super awakeFromNib];
}

Basically we only set a starting position and scale for both our cubes here and (as we already did last time) reset the frame clock.

Now let’s rethink the drawing function:

We will leave the time calculation procedure alone, as we still need that unchanged. The first change will be therefore, that we store the calculation result of our rotation calculation in the cube class, instead of our own class variable and will not worry about applying that to OpenGL, since the Cube takes care of that as well. We will remove the cubeRotationAngle and the following call to glRotatef() and instead put the code below, where the old calculation was, just after the call to glLoadIdentity.

[mainCube setRotationAngle:[mainCube rotationAngle]+(.2*delta_t)];

Now for the best bit: The while code to actually draw the cube; from the calculation of the scale factor (float s = …) to the end of the glBegin … glEnd block can be deleted. That feels good, right?

Instead we will simply be calling the cube, to draw itself:

[mainCube draw];

Drawing the satellite

We are now at the point, where we have exactly the same scene back, as we started with. But since our code now is overall cooler than before, we can easily extend it.

Let’s draw that satellite, that we initialized earlier.

Since it should be a satellite, it should rotate around the cube in it’s center at a constant pace and in an elliptical track. Which leaves us with our first mathematical riddle we have to solve: Given a time in milli-seconds, how can we calculate an elliptical movement around a center?

The answer to that riddle comes in the form of two well known mathematical concepts that most of you may have already forgotten after leaving school: Sinus and Cosinus. Given a two-dimensional movement around the center of a graph-system, you can calculate pretty accurate circle-movements, using the Sinus over t over one axis and the Cosinus over t along the other. That movement will draw a circle with a radius of one. We can now finish that off, with applying different scale factors for each of the axis and end up, with an ellipsis.

In code, that means two things:

1. We need an amount of time, rather than the delta, we used until now
2. We need to set the position of our little satellite according to a calculation, that reassembles the concept, explained above.

To get a full amount of time, put the following line right after your existing delta_t calculation in the drawRect function.

long t = ticks/(CLOCKS_PER_SEC/1000);

To set the satellite’s position we first need to import the math.h library. Then we need to put the following bit of code just below the call to draw the main cube:

[flyingCube setPositionX:sinf(t*.025)*4 Y:0 Z:cosf(t*.025)*2];
[flyingCube draw];

That’s it. now we have a flying satellite of our main cube.

The Camera goes crazy

All that is left to prepare the scene, to actually have some sound in there, is now, to move the camera around the main cube as well, on some arbetrary path.

Thanks to a helper method called glLookAt() that is quite easy to do. It is a convenience method of OpenGL, that takes an X/Y/Z coordinate for the camera in space, an X/Y/Z coordinate for whatever the camera should look at and an Up-Vektor, to determine, how the camera is rotated and then sets up the scene appropriately.

The code below should be no problem to understand by now, and goes before the drawing code for the main cube.

float camZ = (sinf(t*.01)*14)-7;
float camX = (cosf(t*.005)*12)-6;
gluLookAt(camX, 5, camZ, //camera
          0, 0, 0,  //lookAt
          0, 1, 0); //UP-Vector

Since we’re now done with the preparations and can dive into actually making some sounds, it seems to be a good time for a recap. If you’re drawing function now looks something like this, you can be happy.

- (void)drawRect:(NSRect)dirtyRect {
    long ticks = clock();
    //delta_t in millis.
    long t = ticks/(CLOCKS_PER_SEC/1000);
    int delta_t = (int)((ticks - lastTicks)/(CLOCKS_PER_SEC/1000));
    int fps = delta_t > 0?(int) 1000/delta_t:1000;
    
    [[self window] setTitle:[NSString stringWithFormat:@"%d fps", 
                             fps]];
    lastTicks = ticks;
    
    glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
    glClearColor(0, 0, 0, 0);
    
    glLoadIdentity();
    
    float camZ = (sinf(t*.01)*14)-7;
    float camX = (cosf(t*.005)*12)-6;
    gluLookAt(camX, 5, camZ, 
              0, 0, 0, 
              0, 1, 0);

    [mainCube setRotationAngle:[mainCube rotationAngle]+(.2*delta_t)];
    [mainCube draw];
    [flyingCube setPositionX:sinf(t*.025)*4 Y:0 Z:cosf(t*.025)*2];
    [flyingCube draw];
    
    glFlush();
    
    [self setNeedsDisplay:YES];
}

Onwards to OpenAL

OpenAL itself is quite simple, if a bit ankward to initialize. But since the alutLoadWAVFile() function, that has been frequently used until recently, is deprecated and not supported by Apple’s implementation of OpenAL, we need to use Apple’s Foundation functions, to actually load our sound files and initialize them for OpenAL.

OpenAL itself is designed around four basic structures: It defines an active Device, representing the hardware, that actually renderes the audio.

Sending completely rendered Audio to the active device is the Listener. This structure represents the listener inside the 3d world, hearing the sounds from different places in the world.

This different sounds are defined as Sources. A Source is a single sound, bound to a certain location in the 3d space.

Sounds are loaded and played through the API, using Buffer objects, which hold the actual wave-form content for the raw audio.

Initialisation

To utilize OpenAL, we need to first add our audio data to the project. As I described at the start of this article, you would want to have one loopable 44k mono wav file for the satellite sound (let’s just call it ‘satellite.wav’ for now) as well as a loopable 44k stereo wav file, for the background music (‘music.wav’).

We add them to the project and with that to the delivered application package in the end, by right-clicking on the ‘Supported Files’ folder in our project and choosing ‘Add Files to “GameDevDiary1″‘. From the appearing file chooser dialog, select your wav files and click ok. You should now see the files in the ‘Supported Files’ folder of your project.

Codewise, we will abuse the prepareOpenGL function of our view for the initialization.

First, let’s initialize the default audio device, by putting the following code after at the end of th current prepareOpenGL function:

    ALenum lastError = alGetError();
    ALCdevice *soundDev = alcOpenDevice(NULL);
    ALCcontext *soundContext = alcCreateContext(soundDev, 0);
    alcMakeContextCurrent(soundContext);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error loading sound device!");
        return;
    } else {
        NSLog(@"Sound device loaded successfully!");
    }

As you can see, initializing the default device is actually pretty straight-forward. The only really interesting bit in that code is the wired error handling.

For every operation in OpenAL, that has a possibility to fail, OpenAL will set an error state, that can be read by calling the alGetError() function.

If no error has occurred, then the returned state will be set to AL_NO_ERROR. Every other state indicates an error of some sort.

To prepare for a new call to the OpenAL engine, it is advisable to read the error state before doing so, since reading it will reset the stored error state to AL_NO_ERROR. That way, you can be sure, that any error you might read afterwards is actually from your current API call and not still stored from an earlier call to OpenAL.

Loading sound from files

After initializing the device, we need to initialize the buffers and sources, we will be using. This mean, loading two different sound files, connecting them to an OpenAL buffer and attaching that buffer to a source, that is configured, to play in loop mode.

Since we will be doing this twice, it makes sense to create a separate function for it, to prevent having to duplicate code.

- (ALuint) loadAudioIntoSourceFromFile:(NSString*) filename 
                            withFormat:(ALenum) audioFormat{
    
    //clear the error cache
    ALenum lastError = alGetError(); 
    
    AudioFileID audioFileId;
    
    //get the file path from the app bundle
    NSString* path = [[NSBundle mainBundle] pathForResource:filename 
                                                     ofType:nil];

    //let OsX open the audio file
    NSURL* audioUrl = [NSURL fileURLWithPath:path];
    OSStatus openResult = AudioFileOpenURL((__bridge CFURLRef)audioUrl, 
                                           fsRdPerm, 0,
                                           &audioFileId);
    
    //handle errors
    if (openResult != 0){
        NSLog(@"cannot open file: %@",audioUrl);
        return -1;
    } else {
        NSLog(@"Loading Audio from file: %@ ...", audioUrl);
    }
    
    //prepare the buffer for OpenAL
    UInt64 outDataSize = 0;
	UInt32 thePropSize = sizeof(UInt64);
    AudioFileGetProperty(audioFileId,
                         kAudioFilePropertyAudioDataByteCount, 
                         &thePropSize, &outDataSize);
    UInt32 outSize = (UInt32) outDataSize;
    
    //read the audio data into the buffer
    void * audioData = malloc(outSize);
    AudioFileReadBytes(audioFileId, 
                       false, 0, 
                       &outSize, 
                       audioData);
	AudioFileClose(audioFileId); //close the file
    
    NSLog(@"Audio loaded. %d bytes of data", outSize);
    
    //create an OpenAL buffer from the data
    ALuint bufferId;
    alGenBuffers(1, &bufferId);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error creating a new buffer: %d", lastError);
        return -1;
    }
    
    alBufferData(bufferId, audioFormat, audioData, outSize, 44100);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error buffering the audio data: %d", lastError);
        return -1;
    }
    
    //create an OpenAL source, for the loaded sound
    ALuint sourceId;
    alGenSources(1, &sourceId);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error creating a source: %d", lastError);
        return -1;
    }
    
    
    // attach the buffer to the source
	alSourcei(sourceId, AL_BUFFER, bufferId);
    free(audioData);
    
    return sourceId;
}

This method uses OsX default calls, to load a sound file from within the application bundle and store the decompressed waveform data in a byte array. This array is then attached to an OpenAL buffer and set up in an OpenAL source, which will be returned by the method.

Setting up the scene

With that method in place, we can now create a source for the satellite sound; set it’s basic OpenAL parameters, which determine, how the source should be played and place the source at an initial value in the 3d space. The code below should be written into the prepareOpenGL function, after the initialization of the OpenAl device.

     satelitteALSourceId = [self loadAudioIntoSourceFromFile:@"satellite.wav"
                                                 withFormat:AL_FORMAT_MONO16];
    
	// set some basic source prefs
	alSourcef(satelitteALSourceId, AL_PITCH, 1.0f);
	alSourcef(satelitteALSourceId, AL_GAIN, 1.0f);
	alSourcei(satelitteALSourceId, AL_LOOPING, AL_TRUE);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error attaching the buffer to the source: %d", lastError);
        return;
    }

Since there is ever only one listener in an OpenAL scene, the Api doesn’t need us to initialize it. But we still have to set it’s position in the 3d space, as well as the position of the satellite source. Also the Api requires us to actively start playing a given source, before it does so. The last line in the code below will do that.

    //set Basic 3d sound parameters
    alListener3f(AL_POSITION, 0, 0, 0);
    alSource3f(satelitteALSourceId, AL_POSITION, 0, 0, 1);
    
    lastError = alGetError();
    if (AL_NO_ERROR != lastError){
        NSLog(@"Error setting 3d sound parameters: %d", lastError);
        return;
    }
    
    alSourcePlay(satelitteALSourceId);

And now some music

We defined at the beginning of this article, that we wanted a stereo background music playing for this scene. Since the OpenAL device itself is now setup, that task is simply a matter of initializing and starting a new stereo OpenAL-source. Since OpenAL doesn’t position stereo sources in the 3d space, we don’t need to set a position for it.

    // start background musik
    ALuint backgroundMusikId = [self loadAudioIntoSourceFromFile:@"music.wav" 
                                                      withFormat:AL_FORMAT_STEREO16];
    alSourcef(backgroundMusikId, AL_PITCH, 1.0f);
    alSourcef(backgroundMusikId, AL_GAIN, 0.01f);
    alSourcei(backgroundMusikId, AL_LOOPING, AL_TRUE);
    alSource3f(backgroundMusikId, AL_POSITION, 0, 0, 1);
    alSourcePlay(backgroundMusikId);

One thing to note here, is the value we set for the AL_GAIN parameter. We set it to a very low level, because in contrast to the satellite sound source, the music will not become more silent, depending on a position in the 3d space and therefore will always play in it’s maximum volume. That’s why we turn it down, to have it actually be a bit less loud than the satellite.

Coming Alive

Until new, we have set up the scene and the graphic moves around, but at this point, the sound is still completely static, since we do not move the satellite’s source or the listener within the draw loop.

To move the satellite, add the following line of code to your drawRect function, right after you did set the position of the satellite.

    alSource3f(satelitteALSourceId, AL_POSITION, 
               [flyingCube positionX], 
               [flyingCube positionY], 
               [flyingCube positionZ]);

At last, to move the listener in sync with the camera, we need to do add another line of code after the call to glLookAt which set’s the OpenGL camera movement.

    alListener3f(AL_POSITION, camX, 5, camZ);

That’s it.

Conclusion

After learning about the basics of OpenGL on the Mac in the last chapter of this series, you now got a bit deeper into the language structure of Objective C, as we refactored our old code, so that we were able to reuse it later.

Also you did learn the basics of how to initialize OpenAL and load sounds into a scene on a Mac platform.

As last time, you can download the full Xcode project for this chapter from here: GameDevDiary1.zip
If you’re simply curious, how the result of this chapter looks like, you can also try out the finished app from here: GameDevDiary1.app

Next time, we will take a look at controls…

So long, and thanks for all the fish…