C:Custom Resource Files

From GPWiki
Jump to: navigation, search

Do you demand more out of life? Are you tired of having your BMPs and WAVs flapping naked in the wind, for all to see? You, my friend, need to gird your precious assets in a custom resource file!

Disgusting imagery aside, custom resource files are an essential part of any professional game. When was the last time you purchased a game and found all of the sprites, textures, sound, or music files plainly visible within the game's directory tree? Never! Or at least, hardly ever!

So, what is a custom resource file? It is a simple repository, containing the various media files needed by your game. Say you have 100 bitmaps representing your game's tiles and sprites, and 50 wave files representing your sound effects and music; all of these files can be lumped into a single resource file, hiding them from the prying eyes of users.

File Format

The format you use for your custom resource file is up to you; encryption and compression algorithms can easily be incorporated. For the purposes of this tutorial however, I'll keep things simple. Here's a byte-by-byte outline of my simple resource file format:

The Header

The header contains information describing the contents of the resource, and indicating where the individual files stored within the resource can be located.

First 4 bytes 
An int value, indicating how many files are stored within the resource.
Next 4n bytes 
Where n is the number of files stored within the resource. Each 4 byte segment houses an int which points to the storage location of a file within the body of the resource. For example, a value of 1234 would indicate that a file is stored beginning at the resource's 1234th byte.

The Body

The body contains filename strings for each of the files stored within the resource, and the actual file data. Each body entry is pointed to by a header entry, as mentioned above. What follows is a description of a single body entry.

First 4 bytes 
An int value, indicating how many bytes of data the stored file contains.
Next 4 bytes 
An int value, indicating how many characters comprise the filename string.
Next n bytes 
Each byte contains a single filename character, where n is the number of characters in the filename string.
Next n bytes 
The stored file's data, where n is the file size.

Example Resource File

Examples tend to make things clearer, so here we go. Numbers on the left indicate location within the file (each segment is one byte), while data on the right indicates the values stored at the given location.

BYTELOC      DATA        EXPLANATION
*******      ****        ***********
0-3          3           (Integer indicating that 3 files are stored in this resource)
4-7          16          (Integer indicating that the first file is stored from the 16th byte onward)
8-11         40          (Integer indicating that the second file is stored from the 40th byte onward)
12-15        10056       (Integer indicating that the third file is stored from the 10056th byte onward)
16-19        9           (Integer indicating that the first stored file contains 9 bytes of data)  
20-23        8           (Integer indicating that the first stored file's name is 8 characters in length)
24-31        TEST.TXT    (7 bytes, each encoding one character of the first stored file's filename)
32-40        Testing12   (9 bytes, containing the first stored file's data, which happens to be some text)
41-44        10000       (Integer indicating that the second stored file contains 10000 bytes of data)
45-48        9           (Integer indicating that the second stored file's name is 9 characters in length)
49-57        TEST2.BMP   (8 bytes, each encoding one character of the second stored file's filename)
58-10057     ...         (10000 bytes, representing the data stored within TEST2.BMP.  Data not shown!)
10058-10061  20000       (Integer indicating that the third stored file contains 20000 bytes of data)
10062-10065  9           (Integer indicating that the third stored file's name is 9 characters in length)
10066-10074  TEST3.WAV   (8 bytes, each encoding one character of the third stored file's filename)
10075-30074  ...         (20000 bytes, representing the data stored within TEST3.WAV.  Data not shown!)

If we had a copy of the file described above it would be 30074 bytes in size, and it would contain all of the data represented by the files TEST.TXT, TEST2.BMP and TEST3.WAV. Of course, this file format allows for arbitrarily large files; all we need now is a handy-dandy program that can be used to store files in this format for us!

Resource Creator Source

In order to create a tool capable of storing files in our simple custom format, we need a few utility functions. We'll start off slow.

int getfilesize(char *filename) {
 
	struct stat file;	//This structure will be used to query file status
 
	//Extract the file status info
	if(!stat(filename, &file))
	{
		//Return the file size
		return file.st_size;
	}
 
	//ERROR! Couldn't get the filesize.
	printf("getfilesize:  Couldn't get filesize of '%s'.", filename);
	exit(1);
}

The getfilesize function accepts a pointer to a filename string, and uses that pointer to populate a stat struct. If the stat struct is not NULL, we'll be able to return an int containing the file size, in bytes. We'll need this function later on!

int countfiles(char *path) {
 
	int count = 0;			//This integer will count up all the files we encounter
	struct dirent *entry;		//This structure will hold file information
	struct stat file_status;	//This structure will be used to query file status
	DIR *dir = opendir(path);	//This pointer references the directory stream
 
	//Make sure we have a directory stream pointer
	if (!dir) {
		perror("opendir failure");
		exit(1);
	}
 
	//Change directory to the given path
	chdir(path);
 
	//Loop through all files and directories
	while ( (entry = readdir(dir)) != NULL) {
		//Don't bother with the .. and . directories
		if ((strcmp(entry->d_name, ".") != 0) && (strcmp(entry->d_name, "..") != 0)) {
			//Get the status info for the current file
			if (stat(entry->d_name, &file_status) == 0) {
				//Is this a directory, or a file?
				if (S_ISDIR(file_status.st_mode)) {
					//Call countfiles again (recursion) and add the result to the count total
					count += countfiles(entry->d_name);
					chdir("..");
				}
				else {
					//We've found a file, increment the count
					count++;
				}
			}
		}
	}
 
	//Make sure we close the directory stream
	if (closedir(dir) == -1) {
		perror("closedir failure");
		exit(1);
	}
 
	//Return the file count
	return count;
}

Things get interesting now. The code above describes a handy little countfiles function, which will recurse through the subdirectories of a given path, and count all of the files it encounters along the way. To do this, a DIR directory stream structure is initialized with a given path value. This directory stream can be exploited repeatedly by the readdir function to obtain pointers to dirent structures, which contain information on a given file within the directory. As we loop, the readdir function will fill the dirent structure with data describing a different file within the directory until all files have been exausted. When no files are left to describe, readdir will return NULL and the while loop will cease!

Now, if we look within the while loop, some cool stuff is going on. First, strcmp is being used to compare the name of a given file entry to the strings "." and ".."; this is necessary, as otherwise the "." and ".." values will be recognized as directories and recursed into, creating a nasty infinite loop!

If the entry->d_name value passes the test, it is then passed to the stat function, in order to fill out the stat structure, called file_status. If a value of zero is returned, something must be wrong with the file, and it is simply skipped. On a non-zero result, execution continues, and S_ISDIR is employed, allowing us to check if the file in question is a directory, or not. If it is a directory, the countfiles function is called recursively. If it is not a directory, then the count variable is simply incremented, and the loop moves on the the next file!

void findfiles(char *path, int fd) {
 
	struct dirent *entry;		//This structure will hold file information
	struct stat file_status;	//This structure will be used to query file status
	DIR *dir = opendir(path);	//This pointer references the directory stream
 
	//Make sure we have a directory stream pointer
	if (!dir) {
		perror("opendir failure");
		exit(1);
	}
 
	//Change directory to the given path
	chdir(path);
 
	//Loop through all files and directories
	while ( (entry = readdir(dir)) != NULL) {
		//Don't bother with the .. and . directories
		if ((strcmp(entry->d_name, ".") != 0) && (strcmp(entry->d_name, "..") != 0)) {
			//Get the status info for the current file
			if (stat(entry->d_name, &file_status) == 0) {
				//Is this a directory, or a file?
				if (S_ISDIR(file_status.st_mode)) {
					//Call findfiles again (recursion), passing the new directory's path
					findfiles(entry->d_name, fd);
					chdir("..");
				}
				else {
					//We've found a file, pack it into the resource file
					packfile(entry->d_name, fd);
				}
			}
		}
	}
 
	//Make sure we close the directory stream
	if (closedir(dir) == -1) {
		perror("closedir failure");
		exit(1);
	}
 
	return;	
}

You may notice that the code above is quite similar to that contained within the countfiles function. I could have removed the code duplication through the use of function pointers, or various other means, but I believe code readability would have suffered; and this is meant to be a quick and dirty resource file creator. Nothing fancy! Besides, the upside is that most of this code is already familiar to us.

Basically, the findfiles routine loops recursively through the subdirectories of a given path (just like countfiles), but instead of counting the files, it determines their filename strings and passes them to the packfile function.

So, bring on the packfile function:

void packfile(char *filename, int fd) {
 
	int totalsize = 0;	//This integer will be used to track the total number of bytes written to file
 
	//Handy little output
	printf("PACKING: '%s' SIZE: %i\n", filename, getfilesize(filename));
 
	//In the 'header' area of the resource, write the location of the file about to be added
	lseek(fd, currentfile * sizeof(int), SEEK_SET);
	write(fd, &currentloc, sizeof(int));
 
	//Seek to the location where we'll be storing this new file info
	lseek(fd, currentloc, SEEK_SET);
 
	//Write the size of the file
	int filesize = getfilesize(filename);
	write(fd, &filesize, sizeof(filesize));
	totalsize += sizeof(int);
 
	//Write the LENGTH of the NAME of the file
	int filenamelen = strlen(filename);
	write(fd, &filenamelen, sizeof(int));
	totalsize += sizeof(int);
 
	//Write the name of the file
	write(fd, filename, strlen(filename));
	totalsize += strlen(filename);
 
	//Write the file contents
	int fd_read = open(filename, O_RDONLY);		//Open the file
	char *buffer = (char *) malloc(filesize);	//Create a buffer for its contents
	read(fd_read, buffer, filesize);		//Read the contents into the buffer
	write(fd, buffer, filesize);			//Write the buffer to the resource file
	close(fd_read);					//Close the file
	free(buffer);					//Free the buffer
	totalsize += filesize;				//Add the file size to the total number of bytes written
 
	//Increment the currentloc and current file values
	currentfile++;
	currentloc += totalsize;
}

This function is really the heart of the program; it takes a file and stores it within the resource as a body entry (which we described above, in the file format section). packfile accepts a filename pointer and an integer file descriptor fd as arguments. It then goes on to store file size, filename, and file data within the resource file (which is referenced with the fd file descriptor). The variables currentfile and currentloc are globals, described in the next segment of code. Basically, they contain values which instruct the packfile function where to create and store this new body entry's data.

Putting these utility functions together is now fairly simple. We just need a Main function, and some includes:

#include "stdio.h"
#include "dirent.h"
#include "sys/stat.h"
#include "unistd.h"
#include "fcntl.h"
#include "sys/param.h"
 
//Function prototypes
int getfilesize(char *filename);
int countfiles(char *path);
void packfile(char *filename, int fd);
void findfiles(char *path, int fd);
 
int currentfile = 1;	//This integer indicates what file we're currently adding to the resource.
int currentloc = 0;	//This integer references the current write-location within the resource file
 
int main(int argc, char *argv[]) {
 
	char pathname[MAXPATHLEN+1];	//This character array will hold the app's working directory path
	int filecount;			//How many files are we adding to the resource?
	int fd;				//The file descriptor for the new resource
 
	//Store the current path
	getcwd(pathname, sizeof(pathname));
 
	//How many files are there?
	filecount = countfiles(argv[1]);
	printf("NUMBER OF FILES: %i\n", filecount);
 
	//Go back to the original path
	chdir(pathname);
 
	//How many arguments did the user pass?
	if (argc < 3)
	{
		//The user didn't specify a resource file name, go with the default
		fd = open("resource.dat", O_WRONLY | O_EXCL | O_CREAT | O_BINARY, S_IRUSR);
	}
	else
	{
		//Use the filename specified by the user
		fd = open(argv[2], O_WRONLY | O_EXCL | O_CREAT | O_BINARY, S_IRUSR);
	}
	//Did we get a valid file descriptor?
	if (fd < 0) 
	{
		//Can't create the file for some reason (possibly because the file already exists)
		perror("Cannot create resource file");
		exit(1);
	}
 
	//Write the total number of files as the first integer
	write(fd, &filecount, sizeof(int));
 
	//Set the current conditions
	currentfile = 1;					//Start off by storing the first file, obviously!
	currentloc = (sizeof(int) * filecount) + sizeof(int);	//Leave space at the begining for the header info
 
	//Use the findfiles routine to pack in all the files
	findfiles(argv[1], fd);
 
	//Close the file
	close(fd);
 
	return 0;
}

The code in function main is primarily concerned with creating the resource file (either giving it the name "resource.dat", or using a string passed as a command-line argument), counting up the files and storing this value in the header, and then calling the findfiles function which loops through all subdirectories and makes use of packfile to pack them into the resource. The user must specify a path command-line argument when executing the program, as this path value will be passed as the initial argument to the findfiles routine. All files within the given path will be found and packed into the resource file! A sample execution:

UNIX:
 ./customresource resource myresource.dat
WINDOWS:
 customresource resource myresource.dat

Calling the program with the command-line arguments shown above would result in a resource file called myresource.dat being created, containing all files found within the directory called "resource" (including its subdirectories).

Source code

  • To download the sample source code (and some media files to play with), click here.
    • NOTE: The above source code will not work with MSVC++, as the dirent.h header file is not included with VC++! If you are using VC++, please download this source code instead (provided by Drew Benton).

Related tutorials

So, we have the power to create custom resource files on a whim... now what? Try one of these lovely tutorials: