Sgt. Conker We are "absolutely fine"

Article: Using XNA Content pipeline extensions for localization.

by Roy Triesscheijn

Introduction

In this tutorial series I will show how to setup a simple Content Pipeline extension. At first it will be used to parse XML files containing text and their translations, in a part 2 we are going to extend our processor to also parse other xml files which contain data about localized textures and sounds. We extend XNA’s content manager to make use of all this extra data.

Doing all this work in a content processor and writing a custom content manager means that we don’t have to change every line of code where we load content to factor in localization. All we need to do is add new content (like a texture with text on it in a different language) and write an xml file that explains our new content manager what files are alternatives to the original file and what language they are in. This approach works on all platforms XNA runs on, because the localization data is compiled into XNB files.

Part 1: Setup, and localizing strings.

As this is part 1 of this tutorial we are only going to focus on setting up the content pipeline extension and using it to display strings in different languages. (Note: that hardcoded strings are never loaded via the ContentManager, so you will have to change all hardcoded strings in your code with a call to our new content manager, there isn’t a way around this unfortunately).

The XML File

Before we fire up Visual Studio we are going to have to think about the XML format for our string translation data. This is what I came up with, but of course you’re free to develop your own format suitable for your needs.

(Note: I’ve called my implementation XNABabylon, you’ll probably see that name pop-up now and then).

This is the XML format I decided on and will use for the rest of this tutorial.

<?xml version="1.0" encoding="utf-8" ?>
<XNABabylonStringFile>
  <version>1.0</version>
  <revisionDate>25-01-2010</revisionDate>
  <author>Roy Triesscheijn</author>
  <language>English</language>
  <!--The package id for a file named PACKAGE.LANGUAGE.bab should be PACKAGE-->
  <package id="MainMenu">
    <key id="start" val="Start Game"/>
    <key id="quit" val="Quit game by pressing escape"/>
    <key id="motd" val="Press space to change language"/>
  </package>
</XNABabylonStringFile>

We will use the package id, which is also the first part of the filename, to load the correct file. (The second part of the filename is the language). The package id we will also be used for namespacing. Each file should contain one package.

Setting up the solution

Now we’ve got the formalities out of the way, let’s start coding!

Open up the solution in which you want to add localization (or create a new XNA Game Project). Right click the solution in the solution explorer and choose “Add new project”

Add a new ContentPipeLineExtension to your solution. Now we need reference this extension in our game project. To do this you have to add a reference to your extension project in both your Game project AND in the games content project.

The Importer

Now we are going to create the classes for our ContentPipeLineExtension project. Create a new class and call it BABImporter, this class is going to Import our ‘xml’ files and convert them to instances of System.Xml.XmlDocument.
As you’ll see this is pretty trivial to do, however we need to make sure XNA understands what our class is for.

Right under the using statements, before the namespace add the following line:

using TImport = System.Xml.XmlDocument;

Here we define that our output type for this importer is going to be an XmlDocument. Now make sure your class inherits from ContentImporter by modifying your class defintion to:

public class BABImporter : ContentImporter

(If needed add the necesairy using statements by right clicking on the text and selecting resolve) Now right above your class definition add the following line:

[ContentImporter(".bab", DisplayName = "XNA Babylon .bab importer", DefaultProcessor = "BABProcessor")]

This data tells Visual Studio that this class has a special meaning in the editor. If we compile our class, all projects that have a reference to this class’ project will add this importer to the list of importers for content, also if we add new .bab files to a project the editor will automatically choose this importer. You see that we already have a DefaultProcessor named. Here you should fill in the processor that you want assigned to .bab files by default. (Fill in the class name for the processor that we are going to make shortly).

Now all we need to do for the importer is overriding the method Import. So create a new method in this class looking like this:

public override TImport Import(string filename, ContentImporterContext context)
        {
            XmlDocument stringFile = new XmlDocument();
            try
            {
                stringFile.Load(filename);
            }
            catch (Exception e)
            {
                context.Logger.LogImportantMessage("The file "
                    + filename + " is not valid: " + e.Message);
                throw e;
            }
            return stringFile;
        }

As you can see here, all we do is creating a new XmlDocument from the file inputted. Looks simple doesn’t it?

The BABFile class

Before we can process this data we need to define a class that can hold all the relevant information after processing. Create a simple class called BABFile like this one that can store all the data from your xml file, make sure to mark it [Serializable].

[Serializable]
    public class BABFile
    {
        public string language;
        public double version;
        public string author;
        public string package;
        public DateTime revision;
        public Dictionary<string string ,> keys;

        public BABFile()
        {
            keys = new Dictionary<string string ,>();
        }
    }

The processor

Add another class to your extension project and call it BABProcessor, this processor is going to receive the output of the importer as input, and processes it into an instance of the BABFile class. The processor is just a simple XML reader, however we do have to mark the class with a few special attributes so that Visual Studio knows what it’s for.
First add the following lines under the using statements above the namespace declaration:

using TInput = System.Xml.XmlDocument;
using TOutput = XNABabylon.Importers.BABFile;

These lines define our input and output types. Now make sure that your class inherits from ContentProcessor (resolve the using statements if needed).
And add this line above your class declaration:

[ContentProcessor(DisplayName = "XNA Babylon .bab Processor")] 

This tells visual studio how to display this content processor in the dropdown menus of content in projects that reference this project. All we need to do now is overide the method Process and add some basic XML reading, I’ll just put the code down right here because I don’t think it’s hard to understand, but feel free to ask questions.

public override TOutput Process(TInput input, ContentProcessorContext context)
        {
            TOutput output = new BABFile();

            //We traverse the XML file as usual and store certain elements into the
            //output object;
            if(input.GetElementsByTagName("XNABabylonStringFile").Count > 0)
            {
                foreach (XmlNode node in input.GetElementsByTagName("XNABabylonStringFile")[0].ChildNodes)
                {
                    switch (node.Name)
                    {
                        case "version":
                            Double.TryParse(node.InnerText, out output.version);
                            break;
                        case "revisionDate":
                            DateTime.TryParse(node.InnerText, out output.revision);
                            break;
                        case "author":
                            output.author = node.InnerText;
                            break;
                        case "language":
                            output.language = node.InnerText;
                            break;

                        case "package":
                            //Loop trough all the keys and store the key id and translated value into a dictionary
                            Dictionary<string string ,> package = new Dictionary<string string ,>();
                            foreach (XmlNode key in node.ChildNodes)
                            {
                                output.keys.Add(key.Attributes["id"].InnerText, key.Attributes["val"].InnerText);
                            }
                            break;
                    }
                }
            }

            return output;
        }

Now compile and resolve any errors you might have (there should be none). Testing the first files Go to your GameProject and add a new xml file to the Content project in there. (Right click it and choose add new). Rename the xml file to “MainMenu.English.bab” and select it in visual studio. Now in this file’s properties make sure to set build action to “Compile” and set the correct importer and processor as show below.

(If the importer and processor don’t show up, make sure that the reference to your extension project is both in the Content project’s references and in the encapsulating game project’s reference the latter we don’t need yet, but why not add it already).

Open up MainMenu.English.bab and just paste in the content from the sample xml file, located at the beginning of this tutorial. Then create another file called MainMenu.Dutch.bab set the same properties and paste in the following xml data:

<?xml version="1.0" encoding="utf-8" ?>
<XNABabylonStringFile>
  <version>1.0</version>
  <revisionDate>25-01-2010</revisionDate>
  <author>Roy Triesscheijn</author>
  <language>Dutch</language>
  <!--The package id for a file named PACKAGE.LANGUAGE.bab should be PACKAGE-->
  <package id="MainMenu">
    <key id="start" val="Start Spel"/>
    <key id="quit" val="Stop het spel door te drukken op escape"/>
    <key id="motd" val="Druk op spatie om van taal te veranderen"/>
  </package>
</XNABabylonStringFile>

So we now have English and Dutch language files. Test if everything still compiles (it should).

The Babylon content manager
We can now import our files and we can load them using the original content manager with something like:

Content.Load(“MainMenu.English”); 

And manually use the BABFile’s dictionary to find our translation. However since in part two we are going to write a new ContentManager, why not start a bit earlier. Go to your Extension project again and create a new class called BabylonContent and make it look like below. I think the comments sum up nicely what the methods do but let’s walk trough them quickly. The most important method, GetString, tries to find a string by looking in a package and it’s keys. First it tries to load the file associated with the current language. If that fails it loads the file for the default language and tries to find the string there. If this fails aswell we simply return an empty string. (For debugging purposes you might want to throw a hard error here, so you don’t miss missing strings).

You see that in the overridden Load method we now have a special section for strings. You can now do Load(“MainMenu.Start”); and get the translation for “Start Game” in the current language. The other methods are just helpers for creating ‘proper’ strings and some bookkeeping. Nothing that should really blow your mind.

public class BabylonContent : ContentManager
    {
        public BabylonContent(IServiceProvider serviceProvider, string rootDirectory)
            : base(serviceProvider, rootDirectory)
        {
            packages = new Dictionary<string , string Dictionary><string ,>>();
            Language = DEFAULTLANGUAGE;
        }

        /// <summary>
        /// Load strings using &quot;PackageName.Key&quot;, load other files normally.
        /// </summary>
        public override T Load<t>(string assetName)
        {

            if (typeof(T) == typeof(String))
            {
                string[] packageKey = assetName.Split('.');
                if (packageKey.Length > 1)
                {
                    return (T)(object)GetString(packageKey[0], packageKey[1]);
                }
                else
                {
                    throw new Exception(assetName + &quot; is not of the format 'PackageName.Key'&quot;);
                }
            }
            else
            {
                return base.Load<t>(assetName);
            }
        }

        /// <summary>
        /// Given a package and a key finds the corresponding string in the
        /// language currently set in .Language
        /// </summary>
        public string GetString(string package, string key)
        {
            return GetString(package, key, language);
        }

        /// <summary>
        /// Given a package and a key finds the corresponding string in the
        /// language requested
        /// </summary>
        public string GetString(string package, string key, string packageLanguage)
        {
            package = package.ToLower();
            key = key.ToLower();
            packageLanguage = ProperString(packageLanguage);

            if (packages.Keys.Contains(package))
            {
                return (packages[package])[key];
            }

            try
            {
                BABFile bab = base.Load<babfile>(package + &quot;.&quot; + packageLanguage);
                packages.Add(package, bab.keys);
            }
            catch (ContentLoadException e)
            {
                //Now try to load the content using the default language
                //but only if the current language isn't the default language
                //else we would get stuck in infinite exception recursion.
                if (!packageLanguage.Equals(DEFAULTLANGUAGE))
                {
                    return GetString(package, key, DEFAULTLANGUAGE);
                }
            }

            if (packages.Keys.Contains(package))
            {
                return (packages[package])[key];
            }

            //The string searched for couldnt be found in the
            //translated file or the default file.
            return &quot;&quot;;

        }

        /// <summary>
        /// Converts LaNGuAGe to Language
        /// </summary>
        private string ProperString(string value)
        {
            if (value.Length > 0)
            {
                value = value.ToLower();
                value = value.ToUpper()[0] + value.Substring(1); ;
            }
            return value;
        }

        private Dictionary<string , string Dictionary><string ,>> packages;

        private string language;
        public string Language
        {
            get
            {
                return language;
            }
            set
            {
                //Set the new language and clear the translation dictionary
                if (language == null || !language.Equals(ProperString(value)))
                {
                    language = ProperString(value);
                    packages.Clear();
                }
            }
        }

        //Change this variable to the default language your content files are in
        //This value will be used as a fallback for untranslated content
        private static string DEFAULTLANGUAGE = &quot;English&quot;;
    }

(In the next tutorial we are going to override the the Load<> method for all types to add ‘automatic’ support for localized textures and sounds).

A usage example

Now we’ve got all this code and nothing to show for. I’m now going to show you a couple of lines that should show you how to use this new content manager to load localized strings from files. (if you don’t want to do this yourself feel free to just download the sourcecode and run the sample code) Go to your game project and open up your game’s class (Usually Game1.cs).
Define the following two members:

SpriteFont font;
KeyboardState oldState;

Now add the following line to the end of the constructor.

Content = new BabylonContent(Content.ServiceProvider, Content.RootDirectory); 

This line overwrites the default ContentManager with our newly created instance which is capable of loading strings and doing some localization. (And which will do much more in part 2).
Add the following line to LoadContent:

font = Content.Load("Arial");

And change the update method to look like this:

           KeyboardState newState = Keyboard.GetState();

            if (GamePad.GetState(PlayerIndex.One).Buttons.Back == ButtonState.Pressed ||
                newState.IsKeyDown(Keys.Escape))
                this.Exit();

            if(newState.IsKeyDown(Keys.Space) && !oldState.IsKeyDown(Keys.Space))
            {
                //We can cast Content to type BabylonCOntent for extra
                //functionality
                BabylonContent BContent = (BabylonContent)Content;
                if (BContent.Language == "English")
                {
                    BContent.Language = "Dutch";
                }
                else
                {
                    BContent.Language = "English";
                }
            }
            oldState = newState;

            base.Update(gameTime);

And finally the draw method to look like this:

GraphicsDevice.Clear(Color.CornflowerBlue);

            spriteBatch.Begin();

            spriteBatch.DrawString(font, Content.Load<string>(&quot;MainMenu.start&quot;), Vector2.Zero, Color.White);
            spriteBatch.DrawString(font, Content.Load<string>(&quot;MainMenu.motd&quot;), new Vector2(0, 150), Color.White);
            spriteBatch.DrawString(font, Content.Load<string>(&quot;MainMenu.quit&quot;), new Vector2(0, 250), Color.White);
            spriteBatch.DrawString(font, ((BabylonContent)Content).Language, new Vector2(0, 350), Color.Gray);

            spriteBatch.End();

            base.Draw(gameTime);

Now as you can see we can just use Content.Load to load our localized strings. Let’s fire up our demo, press space to change between English and Dutch.

Conclusion

With a relatively little amount of code we made feature full string localization that can be plugged into games easily. If you have a lot of hardcoded strings in your game it might take a while to put them all into xml files and change them to Content.Load<> instructions. In the next part we are going to make our content manager localize textures and sound files. Because there we already use Content.Load<> no code has to be changed after plugging in the new content manager. All we need to do is make some xml files.

This approach to localization is flexible and enforces a couple of good practices (like not using hardcoded strings). I think it’s easy to built in to an almost completed game. And games that are available in many languages might be more enjoyable for people who are not native speakers in the language your game uses. Using xml files in a package like structure makes it easy for multiple translators to work on different localizations of the strings at the same time, without having to look at source code or using complex tools.

Only loading the languages that are requested make this code pretty efficient, however the code might need a few iterations of ‘clean up’ which is always the case with “first release” code.

Download Sample Project