Article: Using XNA Content pipeline extensions for localization. (Part 2)
by Roy Triesscheijn
Recap
So… last time we made our strings localizable and added-in a new content manager. Today we are going to make all our content localizable, but first we should revisit and refactor “yesterday’s code”.
After publishing part one and having a good week to think about part two, I found that some of my design choices, which seemed nice, where actually at bit cumbersome.
Also the xml parsing (in BABProcessor) wasn’t as robust as I wanted it to be because I forgot about localized formats for dates and numbers.
Fixing our xml loading is easy, so let’s first fix that!
Refactoring BABProcessor
Load up your previous code, or the source code from last week and go to the BABProcessor.cs file. In the switch in our method Process we parse our version number without caring for localized number formats.
Because of this the version number “1.0” might be parsed as 10 instead of 1 on some computers. So change the code in case version to this:
Double.TryParse(node.InnerText, NumberStyles.Any, CultureInfo.InvariantCulture,out output.version);
Here we tell Double.TryParse to parse the number with an InvariantCulture, so 1.0 will always be interpreted as 1.
The next troubling line is where we parse dates. In Europe we usually write dates as 31-01-2010 (dd-mm-yyyy) but in the U.S. 01-31-2010 (mm-dd-yyyy) is more common, and in Japan they use 2010-01-31 (yyyy-mm-dd). Based on the computers local settings dates will be parsed differently,
so let’s drop down to the revision date case and change the code to this:
DateTime.TryParseExact(node.InnerText, "dd-mm-yyyy", CultureInfo.InvariantCulture, DateTimeStyles.None, out output.revision);
Here we explicitly specify the format used for parsing the revision date in our xml file. (Of course you can change this to something of your liking).
One last thing, I forgot to delete the following line from the package case:
Dictionary<string, string> package = new Dictionary<string, string>();
You can remove this as it’s not used at all, sorry about that, I totally missed that line when doing my final checkup.
Alright so that’s it for cleaing up our BABProcessor.
Refactoring BabylonContent
We’re also going to refactor our new content manager. As I was tinkering with the code for localizing other content/assets I noticed that the code for strings was doing some extra unneeded caching.
I forgot that all content we load using our Content Manager is cached for us, so we don’t need the extra caching. (This is not the first time I forgot this).
But remembering this again allows us to make our content manger much simpler.
Remove the following lines:
private Dictionary<string, Dictionary<string, string>> packages; language = ProperString(value);
Now replace the entire GetString method with this method:
private string GetString(string path, string package, string key, string packageLanguage)
{
try
{
BABFile localization = base.Load<BABFile>(path + package + "." + packageLanguage);
if (localization.keys.Keys.Contains(key))
{
return localization.keys[key];
}
}
catch (ContentLoadException e){}
//Now try to load the content using the default language
//but only if the current language isn't the default language
//else we would get stuck in ãn infinite loop.
if (!packageLanguage.Equals(DEFAULTLANGUAGE))
{
return GetString(path, package, key, DEFAULTLANGUAGE);
}
else
{
//The string searched for couldnt be found in the
//translated file or the default file.
//Replace this by an exception if you want to test your game better
return "";
}
}
As you can see this method is a lot shorter and no longer an exception is thrown if the key doesn’t exist. It also fixes another problem I had with the GetString method, you couldn’t do this:
Content.Load<String>(@”SomePath\Package.key”);
So I fixed that in this version, nothing really new here so I’m moving on to the next subject. Add a new Load method.
public T Load<T>(string assetName, string specificLanguage)
{
if (typeof(T) == typeof(String))
{
string folder = assetName.Substring(0, assetName.LastIndexOf('\\') + 1);
string file = assetName.Substring(assetName.LastIndexOf('\\') + 1);
string[] packageKey = file.Split('.');
if (packageKey.Length > 1)
{
//we convert to object and then T because we can't convert a valuetype to a (T) directly
return (T)(object)GetString(folder, packageKey[0], packageKey[1], specificLanguage);
}
else
{
return base.Load<T>(assetName);
}
}
This parses the path for us so our new GetString method works correct. Now replace the method with the signature Load(string assetName) with this one:
public override T Load<T>(string assetName)
{
return Load<T>(assetName, language);
}
Well this means we are all done refactoring. These last changes allow us to load strings (and later on other content) in a specific language, other than the language currently set. While still having our old behaviour when someone just calls the normal Load method.
The XML file, BACFile, Importer and Processor
Just as last time we are going to define an xml format, load that xml file and process it into a special class, this time called BACFile (for Babylon Content).
Remember the general idea is that you can add in new languages fairly easy. We are going to do this by adding an xml file in every folder that contains content. We call this xml file “LocalizedContent.Language.bac”. In this xml file we specify what files are localized versions of other files. Now every time our new content manager is asked to load content it first opens the “LocalizedContent.Lanuage.bac” file (where Language is the current language, note that the content manager automatically caches these files for us). If the file can’t be loaded or if there exists no key for the given asset, we just load the asset as normal. If there is a file with a key referencing the file we want to load, we get a new filename from the key, and load that one instead.
Simple huh? Let’s first define a suitable xml-file:
<?xml version="1.0" encoding="utf-8" ?>
<XNABabylonContentFile>
<version>1.0</version>
<revisionDate>3-2-2010</revisionDate>
<author>Roy Triesscheijn</author>
<language>English</language>
<!--Files that are already in the correct language can just reference themselves-->
<!--Or can be left out-->
<localization>
<key con="Texture01" loc="Texture01"/>
</localization>
</XNABabylonContentFile>
As you can see this file defines that for the content (con) “Texture01” the localized english version is “Texture01”.
Now because parsing and processing these .bac files is gonna be almost the same as processing the bab files from part1, I’m just going to put down the code here, with little comments. (You could also just download the sourcecode and copy BACFile.cs, BACImporter.cs and BACProcessor.cs to your project).
BACFile.cs
First we need a container, create a new class, mark it serializable and write code simmilar to this in the class body:
public string language;
public double version;
public string author;
public DateTime revision;
//Other than in the BABFile the BACFile
//doesnt store a direct translation but a string
//naming the translated file.
public Dictionary<string, string> keys;
public BACFile()
{
keys = new Dictionary<string, string>();
}
BACImporter.cs
Now we need an importer, create a new class called BACImporter, and just copy the contents of BABImporter into the new class file. All you need to change is the class name (obviously to BACImporter). And the ContentImporter directives (register extension “.bac” instead of “.bab”, change the display name, and set defaultprocessor to “BACProcessor”.)
Keep the method Import the same, but change the error message so we know it was a babylon content file, and not a babylon string file, that was invalid.
BACProcessor.cs
Again just copy the content of BABProcessor into a new class called BACProcessor. Now change
using TOutput = XNABabylon.Importers.BABFile;
To
using TOutput = XNABabylon.Importers.BACFile;
Change the display name to “XNA Babylon .bac Processor” . Now in the method Process change the line:
TOutput output = new BABFile();
To:
TOutput output = new BACFile();
And change the lines:
if(input.GetElementsByTagName("XNABabylonStringFile").Count > 0)
{
foreach (XmlNode node in input.GetElementsByTagName("XNABabylonStringFile")[0].ChildNodes)
To:
if (input.GetElementsByTagName("XNABabylonContentFile").Count > 0)
{
foreach (XmlNode node in input.GetElementsByTagName("XNABabylonContentFile")[0].ChildNodes)
So almost there. Locate the case package, remove it and add this new case instead
case "localization":
//Loop trough all the keys and store the name of the original content (con)
//and name of the translated file (loc) into a dictionary
foreach (XmlNode key in node.ChildNodes)
{
output.keys.Add(key.Attributes["con"].InnerText, key.Attributes["loc"].InnerText);
}
break;
Test our new additions to the content pipeline extension by creating the .bac file provided a bit above and adding it to your test project, make sure to set the Build Action to compile and to set the bac importer and processor. If everything is well, you should be able to compile without problems.
Adding new methods to our content manager
With all this new data available, we should update our content manager to use it. Go to BabylonContent.cs and let’s create a new method with the following signature:
private T GetContent<T>(string path, string file, string packageLanguage)
This method will function just like the GetString method. And will be very short tbh. All the code in the method body is this:
try
{
BACFile localization = base.Load<BACFile>(path + "LocalizedContent." + packageLanguage);
if (localization.keys.Keys.Contains(file))
{
return base.Load<T>(path + localization.keys[file]);
}
}
catch(ContentLoadException e){}
//If we got to here we couldnt load the localization file or localized content
//So try to load the unlocalized/default content.
return base.Load<T>(path + file);
As you can see we do nothing more than loading the LocalizedContent.Language.bac file in the requested folder, checking if there is a key for the requested asset, if so load that one, else, or on an exception, just load the normal one.
We can use this for all content types, but that means that all content needs to take the extra steps trough localization, which would be silly if we know in advance that some piece of content doesn’t need to be localized.
We can control it ourselves by adding this method (which we do):
public T Load<T>(string assetName, bool skipLocalization)
{
if (skipLocalization)
{
return base.Load<T>(assetName);
}
else
{
return Load<T>(assetName);
}
}
But there might be some types of files that we never want to be localized (spritefonts for example). Therefor we are going to make a list of types we do want localized.
Todo this, add the folowing list and helper functions:
//Stores all the types we want our processor to handle
//Types not in this list will just be directly forwarded to the base class
private List<Type> processable;
//'Properties' for the processable list
/// <summary>
/// Add a new type to the processable list.
/// Assets of this type will now be localized
/// when loaded (if possible).
/// </summary>
public void AddProcessableType(Type t)
{
processable.Add(t);
}
/// <summary>
/// Removes a type from the processable list
/// Assets of thys type will no longer be
/// localized when loaded.
/// </summary>
public void RemoveProcessableType(Type t)
{
processable.Remove(t);
}
/// <summary>
/// Clears all the types from the processable
/// list. No more content will be localized
/// when loaded from now on. Until new
/// types are added via AddProcessableType(..)
/// </summary>
public void ClearProcessableTypes()
{
processable.Clear();
}
/// <summary>
/// Returns an array of all types currently
/// in the processable list. Assets of these
/// types will be localized (if possible).
/// </summary>
public Type[] GetProcessableTypes()
{
return processable.ToArray();
}
This way we can adjust, at runtime, what types of files should not be localized.
In the constructor add the following lines:
processable = new List<Type>(); //Add some default types that we process processable.Add(typeof(String)); processable.Add(typeof(Texture)); processable.Add(typeof(Texture2D)); processable.Add(typeof(SoundEffect));
This way we have some sensible default types that will be localized (unless they are removed from our processable list).
All we need to do now, is adjust our Load<> function to incorporate these changes, and ofcourse we still need to add a call to our GetContent method!
Add these lines in the method with signature
public T Load<T>(string assetName, string specificLanguage)
In between the if and the else (so we get an if, else if, else).
else if (processable.Contains(typeof(T)))
{
string folder = assetName.Substring(0, assetName.LastIndexOf('\\') + 1);
string file = assetName.Substring(assetName.LastIndexOf('\\') + 1);
return (T)GetContent<T>(folder, file, specificLanguage);
}
Ah now it should all start coming togheter. Still a few checkups though. The normal (overriden) load method should look like this:
public override T Load<T>(string assetName)
{
return Load<T>(assetName, language);
}
Okay now we have 3 Load methods, let’s recap a bit what they are for.
- 1. Load(string assetName)
- 2. Load(string assetName, bool skipLocalization)
- 3. Load(string assetName, string specificLanguage)
Okay 1 is just the normal load function. This should normally be used.
Now 2 allows us to bypass all the fancy stuff and just load the file we want, without the localization overhead (for example, for stuff that doesn’t have to be localized).
And 3 is the workhorse, it is called by 1, with as specificLanguage argument, the language currently set to the application. We can aslo call it ourselves with a language we want translation to. (For example for the title screen where we want to display instructions in multiple languages).
(note that only 1 is visible if you don’t cast to BabylonContent).
Sample project and conclusions
This time I will not go into more details in how to make a test project. Please download the source code, a test project is acompanied there. It will show you two flags, and based on the flag chosen it will show the rest of the ‘game’ in that language. It will use all 3 different Load methods to demonstrate there uses.
I think it’s fair to conclude that this way of localization is fairly easy, the amount of code we need is pretty slim. And it’s hot pluggable into any existing game. Once you h ave created new localized content it’s easy to update your game to use it. (Just update the xml file). I think this method of localization has never been applied before, but I do think it’s a valid, and efficient way.
Please download the sourcode HERE.
