Article: Derived Effect Classes in XNA 4

by Dave Carlile

One of the nice API changes in XNA 4.0 is the simplified syntax used when drawing with an Effect. In previous versions you would draw things like this:

effect.Begin();
foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
        pass.Begin();
        // draw your stuff
        pass.End();
}
effect.End();

Now you can draw things like this, with EffectPass.Apply handling all the magic:

foreach (EffectPass pass in effect.CurrentTechnique.Passes)
{
        pass.Apply();
        // draw your stuff
}

Related to this is the new Effect.OnApply virtual method which gets called by EffectPass.Apply. By overriding this method in a derived class you can do things like perform pre-shader calculations and set shader parameters from properties. This is how BasicEffect and the other new standard shaders work, and it would be nice to be able to duplicate that functionality in our own derived effects.
Ideally we would like to be able to do something like this:

        MyEffect effect = Content.Load<MyEffect>(“myeffect”);
        effect.World = Matrix.Identity;
        effect.Color = Color.Blue;
        ...
        effect.Apply();
        ...

The first line is where we begin running into issues. In order to load a derived class using the content pipeline you have to write your own content processor, reader, and writer, for every single class. You might think that you could load an Effect instance using the content manager, then create an instance of your derived Effect class and pass in the code. Unfortunately, there is no way to get at the code in the Effect instance (okay, that is not entirely true, but this solution requires the full Game Studio install).
If you want to store the class properties (e.g. MyEffect.World, MyEffect.Color) as content then extending the content pipeline for each class is your best option. But what if you don't need to store the properties? What if you just want a cleaner interface to your shaders? Extending the pipeline each time for these cases seems like a lot of unnecessary work.

A Solution

This remainder of this tutorial will show you how to load any derived Effect class from the content pipeline without having to extend the pipeline for each class. To accomplish this we're going to create a content pipeline extension library that will compile the effect into an intermediate object that will let us get at the compiled code. We will also create a game library that will define the intermediate class so we can get to it at run time. And finally we’ll add an extension method that will do the work of loading our derived Effect class.

Setup

Let's start by opening Microsoft Visual Studio 2010 Express for Windows Phone and setting some things up. It’s important that you name things correctly, so make sure you use the names given while you follow these steps:

• Create a new Windows Game (4.0) project, call it DerivedEffectsTutorial.
• Add a new Windows Game Library (4.0) project to the solution, call it DerivedEffects.
• Add a new Content Pipeline Extension Library (4.0) project, call it DerivedEffects.Pipeline.
• Reference the DerivedEffects game library in both the DerivedEffects.Pipeline and DerivedEffectsTutorial projects.
• Reference the DerivedEffects.Pipeline in the DerivedEffectsTutorialContent project which Visual Studio generated for you.

Make sure you can build the solution. All three projects should compile with no errors.

Extend the Pipeline

Now that everything is set up, the first thing we're going to do is extend the content pipeline so we can compile a shader in such a way as to provide us access to the compiled code at run time. This requires these steps:

• Create an intermediate class to store compiled shader code.
• Implement a ContentProcessor to compile the code and store it an instance of the intermediate class. Note that we don't need to define a content importer because we can use the existing effect importer.
• Implement a ContentTypeWriter to write out the intermediate class at compile time.
• Implement a ContentTypeReader to read the intermediate class at run time.

Intermediate Class

First we need to create an intermediate class to hold the effect code after it's compiled by the content processor. Some types of content require a different class at compile time versus run time for various reasons. But in this case the same class will work for both. Since this class is used at both compile time and run time it needs to live in the DerivedEffects game library.
Visual Studio created a Class1.cs file when we added the game library project. Right click on the file in Solution Explorer, select Rename, and change the name to DerivedEffect.cs. You should be prompted to change the name of the class to match, click “Yes”.
Now open the file. You should see the empty DerivedEffect class defined in the DerivedEffects namespace. Make the class public so it looks like this:

namespace DerivedEffects
{
        public class DerivedEffect
        {
        }
}

Now add a private member to store the compiled code.

        private byte[] compiledCode;

Add a constructor which accepts the code as a parameter and stores it in the private member.

public DerivedEffect(byte[] compiledCode)
{
        this.compiledCode = compiledCode;
}

And finally add a public property to provide access to the code.

public byte[] CompiledCode { get { return compiledCode; } }

The final class should look like this:

namespace DerivedEffects
{
        public class DerivedEffect
        {
                private byte[] compiledCode;

                public DerivedEffect(byte[] compiledCode)
                {
                        this.compiledCode = compiledCode;
                }

                public byte[] CompiledCode { get { return compiledCode; } }
        }
}

Make sure everything still compiles before moving on to the next step.

Content Processor

The content processor's job is to compile the shader code and return an instance of our intermediate DerivedEffect class. Since it is used at compile time it lives in the DerivedEffects.Pipeline project. Visual Studio created a ContentProcessor1.cs file for us when we added the pipeline extension project. Right click on the file in Solution Explorer, select Rename, and change the name to DerivedEffectProcessor.cs. You should be prompted to change the name of the class to match, click “Yes”.
Now open the file and you should see the empty DerivedEffectProcessor class defined in the DerivedEffects.Pipeline namespace. The class should look like this (with generated comments removed for brevity):

using TInput = System.String;
using TOutput = System.String;

namespace DerivedEffects.Pipeline
{
        [ContentProcessor(DisplayName = "DerivedEffects.Pipeline.ContentProcessor1")]
        public class DerivedEffectProcessor : ContentProcessor<TInput, TOutput>
        {
                public override TOutput Process(TInput input, ContentProcessorContext context)
                {
                        throw new NotImplementedException();
                }
        }
}

The Process method takes an input type and returns out output type. The input type in our case is the EffectContent class which is output by the default effect importer (this is the “Effect - XNA Framework” content importer you see when you select an .fx file in solution explorer and look at the file properties).

The output type is the intermediate DerivedEffect class we defined earlier.

Change the TInput and TOutput aliases to the fully qualified names of our input and output types.

using TInput = Microsoft.Xna.Framework.Content.Pipeline.Graphics.EffectContent;
using TOutput = DerivedEffects.DerivedEffect;

Alternatively you can remove the aliases and replace all instances of TInput with EffectContent, and all instances of TOutput with DerivedEffect.

Change the ContentProcessor attribute to give the processor a better name. This is the name that shows up in the Content Processor property when you select an .fx file in Solution Explorer.

[ContentProcessor(DisplayName = "Effect – Derived Effects")]

Now, replace the contents of the Process method with this code to compile the effect.

EffectProcessor compiler = new EffectProcessor();
CompiledEffectContent compiledContent = compiler.Process(input, context);

This creates an EffectProcessor instance, passes the EffectContent input to the Process method to compile the effect, and returns a CompiledEffectContent instance which will allow us access to the compiled code.
Now we need to create an instance of our intermediate content class, passing it the compiled effect code – remember TOutput is the alias we defined previously for our DerivedEffect class.

return new TOutput(compiledContent.GetEffectCode());

And that's it for the content processor. The final code should look like this (again, with the generated comments removed):

using TInput = Microsoft.Xna.Framework.Content.Pipeline.Graphics.EffectContent;
using TOutput = DerivedEffects.DerivedEffect;

namespace DerivedEffects.Pipeline
{
        [ContentProcessor(DisplayName = "Effect – Derived Effects")]
        public class DerivedEffectProcessor : ContentProcessor<TInput, TOutput>
        {
                public override TOutput Process(TInput input, ContentProcessorContext context)
                {
                        EffectProcessor compiler = new EffectProcessor();
                        CompiledEffectContent compiledContent = compiler.Process(input, context);
                        return new TOutput(compiledContent.GetEffectCode());
                }
        }
}

Now make sure everything still builds before continuing on. If there are problems, double check that you've set up the references correctly.

Content Writer

Next up is the content writer. This class is responsible for writing an instance of the DerivedEffect class to an .xnb file during compile time. Since the content writer is executed at compile time, where does it go? That's right, the pipeline extension library. Right click on the DerivedEffects.Pipeline project and select Add/New Item. Make sure XNA Game Studio 4.0 is selected in the Installed Templates list on the left, then choose Content Type Writer, name it DerivedEffectWriter.cs, and press the Add button.
In the DerivedEffectWriter class, start by changing the TWrite alias to our DerivedEffect class.

using TWrite = DerivedEffects.DerivedEffect;

Remove all of the code from the Write method and replace it with this:

output.Write(value.CompiledCode.Length);
output.Write(value.CompiledCode);

These two lines write out the length of the compiled code array, followed by the code itself. Later on when we implement the content reader it will be able to read in the length first so it knows how many bytes of code to read.
Next, replace all the code in the GetRuntimeReader method with this:

return typeof(DerivedEffectReader).AssemblyQualifiedName;

This line returns the name space, class name, and assembly name for the DerivedEffectReader class. It basically tells the pipeline which content reader to use when it needs to read in our DerivedEffect class.
Note that we haven't defined our content reader class yet, so there will be errors if we try to compile at this point. That will be taken care of next, but first, here is what the final content writer should look like:

using TWrite = DerivedEffects.DerivedEffect;

namespace DerivedEffects.Pipeline
{
        [ContentTypeWriter]
        public class DerivedEffectWriter : ContentTypeWriter<TWrite>
        {
                protected override void Write(ContentWriter output, TWrite value)
                {
                        output.Write(value.CompiledCode.Length);
                        output.Write(value.CompiledCode);
                }

                public override string GetRuntimeReader(TargetPlatform targetPlatform)
                {
                        return typeof(DerivedEffectReader).AssemblyQualifiedName;
                }
        }
}

Content Reader

We can compile an effect and write the code out to an .xnb file. Now we need to be able to read it back from the file, which is the content reader's job. The content reader is used at run time, which means it goes into the game library. Right click on the DerivedEffects project and select Add/New Item. Make sure XNA Game Studio 4.0 is selected in the Installed Templates list on the left, then choose Content Type Reader, name it DerivedEffectReader.cs, and press the Add button.
Change the TRead alias to our DerivedEffect class.

using TRead = DerivedEffects.DerivedEffect;

Now replace the contents of the Read method with this code:

int size = input.ReadInt32();
byte[] code = input.ReadBytes(size);

Recall that the content writer first wrote out the length of the code, then the code. The reader first reads in the length of the code so it knows how many bytes to read, after which it proceeds to read that many bytes into an array.
And finally, the read method needs to return a new instance of our DerivedEffect class – remember that TRead is the alias we defined for our DerivedEffect class.

return new TRead(code);

The final content reader looks like this:

using TRead = DerivedEffects.DerivedEffect;

namespace DerivedEffects
{
        public class DerivedEffectReader : ContentTypeReader<TRead>
        {
                protected override TRead Read(ContentReader input, TRead existingInstance)
                {
                        int size = input.ReadInt32();
                        byte[] code = input.ReadBytes(size);
                        return new TRead(code);
                }
        }
}

Now you should be able to compile successfully again. Make sure that is the case before moving on.

Testing the Content Pipeline Changes

That brings us to the end of the content pipeline extension. We've created a content processor to compile the effect, a content writer to save it out to an .xnb file at compile time, and a content reader to read it back in at run time.
Allegedly we can now compile an effect file and load it into a DerivedEffect instance at run time using the content manager. Let's make sure everything is working so far.
To start with we need an .fx file. Right click on the DerivedEffectsTutorialContent project and select Add/New Item. Choose Effect File, name it MyEffect.fx, and press the Add button. You don't need to change anything in the file.
Select the file in Solution Explorer. Find the Content Processor property in the file properties; click the down arrow to show the available content processors. You should see our new content processor “Effect – Derived Effects” - select it. If you don't see it, verify that you've correctly referenced the DerivedEffects.Pipeline project in the DerivedEffectsTutorialContent project (refer back to the Setup section for instructions on setting up all of the references).
Open the Game1 class in the DerivedEffectsGame project. Add a using statement.

using DerivedEffects;

Declare a new member in the Game1 class.

DerivedEffect myEffectCode;

And a line to the LoadContent method in order to load the derived effect.

myEffectCode = Content.Load<DerivedEffect>("myeffect");

Now comes the big moment – compile the project. If everything compiles then the compile time part of the content manager is working correctly. If not, you'll need to go back through the tutorial and find out what you missed.
Now we need to test the run time – go ahead and run the game. If it works you'll see the beautiful cornflower blue window appear. If the content manager has a problem, such as not being able to locate the content reader for DerivedEffect, it will throw an exception.
You can also set a breakpoint and examine myEffect.CompiledCode to see that it is indeed loaded with what you can assume is compiled shader code.
At this point we could create an Effect object by passing the CompiledCode to the Effect constructor, like this:

Effect effect = new Effect(GraphicsDevice, myEffect.CompiledCode);

And that is why we've done everything we have up to this point – the need to get access to the compiled code so we can pass it to the Effect constructor. If the existing Effect class would expose a property like this we could do what we're trying to do without involving the content pipeline at all. I'm sure the powers that be have a good reason for not exposing it, but it makes things a little awkward.
Now we need to work on loading our own effect descendent. Based the previous line of code, you've probably already figured out what we need to do.

Derived Effect Class

Let's start by creating our derived effect class. Right click on the DerivedEffectsTutorial game project, select Add/Class, name it MyEffect.cs, and press the Add button.
You'll need to add a couple of using statements.

using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;

You'll also need to have the class descend from Effect, and create one of the Effect constructors. It should look like this:

using System;
using Microsoft.Xna.Framework;
using Microsoft.Xna.Framework.Graphics;

namespace DerivedEffectsTutorial
{
        class MyEffect:Effect
        {
                public MyEffect(GraphicsDevice graphicsDevice, byte[] effectCode)
                        : base(graphicsDevice, effectCode)
                {
                }
        }
}

Now back in the Game1 class, add a new class member.

MyEffect effect;

And add a line to LoadContent to create a MyEffect instance.

effect = new MyEffect(GraphicsDevice, myEffectCode.CompiledCode);

The LoadContent method should look like this:

protected override void LoadContent()
{
        myEffectCode = Content.Load<DerivedEffect>("myeffect");
        effect = new MyEffect(GraphicsDevice, myEffectCode.CompiledCode);
}

And there you have it, an instance of your very own shader class derived from Effect! It will work anywhere you would normally use an effect. You can override OnApply and have it do things when EffectPass.Apply is called, add your own properties, and so on.
But, now we need to kick it up a notch.

BAM!

In the beginning we decided we'd like to be able to do this.

MyEffect effect = Content.Load<MyEffect>(“myeffect”);

Well, we can't do that using this solution, but how about this instead?

MyEffect effect = Content.LoadDerivedEffect<MyEffect>("myeffect");

We can accomplish this by adding an extension method to the ContentManager class, so let's get started.
Add a new class to the DerivedEffects game library, call it ContentManagerExtensions. Make sure the class is static and public.

public static class ContentManagerExtensions
{
}

Add some using statements.

using System.Reflection;
using Microsoft.Xna.Framework.Graphics;
using Microsoft.Xna.Framework.Content;

Add the LoadDerivedEffect extension method to the class. Notice the use of generics, and that the allowed classes must be derived from Effect.

public static T LoadDerivedEffect<T>(this ContentManager content, string assetName)
  where T:Effect
{
}

Inside the function, we need to be able to get to the graphics device. We can do that by finding the graphics device service.

IGraphicsDeviceService graphicsDeviceService =
    (IGraphicsDeviceService)content.ServiceProvider.GetService(typeof(IGraphicsDeviceService));

This uses the content manager's ServiceProvider property to look for the graphics device service.
Next we load the compiled effect code.

DerivedEffect effect = content.Load<DerivedEffect>(assetName);

This is the same code we used before to load the DerivedEffect instance, but we're now using the asset name that was passed to LoadDerivedEffect.
Now comes the interesting part. It would be nice if we could do this:

return new T(graphicsDeviceService.GraphicsDevice, effect.CompiledCode);

Unfortunately, you can't do that with generics – you can only create objects with parameterless constructors. But, we can use reflection to find and call the constructor we want.

Type classType = typeof(T);

ConstructorInfo classConstructor = classType.GetConstructor(
                BindingFlags.Instance | BindingFlags.Public, null,
                new Type[] { graphicsDeviceService.GraphicsDevice.GetType(),
                        effect.CompiledCode.GetType() },
                null);

This code first gets the type for the generic class (e.g. MyEffect). It then uses that to locate the constructor that takes GraphicsDevice and byte[] parameters. Once we have that we can create and return and instance of the class by calling the constructor and passing the graphics device and compiled code.

return (T)classConstructor.Invoke(new object[]
    { graphicsDeviceService.GraphicsDevice, effect.CompiledCode });

The final code should look like this:

namespace DerivedEffects
{
        public static class ContentManagerExtensions
        {
                public static T LoadDerivedEffect<T>(this ContentManager content, string assetName)
                        where T : Effect
                {
                        IGraphicsDeviceService graphicsDeviceService =
                                (IGraphicsDeviceService)content.ServiceProvider.GetService(
                                        typeof(IGraphicsDeviceService));

                        DerivedEffect effect = content.Load<DerivedEffect>(assetName);

                        Type classType = typeof(T);
                        ConstructorInfo classConstructor = classType.GetConstructor(
                                BindingFlags.Instance | BindingFlags.Public, null,
                                new Type[] { graphicsDeviceService.GraphicsDevice.GetType(),
                                        effect.CompiledCode.GetType() }, null);

                        return (T)classConstructor.Invoke(new object[]
                                { graphicsDeviceService.GraphicsDevice,
                                        effect.CompiledCode });
                }
        }
}

Now to make sure it's working, go back to the Game1 class, remove the DerivedEffect member and replace all of the code in LoadContent with:

effect = Content.LoadDerivedEffect<MyEffect>("myeffect");

The project should compile and run with no errors, and you now have an instance of your derived effect class.

What Next?

The downloadable code included with this tutorial extends the example with a more fully implemented MyEffect class, including OnApply and pre-shader calculations, as well as some drawing code that uses it. Going through all of that here would make a long tutorial prohibitively long.
It was mentioned earlier that the DerivedEffects and DerivedEffects.Pipeline libraries are reusable. All you need to do in your new game projects is reference DerivedEffects.Pipeline in your game's content project, and DerivedEffects in your game project, and make sure the run time library is distributed with your game. That's all there is to it.

Finale

This was a lot of work to do something that should be simple. If the Effect class exposed the compiled effect code this would boil down to just the content manager extension method. But since it doesn't, we can leverage the content pipeline to help us out by providing the data we need.
Implementing derived effects can really improve code readability and reuse, especially with the latest Effects API changes. Hopefully this method of loading your derived classes through the content pipeline makes it that much easier to use them. Happy sub-classing!

Download sample code: DerivedEffectsTutorial.zip (license)