Sgt. Conker We are "absolutely fine"

Article: Input Playback

by Jesse Chounard

Lately I’ve noticed a trend in games to allow players to save and play back recordings of their play upon completion of a match.  This allows players to analyze their mistakes, as well as learn new tactics and techniques from their opponents.  This feature is becoming common in all sorts of games.  You can find it in real time strategy games like StarCraft, first person shooters like Halo 3, and fighting games like Super Street Fighter 4.

In many cases, these gameplay recordings are often uploaded to a server on the internet, and are available for download by anyone.  The recordings don’t require buffering or waiting for a big download to finish.  Instead, the playback just starts right up.  How is this possible?  Great looking high definition video files are gigantic.  Also, video compression takes a huge amount of computation, so wouldn’t it slow the gameplay down to record everything?

It turns out that there’s a really simple trick.  We’re not recording video.  Instead, we just store all of the user input, and then later we can use those inputs to exactly recreate the gameplay in our game engine.  So while it looks like we’re watching a stored video, instead we’re just watching the computer playing the game using our stored inputs instead of using the game controller for input.

In this tutorial, I’ll show you a simple method that you can use to achieve gameplay recording and playback, and also list a few other uses for this technique.

What exactly do we need to record?

1.    The most obvious thing to record is the input from the players.  The standard Xbox 360 gamepad has fourteen digital buttons (4 D-Pad directions, A, B, X, Y, LB, RB, Start, Back, and pushing down on either analog stick), and six analog inputs (X and Y for both analog sticks, and left and right triggers.)
2.    If our game runs with a variable time step, we need to record the amount of time that has elapsed since our last recording.  This is simple enough, since the XNA framework provides this in the GameTime class that’s passed in to the default Update method.

Let’s take a look at how we might manage this data in our code.

Once per frame, we need to store a flag to tell us whether each of our digital buttons is held down or not.  We could store a boolean for each of the buttons, but it’s probably better to pack them down a bit more.  We can use an enumeration, declared with the Flags attribute to contain all of our digital button states.

[Flags]
public enum ButtonFlags: short{
        None = 0x0000,

        Up = 0x0001,
        Down = 0x0002,
        Left = 0x0004,
        Right = 0x0008,

        A = 0x0010,
        B = 0x0020,
        // etc.
}

Then it’s a pretty simple matter to set or check a button’s status.

        ButtonFlags buttonFlags;

To set a button pressed:

        buttonFlags |= ButtonFlags.A;

To check for a button press:

if((buttonFlags & ButtonFlags.A) != ButtonFlags.None)
{
         // button A is pressed
}

All of the analog inputs on the Xbox 360 gamepad are reported with a single floating point number.  We can also represent the amount of elapsed time in a float.  If we put all of those together in a structure, we arrive at something that looks like this:

public struct InputState
{
         public float FrameTime;
         public float LeftStickX, LeftStickY;
         public float RightStickX, RightStickY;
         public float LeftTrigger, RightTrigger;
         public ButtonFlags InputFlags;
}

And that structure is all we need to store to represent an entire frame.  That’s a whole heck of a lot smaller than a frame of video, especially in high definition.  And you can make it even smaller if you don’t need all of the inputs for your game, or if you can treat the analog inputs as digital ones.  (If you only care if a trigger is pushed, but not to what degree it’s depressed, you can move it in with the digital buttons.)

Now we need to tie recording in to our gameplay.  At the start of each Update method (once per frame), fill out an InputState, and store it away.  Then update your game normally.  If you can move your update code to a new function that takes an InputState as a parameter, it will simplify things when it comes time to play back the data.

Here’s what our Update method might look like:

protected override void Update(GameTime gameTime)
{
        if (currentState == GameState.Recording)
        {
                InputState inputState;

                SetInputState(out inputState,
                (float)gameTime.ElapsedGameTime.TotalSeconds);
                SaveInput(inputState);
                UpdateGame(inputState);
        }
}

SetInputState just checks each button, and stores whether or not it’s pressed.  SaveInput would store the information to disk, or to a temporary buffer where we could read it later.  And UpdateGame might look like this:

void UpdateGame(InputState inputState)
{
        if ((inputState.InputFlags & ButtonFlags.Left) != ButtonFlags.None)
        {
                // move player left
        }
       else if ((inputState.InputFlags & ButtonFlags.Right) != ButtonFlags.None)
        {
                // move player right
        }
}

And that’s it for recording.  Nice and easy!  Now how about the playback?  Well, that’s pretty easy too.

We’re going to need a couple of variables to keep track of where we are in the play back, and how much time has passed since we started playing.

float recordingTime = 0, playbackTime = 0;
int playbackPosition = 0;
float playbackTimeScale = 1;

The first two (recordingTime and playbackTime) are used to compare how much time has elapsed in playback versus how much time elapsed in the original recording.  We can compare them to determin when we need to move to the next frame.   playbackPosition is the frame index in the recording.  playbackTimeScale determines how fast we play back the recording.  We’ll play around with that in a second.

protected override void Update(GameTime gameTime)
{
        if(currentState == GameState.Recording)
        {
                 // see above
         }
         else if (currentState == GameState.Playback)
         {
                 playbackTime += (float)gameTime.ElapsedGameTime.TotalSeconds * playbackTimeScale;

                 while (playbackTime >= recordingTime)
                 {
                         InputState inputState;
                         ReadInputState(out inputState, playbackPosition)
                         playbackPosition++;
                         recordingTime += inputState.FrameTime;
                         UpdateGame(inputState);
                 }
        }
}

And that’s all it takes to play back our recordings.  It’s truly very simple.  If you play around with the playbackTimeScale variable you get really cool slow motion or fast forward.  (0.5f would move at half speed, where 2.0f would be double speed.)

In the included sample application, you can switch between recording and playback mode using F1 and F2 on the keyboard.  You can use the gamepad D-Pad or the keyboard arrow keys to move around.

But what else can we do with this technique?

1.    You could implement a training mode, and use a recorded playback to show the player how to use a new technique.
2.    You could use a playback in a puzzle game to show the puzzle solution if the player gets stuck and gives up.  (This was used in Professor Fizzwizzle, one of my favorite indie games.)
3.    This could be a big help in debugging.  If you find a way to crash your game, but it’s difficult to repeat, you could record the inputs and get exactly the same inputs every time.  Also, if you have very fast gameplay and you’re having trouble seeing what’s going wrong, you can slow the playback time down and see everything in super slow motion.
4.    If you’re making a gameplay trailer, but don’t have enough horsepower in your PC to capture the video (with an application like Fraps) without slowing down the game too much, you could use a playback to help.  Advance the playback 16.66 milliseconds per frame, and write out a screenshot.  Repeat that until the end of the playback.  Then you can use an external program to stitch all of the screenshots together into a 60 frame per second movie.  (Alec Holowka used a technique like this to make the HD trailer for Aquaria.)

That pretty much covers everything.  Please leave a comment if anything here is unclear or if you see a way this could be improved.  I’d love to hear what interesting uses you come up with for your playbacks.

Download sample code: PlaybackSample.zip (license)

About Absolutely Fine Tutorial Contest

Look out for the complete list of entries in our tutorial contest! Coming soon (it's being built step by step)!