Gameplay Analysis

One of the biggest hardships of today's videogame industry is the uncertainty inherent in predicting player engagement. While this wasn't as much of a problem in the bygone era of games made on-the-cheap, today's games have far too much at stake to rely on such abstract practices. Hundreds of games have floundered while millions of man-hours and hundreds of millions of dollars have been lost because of these easy-to-make mistakes.

To fix this problem, my former employer, David S. Robinson, developed what he called the Game-Play Analysis, or GPA, system. When he hired me on a few years ago, he taught this system to me. Since then, I've been working on improving the GPA. So now let me share with you some of what I know the GPA system can do.

What is this system and what makes it special? The strength of the original GPA system lies in the general-purpose nature of its utility. Most design tools I've seen have been limited to an over-specific purpose only offering insight late in the development process. By this time point repairs might not be practical. For example, heatmap systems can offer insight into how players are behaving in-game, but they can only be used in the playtesting stage. By that point, any problems discovered will cause the a lot of the previously invested effort to be wasted or the problems will be ignored at the cost of customer satisfaction.

The GPA system, in contrast, is designed to be usable in multiple different stages of the design process. It can be used in the early design phases to create a more detailed and thought-out vision of the final product. If introduced later in development, the GPA system can give more precise feedback than what can usually be obtained from playtesting alone. (In fact, the GPA tool can work particularly well in the hands of playtesters, but we'll get to that shortly. At any point in development, the GPA system can be used to predict how hypothetical design changes might effect the final game without needing to make a separate build. In some cases, it can even predict the actual monetary cost of a possible design change. Best of all, unlike most other tools in the modern game designer's toolkit, the GPA system doesn't require complicated or expensive plugin software. Instead, it only requires a spreadsheet program or even pen and paper.

So, enough hype, how does the GPA system work? At its most basic, analyzing a game using the GPA system is analogous to being the stat-keeper for a sport like basketball, (American) football, etc. A stat-keeper needs to record anytime a specific action is performed by a given player. A given player in American football, for example, has a value associated with them for the amount of yards they've rushed, passes thrown, catches made, interceptions made, etc. Each of these values can be thought of as part of a representation of the total value of that player. (In other words, a statistician might be able to discover a mathematical function that takes each of these variables and outputs an objective value of their value as a player. Like the whole “Moneyball” story.

The GPA system is very much the same. For every “game-play area” (think of them as a the units which comprise the levels of a game, the game-play analyst must record the number of various events, such as new mechanics, puzzles, or tools, that occur during the course of playing through that game-play area. The reason why these game-play areas are used instead of the levels is that levels can vary dramatically in length and amount of game-play even in the same game. On the spreadsheet, Each game-play area is a column in order from the beginning to the end of the game, with each row corresponding to a specific event type. Then, each entry for a specific variable is multiplied by a specific “interest value.” This multiplier corresponds to the level of engagement a player gets from each instance of an event. This is akin to how different types of scoring in certain sports can result in different numbers of points for the team. For example in basketball, a shot from outside the 3-point line results in 3 points while a shot inside the line is 2 points. The analyst then takes these products and adds them together to get a value for the area, called the “raw interest.”

In order to actually figure out what all this data means, the interest values for each game-play area is plotted on a regular line graph each x-coordinate is a game-play area while the y-coordinates is the interest of each area. Here's the surprising part, this game-play graph can be compared to the kind of plot diagram shown below.

The closer a game's GPA curve is to the narrative arc curve pattern, the more likely it is to be a much better game. If a particular portion of the game-curve deviates from this narrative curve more than other parts of the game, that portion will usually be predictably worse than the rest of the game. The play in these areas will either have the player bored or overwhelmed. In other words, the ideal flow/game-play progression of a game follows the same pattern as the narrative arc concept from storytelling. When you consider the fact that game mechanics and stories are both similarly vying for human attention, it makes sense that they'd both follow the same pattern.

Notice, however, that none of this actually requires an actual game to be involved. A hypothetical game is just as viable an analysis subject as a real one. In this way, the GPA system can perform some of the quality control functions of play testing before any development has occurred. The GPA system can allow game designers to iterate far more efficiently by catching many mistakes before they can suck up too much of your team's time and money.

That's the GPA system as I learned it. However, I have since been working to improve it. The GPA system David designed was specifically tailored to the games he was used to making, highly linear, single-player experiences that were light on story content. Each of these limitation presents new challenges for this system. Accommodating non-linear games requires quantifying the exponentially increasing number of possible player experience permutations. Adapting the GPA system to games that are either story-driven or driven by story and mechanics in concert would require determining the statistical value of multiple game-play events that don't exist in the old GPA system. Most recently, I've been inspired by a talk at GDC into researching how gamers with different motivations might have different event interest multipliers, a path which could lead to the GPA system allowing games to be more concisely tailored with different motivation-demographics in mind.

To accomplish these feats, I need a lot of 1 thing: data points. Like any other statistical model, the accuracy increases the more samples I acquire. Sure, I could continue to analyze lots of on-the-market games, but that does little to prove the predictive power of the GPA system and could skew my data with bias. In essence, I need to analyze games before they reach the market to ensure my data is isolated from the public opinion. So, I need your help. If you are a developer, at any stage of development, please consider letting me test your current build or running me through your planned game and letting me perform my analysis. In exchange, I'll give you detailed results of my analysis along with suggestions as to where you might want to change things. Of course, anything I learn from my testing would be kept confidential.

I'm exited about taking our proven GPA system into the future of video games to help you bring the most predictably enjoyable product to your fans.