Time to make a big listOk, we know what a black box is for and why we need one, but which information should it actually mine and process?
This brings us to some actual work. Most people, including many game developers, don't actually know what game designers do. When people say they want to be game designers, they're usually thinking of what creative directors or executive producers do. Actual game design is filled with unglamorous tasks such as generating thorough lists of things: possible problems, necessary assets, edgecases, tuning hooks, etc. (In fact, making lists is such a fundamental skill for designers that "generate a big list of things you'd need to do x" is a common question on screening tests for design applicants.)
Hopefully, you're already collecting as much data as possible about your players and how they are playing the game. Using WoW as an example, here's the data I'd want my black box to have access to:
Character DataBasic information about character demographics, advancement, and investment.
- Number of current active characters (over level 20, played more than 10 hours in the last month)
- Number of total characters
- Class breakdown of active characters
- Total /played time per character, active character, and account
- Percentage of time played by active character and by class per day/week/month
- Level, server type, spec, stats, cash, honor, reputation, arena points, etc for each character
- Comparison of all stats to an average player at their level, stats compared to average player at their level, as well as average player of their class at their level
- Guilded status, as well as guild ranking/stats/size if applicable
- Time since guilded, if applicable
- Time since unguilded, if applicable, and whether quit or kicked.
- Time since each active character has been logged onto
- Time since any character on the account has been logged onto
- Average play session length per character and per account
- Play session lengths per character and per account in the last month
- Time spent crafting per day/week/month
- Time spent in PvE instances per day/week/month
- Time spent in PvP instances per day/week/month
- Time spent in combat in the world per day/week/month
- Time spent online but not in combat, crafting, or in instances per day/week/month
- Character map coordinates at login and logout
- Combat and chat log data for 10 minutes before each recent logout.
- All player deaths, the time between them and their cause (which player/mob)
- Kill-death ratio per hour/level/day/month/lifetime, by player and by mob
- Cause and position of recent deaths where the player chose to take the death penalty
- Time at logout since last level
- Time at logout since last large cash expenditure, and what it was
- Time at logout since last pvp fight
- Time at logout since last pvp death
- Time at logout since last quest accepted
- Time at logout since last quest completed
- Most recently failed quests
- Time from acceptance to completion, for most recently accepted quest
- Time from acceptance to completion, for most recently completed quest
- Current number and names of accepted but uncompleted quests
- Percentage of quests completed for current zone
- Amount of rested xp at login and logout
- Ratio of failed attempts versus successful attempts to clear each instance/raid boss
- Time spent in PvE instances since the last successful boss takedown
- Number of deaths in PvE instances since the last successful boss takedown
- Time at logout since last loot drop of rarity uncommon/rare/epic
- Time spent in PvE instances since the last uncommon/rare/epic item successfully looted.
- Ratio of items rolled on versus rolls won
- Percentage of current tier items attained, if applicable
Battleground DataPvP data for Battlegrounds. We could also include a section about about world PvP, but since it's not emphasized in WoW we won't need any data aside from those covered in the first section.
- Total time spent in BGs for character lifetime
- Total time spent in BGs per character level
- Total honor/PvP gear tokens/enemy kills over time
- Total rounds played in each Battleground map per day/week/month
- Number/percentage of abandoned Battlegrounds per BG map
- Average ranking on each scoreboard (kills/dps/healing/captures/etc) per day/week/month
- Average difference between each score and the average teammate score/average enemy score, for all players
- Average difference between each score and the average teammate score/average enemy score, for players of the same class
- Win/loss ratio per day/week/month
- Average duration of BGs per day/week/month
- Average duration of BGs won per day/week/month
- Average duration of BGs lost per day/week/month
- Average duration of BGs abandoned per day/week/month
- Percentage of available PvP gear that player has attained for their bracket
Arena DataHow well is the player doing in ranked PvP, if they participate?
- Arena points over time
- Arena Rankings over time for 2s/3s/5s
- Arena Rankings over time for 2s/3s/5s
- Average duration of matches won/lost/both per ladder
- Average life span of character in matches won/lost/both per ladder
- Average difference in ranking between their teams and opponents per ladder
- Percentage class compositions of enemy teams for matches won/lost/both per ladder
- Average matches played per week
- Matches played per week, over time.
- Percentage of current tier arena items attained
Misc DataData pertaining to the general state of the game or other games that may be drawing players away.
- Server performance the minute/hour/day of logout
- Client's framerate the minute/hour/day of logout
- Client/Server crashes the minute/hour/day of logout
- GM tickets submitted the day/week/month of logout, and how long they took to be resolved
- Most recent patch notes at the time of last logout
- Competiting companies' new game releases or major patches the week/month of last logout
The data that I've listed here is the kind of data I suspect would reveal why a player is quitting the game. Of course if you've left some important data off the list you won't have a very effective black box, so when in doubt, mine more data than you think you need to.
Next, we'll take all the data we've gathered up and start sifting through it for clues.