February 28, 2010

The Prisoner's Dilemma, The Striatum, And The Dragonmaw Shinbones

I just finished rereading Satisfaction, which is one of my favorite psychology books (and therefore one of my favorite game design books). I really recommend reading it yourself, but here's one of the theses in a nutshell: Greg Berns believes that the Striatum is the region of the brain that controls how satisfied we feel, and that it is best activated by unpredictable stimuli.

Have you heard of the Prisoner's Dilemma? It's a scenario in game theory that examines how willing people are to make themselves vulnerable to betrayal:

Two suspects are arrested by the police. The police have insufficient evidence for a conviction, and, having separated both prisoners, visit each of them to offer the same deal. If one testifies (defects from the other) for the prosecution against the other and the other remains silent (cooperates with the other), the betrayer goes free and the silent accomplice receives the full 10-year sentence.

If both remain silent, both prisoners are sentenced to only six months in jail for a minor charge. If each betrays the other, each receives a five-year sentence. Each prisoner must choose to betray the other or to remain silent. Each one is assured that the other would not know about the betrayal before the end of the investigation. How should the prisoners act?
If both prisoners implicate each other, both will be punished. If both "cooperate," they will each be punished minimally. If one tries to cooperate but the other betrays them, the betrayer will go free, but the cooperator will be punished severely.

By remaining silent, a prisoner opens the possibility to not be punished at all, but must also willingly make themselves vulnerable to the worst punishment if they are betrayed.

Uncertainty can be very satisfying

In the last chapter of the book, Berns returns home after traveling the world to apply everything he has learned to his marriage. While examining the psychology of relationships and the temptation of infidelity, he examines the effects of the Prisoner's Dilemma on the Striatum:

In a brain imaging study of the prisoner's dilemma, Jim Rilling, a postdoctoral student in my department, found that parts of the striatum were activated when people cooperate. Given the close relationship of the striatum with reward and action, he naturally concluded that social cooperation is rewarding to the human brain. But it was not the act of cooperation alone that activated the striatum; it was mutual cooperation.

By design, mutual cooperation does not always result in the best outcome for each participant, not only because cooperation entails risk but also because it depends on making yourself vulnerable, which, in turn, creates opportunities for betrayal.

Whether it is a romantic relationship or a business deal, cooperation means uncertainty. When you do cooperate, and the act is reciprocated, the novelty of this outcome is picked up by the striatum. Perhaps that is the reason mutually reciprocated acts feel so good. The fact that cooperation doesn't always happen is exactly why it is so satisfying.

If you're like me, it's not hard to think of lots of moments from games when people were supposed to be cooperating but didn't: people getting the team killed in MMOs, Spies in EVE, players in shooters griefing the hell out of their teammates, etc.

What reading this quote really brought to mind for me, though, was a moment in World of Warcraft when an enemy and I managed to cooperate [I played on a PvP server]. As Bern predicted, it was very satisfying and has became one of my favorite memories from that game.

Dragonmaw Shinbones

At around level 30 in WoW, players of the warrior class are sent on several quests to gather resources which NPCs then turn into set of Brutal Armor for them. By level 30, most areas players quest in are contested zones, meaning that on PvP servers players often run across players from the enemy faction and have to fight.

Blizzard is careful to make sure that quests lead players from opposite factions to the same places at around the same level, to facilitate this conflict. One such place is the Angerfang Encampment, an out of the way location most people would never go except for the few quests that lead there (one of which being the quest for the Brutal Armor).

I was sent to the encampment for some Dragonmaw Shinbones, and happened to get there at the same time as a warrior from the other faction, presumably on the same quest. We traded deaths back and forth a few times, until ultimately one of us accidentally aggroed the local miniboss, whose army killed both of us. I don't remember whose idea it was, but one of us started to try and call a truce so that we could work together.

At this point I should mention for those that haven't played WoW that Blizzard does a great job of making the two factions feel at odds. Words typed by one faction are garbled to the other, and players on PvP servers aren't allowed to have characters on more than one of the factions. So the only way the other warrior and I could begin to form a truce and kill the boss was to use lots of emotes such as /bow /point /wait /sorry /ready etc.

We spent the next hour fighting enemies in the quest zone for our shinbones, and killing the miniboss as he respawned. We had to learn some new ways to fight, because all of our AoE powers would affect each other as well as the mobs. It was hilarious and terrifying when we died because I accidentally sent him running into a crowd of enemies. At one point, more players from his faction showed up and he even stopped them from killing me.

It was the only in an online game that I felt I'd had a reason to form an alliance with an enemy. That warrior and I probably killed each other hundreds of times after that in the battlegrounds without even noticing, but that temporary alliance was a great experience that shook up my expectations and made me feel as though I was part of a real world for a little while. I'm sure this sort of thing must've happened in multi-faction PvP games such as DAoC and Shadowbane, but I never played either of those very seriously.

Surprising doesn't have to mean punishing

There are actually lots more examples of this kind of thing in WoW, which is surprising because it's known for being so accessible. Being surprised in an online game doesn't necessarily imply having all your possessions stolen or your corpse camped for hours at a time. Sometimes it means players or a giant monster invading your city, or a bunch of level 1 players ganging up on a level 25. If you ask players if they want to be surprised, they (and I) will almost always say no, but the truth is we like the way it feels from time to time.

WoW is a game that's very good at leaving loopholes for interesting experiences while still being a "casual-friendly" game. These kinds of shenanigans don't really hurt anything, and are usually pretty easy to avoid if you're not in the mood. WoW could probably be better about making important areas like auction houses inaccessible to enemies, and the zombie event should probably have never happened on PvE servers, but PvP Servers in WoW usually manage to feel chaotic and dangerous without making player feel as though they have lost control over their playtime. This is something worth emulating.


Mike Darga said...

Hrm ok, maybe that level 1 gnome invasion was a bad example, since it apparently crashed some servers.

Eolirin said...

That's actually not quite right on the Prisoner's Dilemma; it's not that they're opening themselves up to the more severe punishment by confessing, it's that confessing is a mathematically superior outcome; if the other person doesn't also confess, you get off completely free as opposed to having a 6 month sentence, and if they *do* confess, you get a reduced sentence compared to what would happen if you stay silent. You want to confess, because confessing is the winning move. Of course, the other person also knows this, and so what will tend to happen is that you'll have both people confess, which is sub-optimal for the group (12 months combined is a lot better than 10 years combined).

It's a contrived situation in which what's optimal for the individual is not optimal for the group, and it shows that in such circumstances, the individual will tend to act in a way that causes harm to the group by maximizing for himself.

I also have had very annoying experiences on WoW PVP servers, to the point that I refuse to play on them. Stuff like getting killed by level 60 rouges repeatedly while being level 10. But then, I have my own set of experiences from the UO days that are similar. I think there's something to be said though, in the story you bring up, for the degree of difficulty in actually cooperating. The harder it is to expect it, the more it pays off if it happens; the magnitude of the surprise is important.

But as you generalize it out even further, there are some interesting ramifications to the whole thing, as it suggests an inverse relationship between scarcity of events and the strength of those events. That means if we want something to have a strong impact, the mechanics need to make it difficult for that event to occur. And that's very counter-intuitive from a more typical usability viewpoint; usually you want to make the things you want your players to be experiencing *easier* not harder to accomplish.

Eolirin said...

Oh, btw, I'd recommend this book if you're interested in the neurological underpinnings of cognition and emotion: http://www.amazon.com/Synaptic-Self-How-Brains-Become/dp/0142001783/ref=sr_1_1?ie=UTF8&s=books&qid=1267430189&sr=8-1

It's a good primer for the current state of the field, but as a result it's not going to be as directly applicable to design as Satisfaction, I think, since it tends to cleave more toward the physiology than the psychology (There's a lot of time spent on neurotransmitters and neural circuits and how plasticity works, though it does cover a lot of cognitive/emotional/motivation related stuff as well. The section on depression and anxiety disorders was especially interesting on a personal level).

But the general concepts are definitely applicable in a broader sense, and it's interesting besides.

Anderoth said...

You'll never be able to program away people's inclination to go for the Nash Equilibrium (heartless... logic... too strong...), best to try and avoid any such situations :)

When are you posting again, I discovered your blog about three weeks ago and can't wait for the next installment! Finally, some game design analysis and commentary that holds some intellectual water.

Hao said...

I have played World of Warcraft for almost 5 years, and I agree with the idea about “Uncertainty can be very satisfying”. Players from opposite factions are cooperating for the same goal. It was very satisfying and has became one of my favorite memories from that game. I think most of all wow players remember “The Opening of the Gates of Ahn'Qiraj”. The players from opposite factions do the quests together, because all of the players want open the gates of TAQ as soon as possble. Horde and Alliance were supposed to be fighting but didn't. At that time, there is no killing and fighting, just cooperating. I think Blizzard does a great job of making the player surprise. Player managed to cooperate with enemy and you can not chat with them, so the only way to make a communication between two factions is to use lots of emotes as well as increase the risk of cooperation. So, as the author saids mutual cooperation does not always result in the best outcome for each participant, not only because cooperation entails risk but also because it depends on making yourself vulnerable, which, in turn, creates opportunities for betrayal. This is also the attraction of world of warcraft.

Ysharros said...

Am figuring you get emails on comments.

Is everything okay? Just checking in, this place has been too quiet too long! Hope it's a GOOD quiet.

Anonymous said...

I'm a little curious about just one thing, or rather, one application aspect of this article.

How do you encourage players to cooperate when their default behavioural pattern as assumed by the game is going to be competition?

Your example story was tremendously compelling, but is very much the exception; is there a way to design a game such that it overtly encourages fractiousness while covertly encouraging cooperation, thus more frequently leading to rewarding experiences like the one you described?

The Student said...

I like the psychological approach to game design that you've taken. Your posts have been entertaining during my long car rides (I wasn't driving!). You should check out my game design site - we post at least a few game design articles a week. It's http://www.dtwgames.com. I'd like to hear your thoughts.

imsmart said...

Although personally I hate psychology, I definitely concede that gamers love to be surprised. For example, the original Super Mario Bros. was chock full of artificial rules created solely to surprise the gamer down the road by being suspended at some later points to create little surprises; like the bridge levels where green paratroopas fly differently than usual, or the single black & white level thrown in, to pick two examples.

Lyndon Parry said...

Humans are more satisfied with many forms of unpredictable stimuli. Movies that have that "shocking twist" at the end generally have more impact than those that don't. Games need unpredictability, but the outcome should be measureable. In the Dragonmaw Shinbones scenario you mentioned, both of you knew your reasons for being at the encampment and the outcome of the quest but on the other hand neither knew you would both run into each other and have to work together in order to reach the goal. I think knowing the destination but not the journey is what makes games fun.

I might also have to get my Game Design lecturer to look up Satisfaction. We've spent a lot of time in class discussing 'Flow' (google Mihaly Csikszentmihalyi) and it seems like it would be some very interesting reading. As you eluded to, Games Design involves a lot of Psychology.