October 25, 2009

Players (And Designers) Learn Faster With Tight Feedback Loops

I will never be good at Starcraft. Never ever ever. It's a great game, and fascinating to think about, but playing it against real people usually makes me quit in frustration pretty quickly. On the other hand, I'm great at learning new songs in Rock Band. My brain lights up like a Christmas tree every time I play that game.

This says a lot about my brain (great at music and sequential tasks, terrible at juggling multiple simultaneous tasks), but it also says something interesting about the nature of these two games.

When I make a mistake in Rock Band, the game makes sure I know about it that very instant: A piece of the UI shatters like broken glass, the audio for the note doesn't play properly, my performance meter goes down, and my note streak is broken. On top of that, notes show up hundreds of times per minute, patterns of notes show up many times per song, and each song is short enough that it can be replayed many times in an hour.

Because of this, Rock Band is one of the easiest games to learn. I don't necessarily mean that it's an easy game to be good at, but it is an easy game to get better at. If even the worst beginner sits down and plays the same song a few times in a row, their performance will immediately start improving.

Conversely, losing a game of Starcraft can take hours, and I can never really pinpoint where I went wrong. Something very early in the game, such as making the wrong number of harvesting units, or failing to expand at the right time, or deciding to progress down the wrong tech tree, can have adverse effects much later. This problem is also exacerbated by the fact that Starcraft, like most strategy games, is a slippery slope game.

Timely information feeds learning

Games like Starcraft and Chess, which can have long periods of time between between decisions and the consequences of those decisions, are very difficult for players to learn. This shouldn't be too surprising – any book on pet training, child rearing, or romantic relationships will tell us that a delayed reward or punishment for an action is confusing.

Your dog can't tell that the treat you just gave him is for the trick he performed 20 minutes ago, and your significant other will only be frustrated when you snap at them today because of something annoying they did last week. Gamers are the same way – the more immediate the feedback to their actions, the more quickly good and bad behaviors can become reinforced.

Strategy games obviously delay their consequences on purpose, and I'm not saying Starcraft should be redesigned to be more like Rock Band. However, I do think Blizzard could add in some sort of special learning mode that would notify the player in real time of mistakes that would not normally be apparent until much later.

If beginners could see status messages like “your economy is falling behind, try producing more gatherers” or “your factory has been sitting idle for x seconds” or “your opponent is producing tier 2 units already, time to upgrade,” it wouldn't help people become experts at the psychological aspects of the game, but it could help terrible players like me start to learn from their mistakes more effectively.

Feedback loops in game development

Just as players learn much faster by seeing the immediate results of their decisions, so to do game developers. If we don't see results for too long after making decisions, it's easy to forget what the other options were, which of our arguments and thought processes were flawed, or even who was responsible for a particular decision.

Worst of all, if a game takes too long to make, employee turnover becomes a problem. Bad people get fired, and good people leave for greener pastures. If the people who witness the results aren't the same people who made the decisions, only the most disciplined designers will manage to glean any useful lessons.

Show me a game designer who has only released code to players twice in ten years, and I'll show you a mediocre designer who is learning much more slowly than they could be [unless they were constantly playtesting that whole time]. Most of the best designers I know have shipped many games in a short period of time, started out on live games where changes are released almost immediately, or worked at studios that are constantly letting people playtest their work before the game ships.

Like all designers, these people started out with a lot of wrong thinking about how to make games, but they very quickly had their bad ideas shattered and their good ideas reinforced. Just like Starcraft, success in game design is a slippery slope. The first year of your career as a game designer sets the trajectory for everything you will ever do, and if a player never even plays one of your designs until your 3rd year in the industry, you've already fallen behind.

For those of you looking to start careers in the game industry, try cutting your teeth on a live team or expansion pack team. Experienced designers usually choose to move off of these teams for the exciting new project, and as a result they are more open to newbies. Likewise, starting out by working on games with very short production cycles is another good option.

The average game project is getting very small these days, so it's a perfect time to start out as a new designer on an iphone game, flash game, facebook game, or live MMO. Shipping a lot of games now will give you the accelerated learning you need before tackling that masterpiece later.


Eolirin said...

As usual, good post.

But I think it's important to understand what you give up when you use tight feedback loops. Yes, you get much better at doing the tasks that the game presents you, but it has a side effect of depreciating the amount of effort you can/need to expend on aspects like exploration and analysis, or lateral thinking and working memory. If all you're doing is stimulus-response, that's all you end up providing for the player.

I've been reading up on brain plasticity and neuroscience a bit lately, and there's some evidence that seems to suggest that these tighter more immediate feedback loops are actually damaging toward our ability to deal with longer tasks that require greater amounts of concentration; the stimulus-response is too rapid so the timings on certain chemical releases are too close together, and the brain gets wired to expect that. When it doesn't get that, it starts looking for it.

Also, you work a very different set of skills when you're forced to figure something out as opposed to having rapid feedback that makes it incredibly clear as to what you should do at any given moment. Since the brain is competitively plastic, if you don't spend sufficient time doing certain types of activities not only do you not get better at them, you can actually get worse.

Conversely of course, if you're put into situations in which you are FORCED to do things that you otherwise would not do, you can reverse that trend. Stroke patients are often treated by having their good limbs bound so that they're forced to use the limb that they've lost functionality in, for instance; that sort of forced usage causes the brain to rewire pathways for the arm, and results in recovered function. Same thing for the skills we train through gaming, as well as pretty much everything else. If it's all stimulus-response all the time, we quickly end up being unable to do much else, but if we're forced to actually think in terms of working memory and long term analysis, we end up developing those skills. It's uncomfortable though. Underdeveloped pathways are exhausting to use. To get the greatest amount of growth, you need to not have a choice in the matter.

And this last point is kind of the crux of the problem; tight feedback loops make for better entertainment products for sure, for all the reasons you've mentioned. But if there is an option between a game that is uncomfortable to play, and one that feeds you constant amounts of endorphins at just the right intervals, it's pretty easy to figure out what you'll end up playing. Every entertainment option competes against every other, right? So there's a trend there that's potentially disturbing, and very likely inevitable.

We may need to start designing programs specifically for the purpose of working on the parts of our brain that that purely entertainment driven games ignore. Stuff like Brain Age, but intended for developing lateral thinking, analysis, exploration and working memory. Branding it as exercise is probably the only way to get people to actually put the time and effort in (And of course, not everyone will want to bother, but if it's all entertainment you've got an even harder sell due to confusion about the point; the whole "Games are supposed to be fun" thing).

Eolirin said...

Oh, oops: the disturbing trend is that for the games industry stimulus-response as a core mechanic is something of a self-reinforcing feedback loop. Games that are more accessible to more people will sell better, and the most accessible types of games are the ones that task you with lots of short quick bursts of stimulus-response. They're as you say, easier to pick up, easier to get better at, but most importantly, reward your brain more often in, and in regular intervals via chemical releases.

What sells best gets made most; and in this case having an overabundance of games that primarily pull those triggers may have wider social consequences than is really being considered.

(Games aren't really the only thing that this effects either, TV and Film's trending toward higher levels of visual and auditory stimulation is being investigated for it's impact on our attention spans too, as are things like twitter, text messaging, and rampant multitasking, and it's looking like there may be some real harm there with consistent and frequent usage.)

Brian 'Psychochild' Green said...

Eolirin's points are really interesting. It seems there's some science to back up the old people muttering about how kids these days can't concentrate on a task.

I also hesitate to apply this universally to game design. I agree that some things are helped by rapid feedback and adjustment cycles; I think my ability to balance gameplay systems was developed from being able to make rapid changes to Meridian 59 after we acquired the rights. On the other hand, another of my passions - storytelling in games, particularly multiplayer games - doesn't seem like it fits the same. I've spent a lot of time learning about storytelling and game development and really considering how these two disciplines work together. I think just trying to keep writing stories for games doesn't necessarily improve ones ability or the state of the art; in fact, the opposite could happen making a lot of sub-optimal (but still acceptable) decisions could lead to reinforcing poor habits. Story in games is currently "good enough" for most people, but a lot of people point out it's still not terribly good. This has lead a lot of people to say that story doesn't belong in games. I disagree, but I still have a way to go before I start throwing my ideas into playable form. ;)

Eolirin said...

And Brian's point brings up another thing: that sort of heavy reinforcement cycle is hard to break once you're used to it... so if you train a suboptimal set of patterns for design, it becomes very difficult to do things in a better way afterward, or rather, in *any* other way afterward. You'd need to actually erode the old system before a new way of doing things becomes viable.

The more entrenched the method, the harder it is to overcome, and rapid reinforcement builds up networks more quickly. So there is a danger in *only* doing rapid prototyping. Deep analysis and being able to foresee the consequences of your choices are also important skills to learn, and design tasks that you can't immediately get a response out of force you to rely on them more often. (And being forced to not use your strengths helps underdeveloped pathways get a chance to grow)

So that balance probably needs to exist too.

I would definitely agree that newer designers need that rapid prototyping though, if only because we don't have the same inherent understanding of the "language" so to speak. But once we have clearer idea of what our choices can mean, once we get the cause and effect, if we let ourselves stay there, we could end up stunting our potential for growth.

Mike Darga said...

Wow! Ok, that's lots of good feedback.

First off, I want to mention that I avoided the word prototyping, which to me specifically suggests doing work that will be thrown away and people won't ever see.

I think the thing that young designers benefit from is releasing the finished game out into the world, reading the forums, and reading the reviews. Just finishing things isn't enough. It has to be finished and polished and put out into the world so people can love and hate it and argue about it.

We need to realize as soon as possible how limited our own perspectives are and how quickly our “obvious” designs fall apart as soon as someone on the dev team isn't in the room walking players through it. I don't just think this is primarily valuable for character-building or hazing purposes though. My next post will be about what I think designers get out of this, but in a nutshell this is how we learn to have good instincts and judgement.

And, of course, I totally agree that there is danger in learning really well to be bad at things. Accelerated learning is a multiplier on whatever direction you were already headed in. If you're headed toward being a good designer, getting there faster is a good thing but obviously designers headed toward the other end of the spectrum get in trouble.

There are a LOT of professional game designers who are terrible at game design, but I don't think that's a reason to try and learn more slowly. If anything, hopefully they can fail faster and have some sort of realization about what needs to change. This is where mentoring is essential.

Mike Darga said...

With regard to how much people should be taught versus how much they need to puzzle through themselves, I think you both touched on my main concern: Certainly games (and career development) have to still feel satisfying in the process of teaching people, or those people will just choose to leave.

These are problems actual teachers have struggled with for decades, so I think there are some answers to be found in pedagogy. In particular I think there's a lot to learn from the ways teachers present creative endeavors such as music, art, and writing.

All my life, my teachers in these areas have told me some variation of the same advice: “You need to learn the rules, so you can know when to break them.”

The way we teach kids is to simplify the world down to some subset with an internal consistency, because those things are the easiest to learn. Even in the sciences a freshman in college gets to discover how much their high school textbooks and teachers have been coddling them.

I think the same philosophy should be applied to teaching game design or to complex games like Starcraft, chess, and go. All of these activities have “correct” or “obvious” patterns that a newbie must recognize, master, and ultimately discard in favor of more situational decision making.

Being taught the vanilla way to play Starcraft or design a game is what keeps us afloat until we gain enough expertise to start evolving on our own. The important thing about these “bridge” strategies is that they actually work sometimes.

Eventually we reach a point where our old standards stop working as well, and we need to begin analyzing what went wrong and why. We then begin to build up the incredibly complex network of exceptions to our rules, and rules about those exceptions, and even to do things explicitly “wrong” just for the sheer surprise of it.

A really great example of how to teach people to teach themselves are the exercises that people have built for chess or go. Players learn pattern recognition in an environment by doing small tasks with short feedback loops and a sense of achievement. Eventually the patterns begin to break down and players build new patterns on top of them, until one day they're experts at the game.

Hrm, I should probably just make all this its own post.

Thanks again for so much good feedback. When are you starting a blog Eo?

Eolirin said...

Huh, you know it actually hadn't occurred to me to start my own blog, I'll have to think about it.

I'm just a hobbyist but I could probably do something interesting with it if I come up with a good hook. Maybe design analysis of existing games would be a good starting point.

But to respond to the rest of that, I don't disagree with your overall point, but it's more complex than *just* being an issue of balancing between being shown the answer and having to puzzle things out... there is an actual impact in the spacing of the feedback loop on a neurological level. It's probably not going to matter all that much with regard to the feedback loop for game design, since that's a relatively long loop by neurological standards, even when you're talking live teams, but the feedback loop in something like Guitar Hero could potentially cause problems if it's used in large doses. The payoff points are so close together that the chemical releases occur too often and too rapidly.

And as for the design end of things, just to be clear...

Disclaimer: I am speaking from a position of inexperience about all this, so if I say something dumb you'll have to forgive me :).

As someone that'd qualify as a new designer, I agree that there's definitely a need for quick feedback. You need to learn the "language" so to speak, be able to get a good intuitive sense of how things work, and doing is the only good way to do that.

I just also think that certain tasks require being able to notice potential long term issues with system interactions that won't be evident in the short term, and it's harder to have quick feedback on that. You basically just have to make a prediction about it and see how it turns out.

So it's not so much that you should be trying to learn slowly, as that some of the things you need to learn can ONLY be learned with bigger loops.

Mike Darga said...

Definitely, there are lots of things a designer can't learn on a short feedback loop, and generally new designers just aren't allowed to work on things like that until later =)

Designers are very often making a prediction and seeing how it turns out, but that's exactly what I'm addressing.

The shorter the amount of time between when a designer makes a guess and when that guess pans out or not, the fresher the thought process is and the better the lesson can be learned. My next post will be about this.

Anyway, this is why I recommend new designers making small, fast games for awhile before trying to jump into a 5 year behemoth.

Eolirin said...

Yeah, I can completely agree with that. :)

Tesh said...

Not to be *too* tangential, but I've long believed that good judgment relies on learning how to make good decisions in the absence of immediate Pavlovian feedback.

There's also the moral/religious aspect of things; if we were always immediately rewarded for doing good and punished for doing evil, we don't really learn how to make decisions so much as learn how to game the reward system. People wouldn't do good because it was the right thing to do or because they have a strong moral conviction to do so, they would be doing it for the "ding", as it were.

There's not much room for personal growth on a short feedback track. Motivation is banal and shallow.

To be sure, players need to learn how to succeed in a game, as we each need to succeed in life, but it's important to have varied sizes of feedback loops. (And some things we'll never really know if we did right until the final reckoning... whatever that means to your particular religious philosophy. And yet we make choices and move on.)

Perhaps it's also interesting to consider game design where there is "One Golden Path" to success vs. those that are more subtle and/or conditional. Guitar Hero has "Do It Again, Stupid" design (thanks to Shamus of Twenty Sided for that phrase), where you can only succeed one way.

Starcraft is more flexible than that. This makes it much harder to even provide that feedback in the first place. Yes, you give examples, but sometimes, just because an enemy has upgraded to level 2 tech, that doesn't necessitate you doing the same. A Zergling/Hydralisk rush might be a better use of early midgame resources, and spending time and resources to upgrade may well cripple you.

Mike Darga said...

Making good judgement in the absence of immediate feedback is the MEASURE of good judgement, but not the method of acquiring it =)

Similarly, making a game that takes 5 years to finish is what you can do to show you know what you're doing, but not how you learn what you're doing.

"There's also the moral/religious aspect of things; if we were always immediately rewarded for doing good and punished for doing evil, we don't really learn how to make decisions so much as learn how to game the reward system."

I'd argue that this is exactly how we teach people morality. You're a dad, right? When young kids do something good or bad, you tell them about it, even if it's just with a smile or a frown.

Kids initially learn moral rules through reinforcement, many of which they must eventually break, or to choose between when they conflict. It would be impossible to navigate complex decisions if we didn't have a huge store of reinforced behavior to fall back on or at least give us a head start toward a decision.

In the Starcraft example, there are aspects of the game that are absolutely correct or incorrect, such as producing units efficiently or micromanaging units.

Even when it comes to strategy, players have to memorize some sort of passable default pattern which they can then begin to deconstruct and modify as they learn more about the times when it will break.

Tesh said...

True enough on the learning. You've got to start somewhere. Perhaps it needs to be a gradient that gradually takes off the training wheels, though.

Tangentially, have you seen these articles on Starcraft?


Mike Darga said...

Yeah, I think finding the right places to let players start figuring things out on their own would be the hard part.

I'm excited to see what the SC2 campaign is like, because I know they're focusing much more on a teaching experience this time around.

Yeah those posts are great, that's exactly what got me thinking about strategy games again =)

J Saintil said...

Sorry, it's not directly related but wondering if you know anything about the possibilities of procedural programming?

I've only a couple of really simple hypotheticals to ask. If you, or anyone could help it would be much appreciated.


Mike Darga said...

Hi J. I'm not a programmer but feel free to send me an email:


golergka said...

This discussion touches at least three essential design paradigms that I try to master for a long time already.
First, it is about the "good fail" and "bad fail". When the game lets the player to fail, it must provide him the feedback on why did he fail and how to avoid it in the future; you example about Guitar Hero and Starcraft describes it perfectly. Personally, I love playing Guitar Hero, but I never could play any multilayer RTS quite for that reason: I just didn't understand, why and where I did something wrong. (Of course, I'm an awful strategy player, but this also says something about the game). Of course (as was already said above by others) long feedback loop doesn't necessary mean "bad design", but when I'll see the player closing the game after the game over screen next time, one of the things I'll think about will surely be making the feedback loop more fast.
Second, it reminds me of the goal progression in MMO that designers use to hook up players in f2p games. First, you should kill the monster (10 seconds), then - complete the quest (1 minute), then - get the first level-up (5 minutes) and this chain progresses to months of play, making sure that you stay in the game. In a sense, these goals are positive feedbacks for your progress, which go from instant to long-term; sadly, MMO games lack decision-making that can bring you positive and negative feedack: there is usually no decisions involved, and negative consequences are very limited.
In the end, it is all about the last concept - the learning curve. We usually picture it as difficulty of the game, raising over time - but the bigger difficulty could also mean the longer feedback loops. Of course, usually the game just introduces other game systems and mechanics for longer feedback loops (you can shoot enemies? Now try managing your HP! You mastered searching for health kits? How about some RPG-like character progression! And so on) - but maybe we should think about ways to make feedbacks longer in the systems player already mastered.

Mike Darga said...

@golergka: I like the terms Good Fail and Bad Fail very much; that's a great way of describing it.

When it comes to goal progression, I really like when a game enables me to have one goal on every scale at a given moment. I should know what I'm trying to achieve during the next 10 seconds, 1 minute, 5 minutes, hour, day, week, month, and lifetime. I find that WoW is particularly good at striking that balance between reacting and planning.

I also dislike it when games lack some form of meaningful choice and failure. In general I've settled on using efficiency and opportunity cost as surrogates for true failure. Yes everybody will succeed eventually, but playing badly will mean you have to put in much more effort and time than someone who plays well. It's not ideal, but it's something.