How can we wrap our heads around the chaos of game development? By understanding that the famous phrase “find the fun” implies something important: discovery. How do you manage the creative process? By acknowledging the latter word of the phrase: process. If you can understand how those terms related – and where they differ – you can appreciate something vital to effective production. That nothing we do in game development is completely devoid of process. And, if you can learn to separate the process from the discovery, then science becomes a weapon against the dark forces of development hell.

---

By Reading This Post, You Will Learn

• The Discovery versus Process framework for understanding your activities
• Why discovery is the source of much of our management woes in game development
• How scrutinizing and improving our processes can help us better contend with the risk of discovery
• The contents of the various parts of the “Game Planning With Science!” series

---

Discovery Versus Process: An Preface to “Game Planning With Science!”

By the best estimates of paleontologists, our genetic predecessors, the hominins, discovered how to make fire approximately 350,000 years agoº. By the time our homo-sapien ancestors entered the picture about 150,000 years later, that discovery, through repetition and iteration, became something else: process. An understood sequence of steps that produced an predictable result.

This observation leads to another: that all human activity falls along that same spectrum. On one extreme, you have activities that are pure process. Boiling water, for example. There’s no mystery there. You apply a certain amount of energy to a given amount of water and it will boil. On the other extreme end is discovery: the unknown. There lies the bleeding edge of science, the depths of space, and new forms of expression.

This spectrum has a few dynamics that are worth recognizing. First, there are few extreme examples of process or discovery. Most activities represent a balance between the two. Cooking for example. Many elements of cooking – fundamental combinations of ingredients, for instance – represent forms of process. But other forms, such as specific temperatures, timings, or novel ingredients, yield discovery.

The second dynamic to recognize is all activities start as discovery and, over time and repetition, move closer to process. This applies both globally and personally. Someone had to be the first person to create pasta sauce, but now it’s commonplace. Likewise, the first time you cooked your own pasta sauce from scratch, it probably involved a lot of discovery. But, by the hundredth time, it was probably closer to process.

Discovery is the Soil in which We Labor

As game developers, much of what we do is discovery. Hell, the phase “Find the fun” even has the word FIND in it. And that is the source of much of the pain we experience. Because discovery, by it’s nature, results in uncertain outcomes. And uncertainty creates risk. Or, as the data scientists call it, variance.

Because that’s what risk is: variance from an expected outcome. If you jump into the ocean with a cinder block tied to your ankles there’s no risk: you are definitely going to die. It’s not risk, it’s certainty. If jump into the ocean with the cinder block tied to your ankles and a knife in your hand,  that’s risk. The spectrum of potential outcomes is far more varied.

The magnitude of the risk is the measurement of width of the spectrum of outcomes. An investment of $5 that could yield $10 in the best case has a fairly narrow variance: best case you’re up $5, worst case your down $5. An investment of $10,000 that could yield $20,000 has a much greater variance (-$10,000 to +$10,000) and thus a much greater risk.

So, if discovery produces so much grief because of variance, should we abandon it? Just stick to making games and features we’ve made in the past? Of course not. Discovery is what makes our industry great. It’s what makes game development rewarding.

Taming The Beast of Variance

But neither do we need to resign ourselves to being at the mercy of variance – if we accept two concepts.

First, variance compounds over subsequent activities. If Activity A directly precedes activity Activity B, then the expected variance of the former magnifies the expected variance of the latter.

Second, let’s also acknowledge that NOTHING we do is pure discovery. Even the most in the weeds, experimental, exploratory design is facilitated by some amount of process. Processes by which features are spec’ed out or the act of coding the features themselves. Or the processes by which those featured are submitted to source control repositories, or subjected to QA testing. Then there are processes for actually generating builds or deploying them to distribution platforms.

So, if we want to get a handle on the variance that causes us so much grief, we should endeavor to make any process oriented activity:

• Efficient
• Predictable
• Whenever possible, automated

Why The Fuss Over Process?

Curtailing the variance of our processes accomplishes two important goals. First, it reduces out overall variance. If, as I said previously, variance compounds across activities, then it also follows that reducing the variance of one or more activities in that sequence reduces the overall variance of the entire sequence.

Second, reducing the variance of some activities of some activities in the sequence increases out ability to absorb variance from the other activities.

And that notion is what “Game Planning With Science!” is all about: how to understand, measure, and compensate for variance. The series of articles covers nitty-gritty topics like statistics and operation science to higher-level concerns like management psychology and long term forecasting.

The Roadmap

Here is a preview of what you will find in the “Game Planning With Science!” series:

Part 1 – Video Game Art Pipelines

The first post covers two of the most fundamental aspects of operation science: process flows and Little’s Law. The post shows you how to analyze any process flow pipeline by identifying its critical path and its bottleneck. I also define Little’s Law and how it defines the relationship between three key aspects of any pipeline process: the amount of units currently in the pipeline, the time necessary to complete any single unit, and the throughput of the pipeline.

Part 2 – Character Art Pipeline Capacity Charts

Part 2 builds on Part 1 by taking those process flow fundamentals and using them to optimally staff a hypothetical character art pipeline. Using a simple tool called a “capacity chart”, you can quickly and easily analyze how staffing impacts the throughput of any process by impacting which activity is the bottleneck. Part 2 is a step-by-step walkthrough of how to assemble and interpret a capacity chart along with some special case calculations.

Part 3 – A Primer on Video Game Statistics

In order to understand and account for variance, you have to have a baseline understanding of statistics. That’s what Part 3 is all about: a crash course on statistical analysis. The post will clarify the difference between statistics and the real world and define a “probability distribution”. It also explains why calculating the arithmetic average of data only tells you part of the story and why it’s essential to pair that average with an understanding of variance.

Part 4 – Planning Games Using the Central Limit Theorem

Building on Part 3’s statistical primer, Part 4 digs into one of statistics’ most important concepts: the central limit theorem. The central lime theorem allows us to make predictions about outcomes even when we have limited data. In the post, I’ll show you, with in depth instructions how to use Excel to make meaningful estimates. For example, you’ll learn how to estimate the probability of hitting a particular delivery date and, conversely, what range of dates you could hit with a desired probability. I also show you how to calculate a “confidence interval” which provides a range out outcomes you can hit with a given probability. This is the most math centric portion of “Game Planning With Science!”, but hang in there because it’ll pay dividends down the line!

Part 5 – Story Points for Feature Estimation

In part five, I show you how to greatly simplify your data gathering using a simple concept from scrum: story points. The post starts by drawing a distinction between accuracy and precision, and then move to an explanation of why forecasting with discrete units of time is inherently problematic. I then define what a “story point” is, why you should use it, and how to perform a proper story point estimate.

Part 6 – User Stories Make for Better Consensus

Having established the value of story points in Part 5, Part 6 describes the value of their counterparts, user stories. The post provides a formal taxonomy for a successful user story. It then moves to an argument in favor of the value of disciplined feature planning. I also define the concept of “omitted variable bias” and how you can use it to your advantage when estimating the scope of user stories. The post concludes by describing how to play “planning poker” and how to avoid the negative impacts of social proof.

Part 7 – Scheduling Video Games Scientifically

Part 7 is a capstone post – it ties the prior 6 posts together and provides a comprehensive methodology for estimating the time to complete a given amount of work while taking variance into account. The post covers scheduling for both art assets and feature creation. It also provides guidance on how to make reasonable assumption if you’re just getting started and lack data.

Part 8 – Sunk Costs and Ugly Babies: On the Value of the Scientific Method

Part 8 offers a counterpoint to Parts 1 through 7. Namely, all of the data and predictive models in the universe are no substitute for responding to real world change. So, as important as it is to gather data and plan carefully, it is just as important to throw those plans away when they no longer match reality. Part 8 offers multiple scenarios where it is important to think critically about what you see. Making games scientifically doesn’t just mean applying complex formulas and statistical calculations. It also means applying the scientific method in order to make the best possible decisions.

Part 9 – Lean Development for Games

1947. Japan, still reeling from the Second World War, lies in economic ruin. And a little company called Toyota finds itself in a highly undesirable strategic position. It’s headquartered in a nation in tatters, and competing with companies in the world’s newest economic super-power: ‘Merca. How could it possible compete against companies with such amazing access to capital, resources and cash-rich customers?

Well, necessity, as they say, is the mother of invention.

And the necessities of Toyota’s strategic realities gave birth to one of the most remarkable feats of management ever achieved: the Toyota Production System. Or, more generically, “lean” development.

Part 10 – Poka-Yoke: The Fine Art of Mistake Proofing

You remember that time you plugged your HDMI cable into your PS4 upside-down and fried it’s video card? Or that time you clamped your ski-boots into your skis backwards and almost killed yourself on a black diamond slope as a result? Or that time you put a k-cup into a Keurig machine sideways an sprayed hot coffee all over the kitchen?

No, you don’t. Because you can’t do any of those things.

Those products are all designed so that they can only be used in the proper manner. The grooves on the HDMI connector, the specific clamps and catches on skis and boots, and the tapered design of Keurig k-cups mean that the proper application of the product is as obvious as its improper use is impossible.

Welcome to the world of poka-yoke (poh-kah yo-kay).

Part 11 – Kanban: The Counter-Intuitive Value Of Pull-Based Production

The word “kanban” sometimes comes up in software and game development circles. But, when it does, it’s usually within the context of agile and scrum. And the interpretation of the term usually begins and ends with “It’s like scrum, but without sprints.”

This is a massive oversimplification.

You see, when I was a freshly minted scrum master, I often told folks that I was all about scrum, but I would happily kick it to the curb as soon as I found a better framework.

Kanban is that framework.

Part 12 – Jidoka: Putting The Robots To Work

As a developer, you’re almost perpetually flat to the boards. There is always too much to do, too many fires, too many needed fixes. You can’t create more time in the day, or more days in the week. But you can do the next-best thing: eliminate repetitive tasks using automation. Welcome to the wonderful world of jidoka.

Part 13 – The Muda of Defects, Or: Finding Freedom Through Discipline

One of the more interesting characters I’ve encountered in my wanderings through the internet is a man by the name of Jocko Willink. He’s the author of Extreme Ownership and a business consultant. Oh, and an ex-Navy SEAL and a black-belt in jiu-jitsu. So, the man knows a thing or two about getting shit done under arduous circumstances. And his personal mantra is “Discipline equals freedom.” And the more I study operation science and the more I learn about software development, the more I see his point. So, in this post, I’m going to walk you through a multi-step process for testing code and how a little QA discipline can avail a lot of freedom.

Part 14 – Heijunka: Why Batching Is Not Your Friend

A commonly held belief is that it’s best to batch work – to handle similar tasks in large, consolidated chunks. The notion makes intuitive sense. It allows you to focus on one activity at a time and avoid so-called switching costs of switching activities. But as with so many other instances of unverified intuition, this particular notion is flat-out wrong. Batching may avoid switching costs, but it greatly protracts flow time, which, in the long run, can end up being far more expensive. Which is why the Toyota Production System introduced the concept of heijunka – “leveling”.

Part 15 – Root Cause Analysis: The Five Whys

On January 28th, 1986, seven astronauts boarded the Challenger for its tenth launch into space. Its previous missions had included the first space walk and, at various times, the first American woman, African-American, Canadian and Dutchman in space. 73 seconds after lift-off, the Challenger broke apart, killing the entire crew. Why? Because the fuel tank exploded. So, the solution, of course, is to send the next shuttle up with a fuel tank that doesn’t explode, right?

Only if you assume that the exploding fuel tank was a 100% isolated incident, completely unrelated to any other events. If that sounds fishy to you, it should. And this is where root cause analysis comes into play, a practice colloquially known as “the five whys”.

Enjoy and Remember: The Internet is Awesome

“Game Planning With Science!” represents months of work. And that’s not counting the time I spend in school learning all this stuff. As such, it represents the best case I can make for using these tools and concepts. I’ve tried to make these posts both as comprehensive and as digestible as I can. But the beauty of a blog is that nothing is set in stone. I will continue to iterate, edit, polish, and update these posts both as I improve as a writer and as I continue to learn.

The other beautiful thing about blogs is that they are a starting point for dialog. So please, as you read these posts, comment if the spirit moves you. Correct me, question me, challenge me. I’m always happy to talk shop and to learn new insights.

Now then, let’s get started! Click this link to go to Part 1: Video Game Art Pipelines

---

Key Takeaways

• All activities fall on a spectrum, with discovery on one end and process on the other
• Discoveries are activities that venture into the unknown
• Processes are activities that are well know and predictable
• Few activities are pure process or pure discover; most are a blend
• Discovery by necessity carries risk, which operation science calls variance
• Variance compounds across sequential activities
• Much of what we do as game developers is discovery oriented, but nothing we do is pure discovery
• For all of the things we do that are process, we should seek to make those activities efficient, predictable and automated whenever possible
• Reducing the variance in our process-oriented activities will, by extension, reduce the variance in subsequent activities
• It will also allow us to absorb more variance from those activities

---

º Source: International Business Times, “When Did Man Discover Fire? Ancestors Of Modern Humans Used Fire 350,000 Years Ago, New Study Suggests”, Philip Ross

---

“Discovery Versus Process – An Preface to Game Planning With Science!” by Justin Fischer is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

If You Enjoyed This Post, Please Share It!