Last week I wrote about improving your project management, today I want to address a related issue, how to minimize surprises during product development. Most experienced leaders in the tech and game space (and probably any product development) have experienced many projects that fail to meet their goals for time, cost or performance. Rather than accepting these failures as part of the development process, an article in the MIT Sloan Management Review, Reducing Unwelcome Surprises in Project Management by Tyson Browning and Ranga Ramasesh, shows how to reduce these undesirable blows.

Browning and Ramasesh discuss projects often miss schedule and projections because of unknown unknowns, problems people do not even know they do not know (rather than issues you are concerned about from the beginning). What the authors point out is that they are not really unknown unknowns but issues that nobody has bothered to find out. Many of these unknown unknowns can be converted to known unkowns (and thus planned around) through a process of “direct recognition.”

Six project domains

There are six project domains where a project uncertainty resides and where recognition of the uncertainty should occur. “Projects operate as systems. Project and performance result not only from individual project elements but also from how the elements work together.” All projects have five key subsystems that when coupled with the project’s context contain both known and unknown unknowns.

1. Result subsystem. The desired result of most projects is a product, service, feature or other deliverable. Results have multiple components, all of which must work together to deliver success. Problems in one area can bleed into other areas.
2. Process subsystem. The labor required to execute and manage a project is another system, made up of activities, tasks and decisions tied to the flow of information, work product and deliverables. Efficient and effective processes depend not only on the specific activity but also on the relationships among the activities. Bad inputs can undermine an otherwise value-adding activity.
3. Organization subsystem. A third type of system incorporates the people, teams, departments and functions working on a project. This is the system that often breaks down due to poor communication.
4. Tools subsystem. Team members need tools, facilities and equipment to manage activities and exchange information. Many tools, however, are unable to transfer information due to various incompatibilities and organizational decisions.
5. Goals subsystem. Almost every project has goals for time, cost and performance/quality. These three areas, however, compete with each other. For example, improving performance by adding features often adds costs.
6. Context. Every project (hopefully) exists within a larger context. A project may be part of a larger portfolio or a feature for a game. It may also have multiple stakeholders with competing visions and metrics for success.

The first step in reducing unknown unknowns is to consider these six subsystems and their relationships.

The factors driving uncertainty

There are multiple elements of a project’s subsystems and context that make impact the likelihood of surprises. If you review your project looking for unknown unknowns you are more likely to convert them to known unknowns (and thus account for them in your schedule/budget). There are six factors that drive uncertainty and help you identify unknown unknowns.

The first of is complexity. A complex system contains many interacting elements that increase the variety of its possible behaviors and results. A project with more tasks, people or requirements is going to be more complex than a project with fewer. When a project’s elements have a greater variety of tasks, complexity also increases. The higher complexity, the greater the chance of experiencing unknown unknowns.

The second element driving uncertainty is complicatedness. Complicatedness is more subjective than complexity. It is often increased if the product is unprecedented or its structure is unintuitive. The more complicated the project, the more difficult it is for the participants to understand and anticipate.

The third factor impacting uncertainty is dynamism. A project’s volatility adds to its complexity. A project’s external dynamics are particularly likely to affect its goals, for example if regulation changes. Changes in goals then add to the complexity and complicatedness and increase the chances of unknown unknowns.

The fourth factor impacting uncertainty is equivocality. Project work requires sharing a great deal of information. If the information is not crisp and specific, then the people who receive it will be equivocal and will not be empowered to make firm decisions.

Mindlessness is the fifth factor driving uncertainty. These are the perceptive barriers that interfere with the recognition of unknown unknowns. They good be too much reliance on intuition or anchoring on past experience and traditions rather than looking objectively at the situation.

The final factor impacting uncertainty is project pathologies. Project pathologies are structural or behavioral conditions in and around project in their entirety (rather than individuals who exhibit mindlessness) that allow unknown unknowns to remain hidden. Project pathologies include mismatches among the project subsystems and context, unclear expectations among stakeholders and dysfunctional cultures.

Reducing unknown unknowns

By addressing the factors that drive uncertainty and understanding projects’ subsystems, Browning and Ramasesh suggest several ways to reduce the chance of unpleasant surprises

• Decompose the project. Modeling a project’s subsystems builds understanding that exposes unknown unknowns. Decomposition should begin with the natural structure of the overall purpose of the project, identifying the sub-problems relating to key areas and complimenting it with experience and experimentation.
• Analyze scenarios. Scenario planning includes building several different future outlooks. It tries to understand and build uncertainty into the reasoning. Rather than being predictions, scenarios are coherent and credible futures built on dynamic events and conditions.
• Use checklists. Organized learning from past projects can inform planning. They can take the form of checklists or prompt lists, though should not be used as a substitute for thinking.
• Scrutinize plans. Project plans are an estimate for how success will occur, including resources needed and results. All participants and stakeholders should scrutinize closely these estimates. This scrutiny can be as project reviews, audits or even external evaluations.
• Use long interviews. Thorough interviews with project stakeholders, subject matter experts and other project participants are effective at uncovering waiting problems and issues.
• Pick up weak signals. Weak signals often come in subtle forms, such as unexplained behaviors, confusing outcomes or a realization that no one in the organization has a complete understanding of the project.
• Mine data. When vast amounts of data, analyzing this data can pull out implicit, previously unknown information. By reviewing data from multiple projects, data mining could help you identify precursors of potential problems.
• Communicate frequently and effectively. Regularly and systematically reviewing decision-making and communication processes, including the assumptions that are factored into the processes, and seeking to reduce information disparities, can help to anticipate and uncover unknown unknowns.
• Balance local autonomy and central control. Using decentralization of control to grant autonomy to the local nodes of a multi-nodal project aids recognition of unknown unknowns. Although decentralization helps project managers compensate for their knowledge gaps, it creates challenges for governance. Local nodes are less willing to report problems. You must balance central authority with local control to ensure unknown unknowns are both discovered and reported.
• Incentivize discovery. One of the best ways to identify unknown unknowns is timely and honest communication of missteps, anomalies and missing competencies. Offering incentives for candor can show people there are advantages to owning up to errors or mistakes in time for management to take action.
• Cultivate an alert culture. If project participants and stakeholders understand how unknown unknowns can derail projects, they will strive to illuminate rather than hide potential problems. You can make the culture more alert by emphasizing systems thinking, stress the limits of what can be known about a project a priori, seek to build a wide range of experiential expertise, develop the characteristics of a high-reliability organization and learn from surprising outcomes.

Avoid the unpleasant surprises

With the techniques above, you and your project managers can uncover and recognize knowable unknown unknowns. By providing guidance on where and why these unknown unknowns exist in projects and how to recognize their clues, you can reduce the number and magnitude of unwelcome surprises.

Key takeaways

1. Unpleasant surprises in product management – particularly software development – occur when issues arise that nobody thought of, referred to as unknown unknowns.
2. To anticipate unknown unknowns, you need to understand the underlying system for the projects and the key factors (complexity, complicatedness, dynamism, equivocality, mindlessness and project pathologies) that drive the uncertainty.
3. You can then reduce the number and magnitude of unpleasant surprises by moving many to known unknowns by decomposing the project, analyzing scenarios, using checklists, scrutinized planning and strong communication (among other techniques).