Picture this: you've assembled your team to have a risk collection session, perhaps doing a Pre-Mortem, and everyone is really engaging with the process. Suddenly, your simple budget has 30+ risks. You could go ahead and put them all into your DartCannon simulation, but should you?
If you couldn't guess already, the punchline is probably not.
Impact of Risks
Lets create a quick simulation - one line item, and 4 1% probability risks, each 50% of our line item. For most projects, this is a decent representation of a "high impact" risk. Lets look at the result:
Even running at the highest resolution on DartCannon, over half of the range of the simulation has no results. Try to present this to stakeholders and instead of a productive conversation about hitting the 75% or 90% marks, you'll immediately get sucked into talking about the ~0% chance of being at 7 or 8 on this chart.
This also assumes that you've correctly estimated the chance of these low probability events.
Over-Expecting Unlikely Events
Humans tend to over-estimate the likelihood on infrequent events. Hollywood-style disasters and an over-availability of stories on negative outcomes, combined with the Availability Heuristic lead to over estimating the probability of rare events.
Particularly for high-impact risks, where even small changes in probability can drastically change outcomes, we get into Hollywood disaster thinking and the result to simulations can be the real disaster. Look again at our simulation results. We've estimated our risks at 1%, so there is a 4% chance that at least one of them will occur (100% - 99% ^ 4), which creates the slightly visible results between 3 and 4. But what if the chance was actually 0.1%? Those outcomes would virtually disappear.
Can you and you team accurately estimate the difference between 1% and 0.1%? It could have a big impact on your results.
What Decisions Are Needed?
Central to any discussion of how best to use any estimation tool should be what decisions will be made. Not going to change anything and move ahead no matter what? Don't bother doing any estimating.
The majority of the time you will be making decisions based on estimates - allocating resources, scheduling personnel, or committing funds. The same extends to risks - will including the risk of a meteor impact change your decision making?
Conclusion
Disaster scenarios can be a great way to start a discussion on risks and get a team engaged and you don't want to shut down any creativity. When you go to document those risks and incorporate them into your planning there should be a degree of managerial judgement.
Ultimately, where you set the bar on which risks to include between '1 in a billion' meteor impact and a 75% chance of an incremental charge is a judgement call you need to make. It depends on the decisions needed and the story you're trying to tell.
Photo by Marc Sqeglat on Unsplash