What is Risk Impact Sensitivity Analysis and how does it relate to on time completion of projects? More importantly, why is this knowledge important to you, as a project professional?
In this blog post I will introduce a methodology that I call “Risk Impact Sensitivity Analysis” or RISA for building project schedules. Risk Impact Sensitivity Analysis is an effective method to evaluate the sensitivity of your project schedule to risks. It will account for risks resulting in more realistic schedule estimates thereby increasing your chances of completing the project on time.
Using a typical construction project as an example, I demonstrate in this blog post how you can apply Risk Impact Sensitivity Analysis.
1. Introduction to Risk Impact Sensitivity Analysis [RISA]
How often do you finish your projects on time?
The likely answer is: Not often! Survey after survey reveals that projects are replete with schedule delays.
The usual suspect? Risks.
Ignoring risks will result in overly optimistic project schedules, which are almost impossible to meet.
There are two types of risks that have a significant negative impact on project schedules: variability risk and events risk [Table 1].
Most project teams develop project schedule estimates using deterministic methods. These methods employ a single—a most likely [same as most probable]—value for estimating the duration of any given task of a project. Using single values for all the tasks that comprise the project scope of work, the overall project duration is calculated by applying the critical path method [CPM]. Deterministic methods do not account for the variability risk associated with the estimates.
Furthermore, events’ risk [caused by specific risk events] is often ignored in estimating the schedules. If at all such risk is considered, qualitative methods are employed to analyze and prioritize the risk events prior to developing response actions.
Qualitative methods involve estimates of most likely probabilities of occurrence for each risk identified and most likely impact of those risks on project schedule using a nominal scale where measurements are based on discrete categories rather than numeric values.
Typical categories include low, medium, and high. If quantitative analysis is applied, deterministic methods are common in estimating risk impacts—again, ignoring the variability risk associated with these estimates.
Table 1: Defining Variability and Events Risk for Risk Impact Sensitivity Analysis
As defined in Table 1 above, risks may have either negative or positive impact on project objectives and are called threats and opportunities, respectively.
However, in many organizations threats are referred to as risks. In this blog post, to keep my illustrations simple and easy to understand, I use only threats and refer to them as risks as well. But the principles and the application of the tools for analyzing threats and opportunities are basically the same.
2. Quantitative Risk Analysis [QRA] and Risk Impact Sensitivity Analysis
Whereas deterministic estimation and qualitative methods for risk analysis are a first step in project scheduling, sophisticated quantitative analysis with stochastic tools offer you more meaningful insights and results.
Stochastic – also referred to as probabilistic – methods involve estimates based on ranges with probabilities rather than single, deterministic values.
Quantitative Risk Analysis [QRA] is an effective approach to identify, analyze, and prioritize risks. You’ll have a better understanding of the risk impacts, so you can develop proper response actions.
How do you develop risk response action plans?
Typical strategies for response actions are avoidance, mitigation, transfer, acceptance, and escalation [Table 2 below].
Table 2: Risk Response Strategies
Risks can never be eliminated altogether. In fact, you may not want to eliminate all risks, because certain risks are unlikely to materialize and so why spend resources on them?
Therefore, a prudent approach is to take proactive measures for significant risks and establish contingency reserves for the overall remaining “residual” risk. Typically, the reserves are calculated as a percentage of estimated total project cost, but QRA offers a more rational approach.
Risk Impact Sensitivity Analysis [RISA] is a form of QRA. It involves Monte Carlo simulations incorporating variability and events risks into project scheduling process. You can analyze the simulation results utilizing various RISA tools [Table 3].
Table 3: Risk Impact Sensitivity Analysis [RISA] Tools for Schedule Risk Analysis
Risk Impact Sensitivity Analysis will help you develop more realistic project duration estimates leading to a higher chance of project schedule success. The key characteristics of the Risk Impact Sensitivity Analysis are:
1] Considers both variability and events risks.
2] Employs stochastic methods involving Monte Carlo simulations.
3] Performs sensitivity analysis to evaluate and prioritize risks.
4] Develops reserves based on rational methods.
I use a hypothetical construction project, called Project Pinot, as an example to illustrate the application of the Risk Impact Sensitivity Analysis [RISA].
3. Project Pinot to demonstrate the use of Risk Impact Sensitivity Analysis
Acme Chemicals Corporation [Acme] is a small specialty chemicals manufacturing company with its major operating facility located in Baton Rouge, Louisiana, USA.
The board of directors is considering a major expansion at this facility to manufacture a new class of chemicals. A significant part of this initiative is managing the large volume of process wastewater resulting from the proposed expansion.
The wastewater currently generated at this facility is discharged into the local municipal sewer. Acme pays a “surcharge” to the city based on the daily volume and pollutant load.
The proposed expansion will increase the wastewater volume as well as pollutant load. An internal team has conducted a preliminary investigation and recommended an onsite wastewater treatment plant [WWTP] instead of using the local sewer. They estimated that the investment will exhibit a rapid payback as well as significant cost savings compared to the municipal surcharges.
The board has authorized a feasibility study that involves bench-scale “treatability” testing of the “simulated” wastewater expected to be generated after the expansion is completed. The study’s objective is to investigate whether a new innovative technology recommended by Acme’s R&D will be effective to remove the pollutants to the levels mandated by the Louisiana Department of Environmental Quality [LDEQ].
Furthermore, the study will help develop the design basis for the full-scale WWTP. The board has also approved funds for developing a conceptual design of the system based on the feasibility study results and defining the scope of work to build the WWTP along with project schedule and cost estimates.
Upon completion of this task, there’ll be a go/no-go decision check point. A project team has been assembled. Project Pinot is born!
3.1 Project Assumptions & Constraints
The project team has met a few times and developed the preliminary scope of work. Their task now is to put together a preliminary schedule. To help the team with this effort, the project manager has provided them with the following information.
- The expected start date of the project is January 2nd of the coming year.
- The feasibility study will be conducted by an external vendor.
- An effluent discharge permit must be obtained from LDEQ before Acme can start discharging the final effluent to the nearby Mississippi.
- Acme has the option to continue discharging its untreated wastewater to the local sewer. But the surcharge cost may make the expansion cost prohibitive.
- The environmental permit application as well as all the construction permit applications will be prepared by the company’s regulatory specialist.
- Acme’s engineering department will be responsible for the front-end engineering and design [FEED], while the detailed engineering will be outsourced.
- A pair of customized pumps is required for the project that could take a long time to procure.
- Construction work including equipment installation will also be outsourced. It’s likely that this effort will take place during the hurricane season.
3.2 Risk Impact Sensitivity Analysis: Methodology Steps & Project Pinot Results
Using the Project Pinot scenario, I illustrate below the key steps involved in developing the project schedule using Risk Impact Sensitivity Analysis [RISA]. As I go through the step-by-step process, I refer to the figures and tables where the RISA data for Project Pinot is presented.
3.2.1 Risk Impact Sensitivity Analysis [RISA] Step 1: Develop The Base Schedule Estimate
First, you develop the base schedule estimate using single values of the most likely durations of each task comprising the project scope of work and CPM model. It’s referred to as the deterministic schedule since the duration estimates are deterministic [Fig. 1 and Fig. 2 below].
Figure 1: Network Diagram used in Risk Impact Sensitivity Analysis Scenario
Figure 2: Gantt Chart used in Risk Impact Sensitivity Analysis Case Study
3.2.2 Risk Impact Sensitivity Analysis [RISA] Step 2: Define Distributions And Their Parameters For Task Durations
For every single task in the schedule estimate, select the distribution type you believe is most appropriate. The distribution is meant to represent variability risk associated with your estimates.
The two most common distributions in project scheduling are pert and triangular. The latter is more conservative resulting in longer task and project durations. The parameters for both distributions are minimum, most likely, and maximum values.
Typically, the distribution type and parameters are selected based on subject matter expertise, team input, and historical data. For the sake of simplicity and illustration purpose:
➡️ I applied pert, triangular, or no distribution depending upon the presumed uncertainty associated with the tasks.
➡️ The minimum and maximum values for tasks where pert or triangular distribution is applied are -10% and +25% of the most likely values, respectively [Table 4 below]. I use the same approach for the impact of risk events discussed in the next step.
Table 4: Variability Risk for Each Task in Risk Impact Sensitivity Analysis Scenario
3.2.3 Risk Impact Sensitivity Analysis [RISA] Step 3: Prepare A Risk Register
Identify threat events for the project. [As mentioned before, for the sake of illustration, I use only threats, not opportunities, in this example.] Estimate each event’s probability of occurrence and the impact if it occurs. Identify the tasks that will be impacted and estimate the extra time duration expected to be caused by each risk event.
In addition, identify if new tasks will be created because of the impact and if so, estimate their duration. Make sure you also identify any risk events that are mutually exclusive and if so, quantify their exclusivity.
Define any changes in task dependencies. Select the distribution [e.g., pert, triangle, etc.] for the impact durations and estimate their distribution parameters [e.g., minimum, most likely, and maximum values].
Select the proper frequency distribution of each risk event and estimate its distribution parameters. You may select Bernoulli, if the event is expected to happen only once during the project implementation phase or Poisson, if it can happen more than once. The distribution parameter for Poisson is called lambda, which represents the average rate of occurrence of an event in a fixed interval of time.
Document the risks identified and their probabilities, impacts, distribution information, etc. in the risk register [Table 5 below]. It’s imperative that this overall effort of preparing the risk register involves input from subject matter experts and the project team.
Table 5: Risk Register used in Risk Impact Sensitivity Analysis Scenario
3.2.4 Risk Impact Sensitivity Analysis [RISA] Step 4: Perform Monte Carlo Simulations
Incorporating the variability and events risks information into the base schedule estimate, perform Monte Carlo simulations for developing the stochastic estimates of the project duration. For a simple description of the Monte Carlo simulation process for project scheduling, please refer to Table 6 below.
The input data for the simulation of Project Pinot’s schedule is already given in Tables 4 and 5.
I performed the Monte Carlo simulation using a software application called HawkEye that my team and I developed. The software not only runs the simulations but also presents the results in different formats using various RISA tools. All the figures presented in this paper are for Project Pinot and were generated using HawkEye.
Table 6: Basic Steps of Monte Carlo Simulation for Project Schedule
Fig. 3 and Fig. 4 below present the probability distribution function [PDF] and cumulative distribution function [CDF] plots of the project schedule, respectively. Each figure includes plots accounting for only the variability risk and both variability and events risks together. [The CDF plots are generally referred to as S curves due to their shape.]
Figure 3: PDF Plots With and Without Risk Events in Risk Impact Sensitivity Analysis Scenario
Figure 4: CDF Plots With and Without Risk Events in Risk Impact Sensitivity Analysis Scenario
3.2.5 Risk Impact Sensitivity Analysis [RISA] Step 5: Conduct Risk Impact Sensitivity Analysis and Prioritize Risks.
You can apply several Risk Impact Sensitivity Analysis [RISA] tools [listed in Table 3] to analyze the simulation results and prioritize the risks. If you want to accelerate the schedule, these results can also give you insights into where you may want to fast-track and by how much.
Fig. 5 below is a tornado diagram showing how the project duration varies as the duration of a task varies. Whereas Fig. 6 is a “half tornado” diagram where the correlation coefficients are displayed. Fig. 7 is a similar diagram showing critical indexes.
Figure 5: Tornado Diagram
Figure 6: Correlation Coefficients
Figure 7: Critical Indexes
Table 7 below includes correlation coefficients, critical indexes as well as schedule impact indicators [SIIs]. It also shows a ranked listing of risk events based on their SIIs. You may perform further sensitivity analysis by asking “what-if” questions. For example:
- What’d be the project schedule if none vs. all the identified risk events are accounted for [Fig. 4]?
- What if you start with an initial schedule accounting only for the variability risk but not the risk events and then add one risk event at a time [Fig. 8]?
- What if you start with a project schedule that considers variability risk and all the identified risk events together and then exclude one event at a time [Fig. 9]?
Table 7: Risk Impact Sensitivity Analysis [RISA] Factors
Figure 8: CDF Plots With Risk Events Added One at a Time
Figure 9: CDF Plots With Risk Events Excluded One at a Time
3.2.6 Risk Impact Sensitivity Analysis [RISA] Step 6: Develop Proper Response Actions For Each Risk
Based on the sensitivity analysis and the ranked listing of risks, develop proper response actions for each risk. Table 8 below lists proposed response actions for each identified risk and Fig. 10 below shows the same information by mapping response actions to individual tasks.
You can also evaluate the simulation results to find ways to reduce the project schedule. An evaluation of task correlation coefficients [relationship between project duration vs. task duration considering all the iterations] and critical indexes [percent times a task is on the critical path considering all the iterations] will give clues as to what task durations you want to reduce to gain significant reductions in project duration.
One key factor is “external” dependencies [e.g., supplies/services from vendors/contractors, permits from government agencies, etc.] over which project teams have less control.
Table 8: Proposed Risk Response Actions
Figure 10: Proposed Risk Response Actions in Risk Impact Sensitivity Analysis Scenario
3.2.7 Risk Impact Sensitivity Analysis [RISA] Step 7: Repeat Monte Carlo Simulation With The Revised Risk Register
Revise the risk register [Table 9] by changing the risk probabilities and impacts and their distribution data to reflect the proposed response actions [Table 9] you’ve decided to implement.
Make sure to include in your scope of work any additional tasks created by your response actions. Furthermore, add any new risks [plus all the required data] that may be created by your response actions.
Then, repeat the Monte Carlo simulation using the revised risk register. The results will reflect the revised scope of work accounting for risk response actions.
Based on these results, you can negotiate with your project sponsor a mutually acceptable project schedule and contingency reserves you want to add to the schedule [Fig. 11 below]. The reserves are designed to account for any residual or remaining risks.
3.3 Understanding the Results Provided by Risk Impact Sensitivity Analysis
An analysis of simulation results using various RISA tools can be revealing. It helps you understand the impact of individual task durations and risk events on the project schedule leading to more realistic project schedule estimates. You can learn what risks you should pay more attention to and how best you can compress the schedule, if need be.
Table 9: Revised Risk Register After Using Risk Impact Sensitivity Analysis
The deterministic project duration estimate for Project Pinot is 325 workdays or 65 weeks [Fig. 1 and Fig. 2]. When the variability risk is accounted for and a simulation is employed, the average project duration increased to 342 days—approximately by 5% [Fig. 3 and Fig. 4].
When both variability and events risks are considered, the average duration turns out to be 379 days—an approximately 17% increase compared to the deterministic estimate [Fig. 3 and Fig. 4].
If you received authorization for the deterministic project schedule of 325 days, the probability of completing Project Pinot under that duration is