### About Global Documents

**Global Documents provides you with documents from around the globe on a variety of topics for your enjoyment. **

**Global Documents utilizes edocr for all its document needs due to edocr's wonderful content features. Thousands of professionals and businesses around the globe publish marketing, sales, operations, customer service and financial documents making it easier for prospects and customers to find content. **

Copyright 2009 AACE, Inc.

AACE International Recommended Practices

AACE International Recommended Practice No. 42R-08

RISK ANALYSIS AND CONTINGENCY DETERMINATION

USING PARAMETRIC ESTIMATING

TCM Framework: 7.6 Risk Management

Acknowledgments:

John K. Hollmann, PE CCE CEP (Author)

Rodney B. Adams, CCE

Hubertus M.T. Brandts, CCE

Alan J. Chilcott, CCT CCE

Dr. Ovidiu Cretu, PE

Charles J. Pospisil

Chinnadurai Ramachandran

Dr. Maarten S.A. Vrijland

Robert F. Wells, CEP

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

AACE International Recommended Practice No. 42R-08

RISK ANALYSIS AND CONTINGENCY DETERMINATION

USING PARAMETRIC ESTIMATING

TCM Framework: 7.6 Risk Management

January 26, 2009

INTRODUCTION

Scope

This recommended practice (RP) of AACE International (AACE) defines general practices and

considerations for risk analysis and estimating cost contingency using parametric methods. Parametric

methods are commonly associated with estimating cost based on design parameters (e.g., capacity,

weight, etc.); in this case, the method is used to estimate contingency based on risk parameters (e.g.

level of scope definition, process complexity, etc.). This RP includes practices for developing the

parametric methods and models (generally empirically-based). Recommended practice 43R-08 provides

example process industry parametric models (including software) [12]. For scheduling applications, there is

less research and reference material available; therefore schedule duration risk and contingency will be

covered in future revisions of the RP.

Purpose

This RP is intended to provide guidelines (i.e., not a standard) for contingency estimating that most

practitioners would consider to be good practices that can be relied on and that they would recommend

be considered for use where applicable. There is a range of useful contingency estimating

methodologies; this RP will help guide practitioners in developing or selecting appropriate methods for

their situation.

While this RP is relatively short, it incorporates a lot of information by reference and it addresses a

complex research and empirically based methodology. It is highly recommended that the reader

understands the research behind this method to avoid significant misunderstanding of risks and

misstatements of contingency.

Background

This RP is new. However, it is based on over 40 years of research, development, and practice. The

development and use of parametric risk analysis and contingency estimating methods evolved in parallel

with industry's recognition that poor project scope definition was often the greatest project cost and

schedule risk driver. This recognition led to the development of project scope development processes

(e.g., phase-gate processes) and scope definition maturity matrices such as those included in AACE's

recommended practice for cost estimate classification[1].

Before the above were accepted as best practices, experts first had to prove their value to project

outcomes. They did this by studying actual projects and developing empirically-based parametric models

that showed how poor scope definition resulted in greater cost growth and wider accuracy ranges. A

paper by Hollmann surveys these parametric developments[4] and highlights the pioneering work of the

late John Hackney, followed by Edward Merrow, et al. at the RAND Institute, and Steven Trost, et al. for

the Construction Industry Institute (CII)[5,6,7]. A paper by Baccarini also provides an extensive survey of

these methods[2]. These and the other sources referenced in this RP are recommended reading for

parametric method practitioners.

It is AACE's recommended practice that whenever the term "risk" is used, that the term's meaning be

clearly defined for the purposes at hand. In expected value practice as described in this RP, risk means

"an undesirable potential outcome and/or its probability of occurrence", i.e. "downside uncertainty (a.k.a.

threats)." Opportunity, on the other hand is "a desirable potential outcome and/or its probability of

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

2 of 8

occurrence", i.e. "upside uncertainty." The method in this RP quantifies the impact of uncertainty, i.e.

"risks + opportunities".

Background Parametric Estimating

This is not an RP on parametric estimating, but a basic understanding of it is required. AACE's Cost

Engineering Terminology defines a parametric estimate as one that has "estimating algorithms or cost

estimating relationships that are highly probabilistic in nature"[8]. Generally, the relationships of the

outcome (e.g., cost growth) and the inputs (e.g., risk drivers) are determined by studying empirical data

using methods such as multi-variable regression analysis, neural networks, or even trial and error. The

following illustrates the typical form of a simple parametric estimating algorithm:

Outcome = Constant + Coefficient 1*(Parameter A) + Coefficient 2*(Parameter B) +..

The "outcome" in this case may be a measure of cost growth (e.g., contingency percent), and the

parameters are various quantified risk drivers such as a measure of the level of scope definition upon

which the estimate or schedule was based. The algorithm can be much more complex employing

logarithmic, exponential, and power series.

Advantages of parametric estimating for risk analysis and contingency determination are that it is

inherently empirical in nature (based on actual measured experience) and it can directly provide

probabilistic information about the distribution of possible outcomes. It is also very quick and simple to

apply.

A disadvantage is that parametric estimating is based on empirical methods such as regression analysis

and these require that the parameters actually have more or less predictable relationships with the

outcomes. This is more important for some risk types than for others. Another disadvantage is that

obtaining empirical data and creating models is a challenging effort; increasingly so as one attempts to

model cost growth and risk drivers at more detail levels. Therefore, the method is typically limited in use

to estimating overall project contingency that results from select risk types. As will be explained in the

next section, this is not a problem for early estimates (i.e., AACE Class 5), but for later estimates (i.e.,

Class 4 or better) the method is best used in combination with range estimating, event tree analysis or

other more definitive methods.

Background Risk Types

In respect to parametric methods, risk types fall into one of two categories; risks that have systematically

predictable relationships to overall project cost growth outcome and those that don't. These categories

have been labeled as "systemic" and "project-specific" risks for contingency estimating purposes (i.e.,

there will be other ways to categorize risk types for other purposes.)[4]. In order to use the methods

properly, it is important to understand the distinctions of these types.

The term systemic implies that the risk is an artifact of the project "system", culture, business strategy,

process system complexity, technology, and so on. Research by Hackney and others has shown that the

impacts of some of these risks are measurable and predictable between projects within a system, and to

some extent within an industry as a whole. Measures of these risks are generally known even at the

earliest stages of project definition, and furthermore, the impacts of these risks tend to be highly dominant

for early estimates. Also, the link between systemic risks and cost impacts is stochastic in nature; this

means it is very difficult for individuals or teams to understand and to directly estimate the impact of these

risks on particular items or activities (for example, the risks of process technology on something like site

preparation or concrete foundations may be dramatic, but is not readily apparent). Finally, systemic risks

tend to be "owner" risks; i.e., the owner is responsible for early definition, planning, technology, and

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

3 of 8

decisions so these risks cannot be readily transferred to execution contractors. The following are typical

systemic risks dealt with using parametric methods:

Process Definition

Basic Design

Level of Technology

Process Complexity

Material Impurities

Project Definition

Site/Soils Requirements

Engineering and Design

Health, Safety, Security, Environmental

Planning and Schedule Development

Project Management and Estimating Process

Estimate Inclusiveness

Team Experience/Competency

Cost Information Available

Estimate Bias

One of the most difficult systemic risks to deal with is "estimate bias". When estimate bias is psychological

or political in nature, it is particularly difficult to measure and quantify because it deals with deception,

intentional or unintentional. To assess the impact of these types of risks (i.e., optimism bias and strategic

misrepresentation), a methodology called reference class forecasting (not covered here), a form of

estimate validation, has been proposed by Flyvbjerg[3]. Whether and how these systemic psychological

and political risks can be better measured, and incorporated in parametric techniques is an area of active

research, particularly for government funded (i.e., politically charged) infrastructure mega-projects. In any

case, estimate validation (to detect bias among other objectives) is always a recommended practice in

conjunction with risk analysis.

The term project-specific implies that the risk is, as it says, specific to the project. The impacts of these

risks are not highly predictable between projects within a system or within an industry as a whole. For

example, rain may have much more impact on one project than another depending on the project

characteristics and circumstances. Measures of these risks are generally not known at the earliest stages

of project definition (e.g., for Class 5 estimates, rain cannot be considered because the location of a

project, the season of its construction, and other circumstances may not be known). Also, the link

between project-specific risks and cost impacts is more deterministic in nature; i.e., they are amenable to

individual understanding and to estimating the impact of these risks on particular items or activities (for

example, the risks of excess rain on something like site preparation or concrete foundations can be

estimated). Finally, these types of risks are more negotiable during project contracting strategy as to who

will carry them. The following are typical project-specific risks (this list is far from inclusive):

Weather

Site Subsurface Conditions

Delivery Delays

Constructability

Resource Availability

Project Team Issues

Quality Issues (e.g., rework)

Etc.

This breakdown of risk types indicates why a combination of risk analysis and contingency estimating

methods should be used for optimal understanding and quantification of risks of different types. The RP

will explain how multiple contingency estimating methods can be used and their results combined. For

Class 5, parametric methods can be used alone given the knowledge of the systemic risk factors (and

lack of knowledge of project specifics) and the dominance of their impacts at this phase. Project-specific

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

4 of 8

risks become more dominant as scope is better defined (and hence some systemic risks are mitigated),

but there are always systemic risks that should be analyzed as thoroughly as practical.

RECOMMENDED PRACTICE

Practices for parametric risk analysis and contingency estimating methods necessarily focus on

development of the parametric model(s) because that is the most challenging aspect; use of parametric

models is relatively simple.

Model Development

Processes Come First

Prior to developing and using any risk analysis or contingency estimating practice, the enterprise's risk

management process should be developed in alignment with the appropriate overall asset management

and project control processes, which in turn should align with business strategy. Process maps show

inputs and outputs of a method which help identify stakeholders in its practice. Example processes are

covered in AACE's Total Cost Management (TCM) Framework[9]. In particular, if the company has no

formal project scope development process, or process or system for project historical database or

knowledge management, empirically-based parametric methods will be difficult to develop or maintain

(however, implementing parametric methods can put emphasis on the company's need to strengthen

these processes).

Determine Requirements

Company processes and stakeholder input will help establish requirements of the scope of the method

and scope of the effort to develop, maintain, and use it. Some typical requirements (and constraints) to

consider include:

Classes of estimates[1].: If your company is a contractor that only deals with Class 3 or better

estimates, and most systemic risks are carried by the owner, parametric methods offer less value.

However, for owner's developing Class 5 estimates, parametric methods are extremely valuable.

Types of projects and risks: If you estimate and fund projects using new technology, complex

processes, complex strategies, and so on, parametric methods increase in value and you will want to

be sure to identify and analyze these types of risks (in addition to the level of scope definition).

Corporate risk management strategies: If you are responsible for analyzing not only cost and

schedule risks, but also technical, health and safety or other kinds of risks, this may affect the

development process (this RP addresses cost risks)

Resources and competencies available: because of the reliance on empirical data analysis, the

development of models requires significant resources with special analysis skills (particularly

statistical). On the other hand, because the methods are very simple to apply, and because they

inherently incorporate empirical learnings, they can be used by project teams with less expert help

than other methods.

Historical Data

Having identified requirements in terms of the types of projects and risks to be addressed, the

requirements for historical or empirical data can be defined. The list of systemic risks provided previously

is a starting point; developers should study the references to this RP for more information on the specific

risk drivers to measure and capture. The primary risks are the level of scope definition, the level of new

technology in the process, and the complexity of the process and the project strategy. How to measure

and record these risks quantitatively must then be determined.

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

5 of 8

Having identified the risks (i.e., parameters of the model), measures of the outcome must be determined.

In general these include cost growth relative to the base estimate excluding contingency.

One systemic risk that is a challenge to measure is the competitiveness and quality of the base estimate.

"Fat" base estimates (i.e., hidden contingency above-the-line) may result in little need for or usage of

additional contingency. Therefore, a process to review and validate the competitiveness and quality of the

base estimate (and total including contingency) becomes an ancillary part of the risk management

process.

Having determined the parameters and outcomes to capture, data collection and management

procedures need to be established. Ideally, these will be part of your project historical database

management process, including project close-out processes.

Reference and External Information

As mentioned, the references to this RP should be studied. The Hackney and Merrow references include

models that have been developed from industry data, and are still generally applicable. AACE has

documented these models in RP 43R-08. These models can serve as a starting point or go-bys for

internal developments. Other external data on risks and their outcomes from benchmarking sources and

other literature [e.g., AACE's technical library and Professional Practice Guides (PPGs)] should be

obtained.

Data Analysis and Tool Development

Having collected project risk and outcome data including quantitative measures for modeling, it must be

cleaned to ensure that the sample to be used for model building or evaluation is free of significant error

and is representative (i.e., no extreme outliers that tend to bias analyses). Outcome data must also be

normalized for (i.e., corrected for) escalation, currency, and scope change impacts which are not covered

by contingency.

Two methods of parametric model building are commonly found in the literature. The most traditional and

widely used is multi-variable linear regression analysis. Standard spreadsheet software generally has this

analytical capability. The other more recent method is neural network analysis. This analytical

functionality is not included in standard "office" software such as spreadsheets and requires special

software designed for neural network analysis. The model building methods used for risk analysis and

contingency estimating tools are the same as those used for general estimating models; the only

difference is in the nature of the parameters and outputs.

Regression analysis will typically find some sort of relationship between one or more of the parameters

and the outcome measure. However, the relationship must be tested and challenged to ensure that it is

statistically significant (e.g., using t or F statistics), that it is causal in nature (i.e., there should be a

rational hypothesis for why a parameter is impacting the outcome to the extent noted), that the variables

are independent and not co-linear, and that the model is not overly biased by outlier data points, and so

on.

Once a valid model is obtained, it is usually implemented in a spreadsheet tool wherein the user enters

the parameter values and the model generates the predicted contingency value, usually as a percentage

of the base estimate value. The regression output represents the mean contingency which for normally

distributed data is equivalent to the p50 value (50 percent of the time the result will be over or under this

value).

After a base model is built, analysts can supplement the base model constants, coefficients, and

parameters with various logical assumptions and adjustments that may not have been included in the

analytical dataset[10]. For example, if database included a set of projects for which project definition was

rated on a scale of 5 to 1 using AACE's scope development maturity matrix (i.e., from RP 18R-97), and

later, AACE adds a new risk-driving deliverable to the maturity matrix, the analyst may have to make

manual adjustments to their model as appropriate to address how the change may affect the 5 to 1 rating.

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

6 of 8

Probabilistic Outcomes

The base model generates the mean or p50 result value. However, best practice for risk analysis and

contingency estimating is to produce a distribution of possible outcomes so that management can decide

how much risk they are willing to accept and therefore how much contingency will be required. The

regression analysis will provide some evidence of the probability distribution. In particular, it provides the

standard error of the estimate for the regression model dataset. However, the regression dataset may be

limited in scope, and cannot always be relied on to fully represent the range of possible outcomes.

There is a simple method, which is consistent with observed industry data (including AACE's RP 18R-97),

to generate a reasonably reliable probability distribution for cost contingency. That method is to assume

that cost outcomes (after allowing for contingency) are more-or-less normally distributed and to further

assume that contingency is equal to the standard deviation of the distribution[11]. With these assumptions,

the normal cumulative distribution can be computed using the NORMINV function in MS Excel [syntax is

NORMINV(probability, mean, std. dev)]. The following is an example of such a distribution.

Given:

Base Estimate (without contingency)

= $100

Contingency from the parametric model

= $20

Total Cost (at p50) = $100 + $20

= $120

Then the Cumulative Probability Distribution is:

p

Total$

NORMINV (probability,120,20)

Contingency$

(Total-Base)

10%

$ 94

$ (6)

20%

$ 103

$ 3

30%

$ 110

$ 10

40%

$ 115

$ 15

50%

$ 120

$ 20

60%

$ 125

$ 25

70%

$ 130

$ 30

80%

$ 137

$ 37

90%

$ 146

$ 46

These results can be reported in the tool in tables or charts as desired.

Dealing With a Lack of Company-Specific Historical Data

Unfortunately, good project data is difficult to collect and analyze. Fortunately, systemic risks and their

impacts for industry projects have been fairly consistent with time. Therefore, lacking any other method,

the parametric models from Hackney and Merrow can be used with some confidence after validating

against your own experience. The models have been included in working versions in recommended

practice 43R-08[12]..

Risk Analysis and Model Use

Identify and Quantify Systemic Risks

Because the parametric model has pre-determined risks (i.e., the parameters), the risk analysis is

simplified. While this is not an RP on how to conduct a risk analysis session or workshop, the typical

practice is to hold a meeting of the key team members and other project stakeholders, and to start with

identifying risks. In this case, the risk types are identified, so the team concentrates on quantifying the

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

7 of 8

parameters; e.g., rating the level of definition of each key deliverable in the project scope maturity matrix,

rating the level of new technology, and so on.

The more difficult challenges are agreeing on subjective systemic risk drivers such as the quality of the

base estimating data and the project team's competency. Because these types of risks are in fact

"systemic" (i.e., an artifact of the company's culture and capabilities that the project cannot do much

about), it is recommended that default ratings be set for these to avoid over-optimism. The ratings can be

changed, but the team must provide specific reasons why this project "bucks-the-system".

Estimating Contingency

Once the parameters are quantified, the contingency and probability distribution for systemic risks are

estimated by simply plugging the parameter values in the model. The user should make quality checks

and validate that the results are reasonable before reporting them to management.

Coordinate with Contingency Estimates for Project-Specific Risks

For Class 5 estimates, parametric methods alone are generally adequate, given the dominance of

systemic risk impacts and lack of knowledge of project specifics. For Class 4 or better, other methods

such as range estimating or event tree analysis should be used in combination with the parametric

analysis. These methods are covered in other RPs.

The most important consideration in combining methods and outcomes is to ensure that risks are not

double counted. After risks are identified in a risk analysis session, each risk must be categorized as

systemic or project-specific. Each risk is then quantified in their respective analyses and contingency

estimates.

Parametric and event tree analysis can be easily combined because event tree (i.e., expected value)

models work by directly estimating the probable cost distribution of the impacts of each risk. In that case,

the results of the parametric model (including its probability distribution) are included in the event tree

analysis as the first risk. Then other project-specific risks (e.g., heavy rain) are quantified and added to

the model. Monte-Carlo simulation can then be applied to the entire combined cost risk model to obtain a

combined probability distribution.

If range estimating is used for project-specific risk analysis, the combination cannot be done through a

combined Monte Carlo simulation to obtain an overall cost outcome distribution. This is because range

estimating does not model the cost impacts of each risk, but the cost range (resulting from many risks) of

critical items in the estimate. Another challenge is that range estimating recommends that the team

consider the extremes for the minimum and maximum cost of critical items and it is difficult, if not

impossible, to parse the impact of any particular risk. For these reasons, it is not the preferred

combination of methods. However, if care is take in not double counting the impact of system and project-

specific risks, the cost values at the various levels of probability can be added for these two methods.

Summary

It is hoped that enough information is provided in this RP to help guide practitioners in developing or

selecting appropriate methods for their situation. Users are encouraged to study the reference materials

provided with this RP. Future revisions of the RP are expected to cover scheduling applications.

REFERENCES

1. AACE International, Recommended Practice No. 18R-97, Cost Estimate Classification SystemAs

Applied in Engineering, Procurement, and Construction for the Process Industries, AACE

International, Morgantown, WV, (latest revision).

Copyright 2009 AACE International, Inc.

AACE International Recommended Practices

Risk Analysis and Contingency Determination using Parametric Estimating

January 17, 2009

8 of 8

2. Baccarini, David, The Maturing Concept of Estimating Project Cost Contingency - A Review, 31st

Australasian University Building Educators Association Conference (AUBEA 2006): July, 2006

(http://espace.lis.curtin.edu.au/).

3. Flyvbjerg, B., From Nobel Prize to Project Management: Getting Risks Right, Project Management

Journal, August 2006.

4. Hollmann, John K., The Monte-Carlo Challenge: A Better Approach, AACE International

Transactions, AACE International, Morgantown, WV, 2007.

5. Hackney, John W. (Kenneth H. Humphreys, Editor), Control and Management of Capital Projects,

2nd Edition, AACE International, 1997.

6. Merrow, Edward W., Kenneth E. Phillips, and Christopher W. Meyers, Understanding Cost Growth

and Performance Shortfalls in Pioneer Process Plants, (R-2569-DOE), RAND Corporation, 1981.

7. Trost, Steven M. and Garold D. Oberlender, Predicting Accuracy of Early Cost Estimates Using

Factor Analysis and Multivariate Regression, Journal of Construction Engineering and Management,

Volume 129, Issue 2, pp. 198-204 (March/April 2003)

8. AACE International, Recommended Practice No. 10S-90, Cost Engineering Terminology, AACE

International, Morgantown, WV, (latest revision).

9. Hollmann, John K., Editor, Total Cost Management Framework: An Integrated Approach to Portfolio,

Program, and Project Management, (Chapter 7.6 Risk Management), AACE International,

Morgantown WV, 2006.

10. Whiteside, J.D., Contingency Modifier Tool, AACE International Transactions, AACE International,

Morgantown, WV, 2007.

11. Rothwell, Dr. Geoffrey, Cost Contingency as the Standard Deviation of the Cost Estimate, Cost

Engineering, AACE International, Morgantown, WV, July 2005.

12. AACE International Recommended Practice No. 43R-08, Risk Analysis and Contingency

Determination Using Parametric Estimating Example Models As Applied for the Process Industries,

AACE International, Morgantown, WV, (latest revision)

CONTRIBUTORS

John K. Hollmann, PE CCE CEP (Author)

Rodney B. Adams, CCE

Hubertus M.T. Brandts, CCE

Alan J. Chilcott, CCT CCE

Dr. Ovidiu Cretu, PE

Charles J. Pospisil

Chinnadurai Ramachandran

Dr. Maarten S.A. Vrijland

Robert F. Wells, CEP