For many years, I've preached the benefits of the Monte Carlo simulation (MCS). For many reasons, it's superior to other analysis paradigms. In network analysis, for instance, it handles the parallel join or 'gate' as no other method will.
In fact, it's this very capability that renders the Monte Carlo superior to other risk adjusted ideas, like the PERT method in scheduling. PERT suffers from an inability to handle the 'merge bias' at gates because PERT does not handle the mathematics of multiplying distributions (required at gates); PERT only provides a means to calculate a statistic for each task.
Architecturally, gates are the weakest construct in schedules, so any failure to handle them is a show stopper in my mind
But, my students ask me invariably: how can the MCS be any better than the 3-point estimates that go into it; if the 3-pointers are guesses, isnt' the whole MCS thing just a crap shoot?
And, the second thing they ask is: who's got the time to triple the estimating problem (from the most likely single point estimate to the trio of optimistic, pessimistic, and most likely) especially in a network of many hundreds, if not thousands, of tasks?
My answer is this: it depends; it depends on whether or not your are the work package leader concerned for a handful of tasks, or the project manager concerned for a network of hundreds (thousands in some cases) of tasks.
If the former, then you should not guess; you should take the time to consider each estimate, based on benchmarks, so that each estimate is has some auditable calibration to the benchmark.
But if the latter, then let the Central Limit Theorem do the work. A few bad estimates, indeed a lot more than a few in most cases, have negligle impact on the MCS results. In other words, the good, the bad, and the ugly tend to cluster around a central value--a phenomenon called central tendency. Thus, at the PM level, you can live without completely solid calibration. Only a few estimates need to be pretty well thought out vis a vis the O,P,ML triplett.
This may sound like heresey to the calibration crowd, but as a politician recently said: arithmetic! Actually, calculus is the better word, since we have to deal with functions. And it's calculus that gives us the tools to understand that a few bad estimates, even really bad estimates, are largely discounted for effect by the integrated effects of all the other estimates. So, let's not get uptight about a few bad eggs!
Nevertheless, to do a MCS, you do need estimates of O, P, and ML. Where do they come from? All the tools allow for defaulting in a trio to every task. Is that a good idea?
Here's my counsel:
In fact, it's this very capability that renders the Monte Carlo superior to other risk adjusted ideas, like the PERT method in scheduling. PERT suffers from an inability to handle the 'merge bias' at gates because PERT does not handle the mathematics of multiplying distributions (required at gates); PERT only provides a means to calculate a statistic for each task.
Architecturally, gates are the weakest construct in schedules, so any failure to handle them is a show stopper in my mind
But, my students ask me invariably: how can the MCS be any better than the 3-point estimates that go into it; if the 3-pointers are guesses, isnt' the whole MCS thing just a crap shoot?
And, the second thing they ask is: who's got the time to triple the estimating problem (from the most likely single point estimate to the trio of optimistic, pessimistic, and most likely) especially in a network of many hundreds, if not thousands, of tasks?
My answer is this: it depends; it depends on whether or not your are the work package leader concerned for a handful of tasks, or the project manager concerned for a network of hundreds (thousands in some cases) of tasks.
If the former, then you should not guess; you should take the time to consider each estimate, based on benchmarks, so that each estimate is has some auditable calibration to the benchmark.
But if the latter, then let the Central Limit Theorem do the work. A few bad estimates, indeed a lot more than a few in most cases, have negligle impact on the MCS results. In other words, the good, the bad, and the ugly tend to cluster around a central value--a phenomenon called central tendency. Thus, at the PM level, you can live without completely solid calibration. Only a few estimates need to be pretty well thought out vis a vis the O,P,ML triplett.
This may sound like heresey to the calibration crowd, but as a politician recently said: arithmetic! Actually, calculus is the better word, since we have to deal with functions. And it's calculus that gives us the tools to understand that a few bad estimates, even really bad estimates, are largely discounted for effect by the integrated effects of all the other estimates. So, let's not get uptight about a few bad eggs!
Nevertheless, to do a MCS, you do need estimates of O, P, and ML. Where do they come from? All the tools allow for defaulting in a trio to every task. Is that a good idea?
Here's my counsel:
- Take the time to estimate the most likely critical path. The MCS tool will identify other near-critical paths, and may even find an alternate path that is critical (not the one you thought)
- Establish, in the risk management plan, a policy about the defaults values (for % of ML)
The policy may have several elements: hardware, software, administration, and process tasks. Each of these will have hard, medium, and easy tasks.
The policy can be a matrix of O, ML, and P defaults for each (Example: for hard software, the policy is to estimate O = 80% ML and P = 200% ML).
These generally come from experience, and that means from actual results elsewhere, so the defaults are not just picking values out of thin air....there's usually back-up
Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!
Read my contribution to the Flashblog
