Some "$10 words" in Risk Management

---

When you get into risk management a bit, there are some biggies that get thrown around -- I call them $10 words -- and there are two that more or less divide risk management along the lines of

1. The unknown that is possibly knowable with some legwork
2. The unknown that is likely to remain unknowable

First case
For the first case, this is all about knowledge, the nature of knowledge, and how better to improve knowledge. Generally, this is called "epistemology" (ka-ching: $10 please) -- understanding the nature and scope of knowledge.

Risks that are subject to a better understanding by simply (I say simply, but often it's not simple) digging out more information are called "epistemic risks"

More simply yet, epistemic risks are those you can do something about -- they are actionable by nature of greater understanding and knowledge development.

In other words, epistemic risks are those for which the uncertainty can reduced -- if you spend the money to try.

And here's another opportunity to make $10: epistemic risks can be set in an organized framework of knowledge, said knowledge frameworks are called ontologies, and so epistemic risks are sometimes called ontological risks (OMG! this just gets better and better)

Second case
For the second case, it's all about the hidden, latent, unknowable (you may not find out what you don't know to ask, etc) that just happens by chance. For example, games of chance, like dice, you simply have no way of knowing what is going to come up next. There's no question you can ask to find out. And, the games are "memoryless" and thus independent; the former outcome has no bearing -- seemingly -- on the next outcome.

Such risks are called aleatoric risks, from the word aleatory meaning "related to random choice or outcome"

The good news, if there is any, is that aleatoric risks have probability distributions that is quantitative description of their random outcomes. If you can discover something about the distribution, you have something to work with re mitigating effects.

Operationally, you can't really reduce the uncertainty surrounding aleatoric risks, but you can immunize your project to the random or chance outcomes -- within limits of course -- by providing slack, buffers, redundancy, loose coupling, etc. In other words, to make the project less fragile and susceptible to the shock of a such a risk outcome.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Fixed price contracting and Agile

---

In my book, Project Management the Agile Way, now in its 2nd Edition, I make this statement in the chapter on contracts:

Firm Fixed Price (FFP) completion contracts are inappropriate for contracting agile.

I got an email from a reader challenging that assertion, to which  responded:

I always start by asserting that agile is a "fixed price" methodology. But, there is a big difference between a contract for your best effort at a fixed price, and a fixed price contract for working product, to wit: completion (agile manifesto objective)

There's no problem whatsoever in conveying best effort at fixed price through a contract mechanism; it's quite another thing to convey fixed price for a working product.

Agile is a methodology that honors the  plan-driven case for strategic intent and business value; but agile is also a methodology that is tactically changeable -- thus emergent in character -- re interpretation of the plan.

Using the definition that strategic intent is the intended discriminating difference to be attained in the "far" future that has business value, a project can be chartered to develop the product drivers for that discriminating difference. Thus,  think of agile as iterative tactically, and plan-driven strategically. The sponsor has control of the strategy; the customer has control of many of tactics.

Re FFP specifically, I was aiming my arguments primarily at the public sector community, and particularly the US federal acquisition protocols. The public sector -- federal or otherwise -- usually goes to a great deal of trouble to carefully prescribe contract relationships, and the means to monitor and control scope, cost, and schedule.

In particular, in the federal sector, only the "contracting officer" -- who is a legal person usually -- has the authority to accept a change in the written description of scope, a change in the cost obligated, or a change in the delivery schedule and location. The CO ordinarily has one or more official "representatives" (CORs) or technical representatives (COTRS) that are empowered to "interpret" changes, etc,

In the commercial business domain, the concepts of contract protocols are usually much more relaxed, starting with the whole concept of a CO and COR -- many businesses simply don't have a CO at all... just an executive that is empowered to sign a PO or a contract. Thus, there are many flexibilities afforded in the private sector not available to public sector

When I say "fixed price", in effect I am saying "not cost reimbursable". Cost reimbursable is quite common in science and technology contracts in the public sector, but almost never in the "IT" sector, public or private. So, I find that many IT execs have little understanding why you might write a contract for a contractor to take your money and not pledge completion.

Working from the perspective of "not cost reimbursable",  I make the point about a FFP completion contract as distinct from other forms of fixed price arrangements, like best effort. In my opinion, agile is not an appropriate methodology for a completion contract in the way in which I use the term, to wit: Pass me the money and I will "complete the work" you describe in the contract when we sign the deal.

However,  there are FP alternatives for a traditional completion effort, the best of the lot in my opinion being a FP framework within which each iteration being a separate and negotiated fixed scope and fixed price job order wherein the job order backlog is planned case by case.

However, even in such a JO arrangement, the customer is "not allowed" to trade or manage the backlog in such a way that the business case for the strategic value is compromised. The project narrative must be "stationary" (invariant to time or location of observation); although the JO nuts and bolts can be emergent.

Typically if the agile principle of persistent team structure is being followed, where the team metrics for throughput (velocity x time) are benchmarked, then the cost of a JO is almost the same every time -- plus or minus a SME or special tool -- and thus the "price" of the contract is "fixed" simply by limiting the number of JOs what will fit within the cost ceiling.

There are other factors which are vexing in the public sector in a FP contract arrangement:
1. Agile promotes a shift in allegiance from the specification being dominant to the customer needs/wants/priorities being dominate. Try telling a CO you are not going to honor the spec as the first precedence!

2. Following on from a shift in allegiance, what then is the contractual definition of "done". Is the project done when the money runs out (best effort); when the backlog is exhausted (all requirements satisfied); or the customer simply says "I've got what I want"? This debate drives the COs nuts.

3. How does a COTR verify and validate (V and V)? In the federal sector, V and V is almost a religion. But, what's to be validated? Typically, verify means everything that is supposed to be delivered got delivered; validated means it meets the quality standard of fitness for use. If the scope is continuously variable, what's to be verified? What do you tell the CO?

4. Can the "grand bargain" be contracted? I suggest a "grand bargain" between sponsor and PM (with customer's needs in the frame) wherein for a fixed investment and usually a fixed time frame the PM is charged with delivering the best value possible.

Best value is defined as the maximum scope (feature/function/performance) possible that conforms to the customer's urgency/need/want as determined iteratively (somewhat on the fly). Thus requirements are allowed to be driven (dependent) by customer's direction of urgency/need/want and available cost and schedule (independent).

Where the customer doesn't usually get a vote is on the non-functional requirements, especially those required to maintain certifications (like SEI level or ISO), compliance to certain regulations (particularly in safety, or some finance (SOx)), or certain internal standards (engineering or architecture)

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

About facts, one more time

---

My favorite expression about facts and estimates--given to me by Dr. David Hulett who was the PMI chair for PMP Chapter 11--is this one:

There are no facts about the future

Now, I see this one posted at Critical Uncertainties, attributed to Friedrich Nietzcshe:

There are no facts, only interpretations 

And, so what do we make of this ... particularly in our current era of "alternative facts"?

Begin here: set aside the idea of "alternative facts"; then consider:
It's all about bias -- we all have a biases, and thus can we ever say anything with true objectivity?

Well, yes, of course, there are immutable international standards for measurements; and measurements certainly make up a lot of "facts", though even here we find arguments about angels on the head of a pin (See Einstein, and the theories of relativity that demonstrate the flexibility of time and space)

And, of course, just put a measuring probe on some things changes them so much that we can't objectively measure them.

And then there is quantum physics with those theories of non-deterministic location; and entanglements that seem provide connectivity where there is none.

Should I go on?

Probably Nietzcshe had it right.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

We don't do manifestos

---

I wrote this a couple of years ago; not much has changed:

You gotta love the first bullet from this conclusion by Stephen Welby, Deputy Assistant Secretary of Defense for Systems Engineering, speaking at the AFEI Agile for Government Summit, November 21, 2013

When he says in the 4th bullet point "upcoming revisions to 5000.2", he is referring to the DoD instruction for operation of the Defense acquisition system, revised in the fall of 2013, and then revised again in January, 2015.

This latest version of the instruction describes six models of how to acquire systems, the third of which, Model 3, describes an incremental methodology that is "agile" like. Of course, there are a lot of non-development swim lanes and pre- and post- tasks, as you would expect in an organization accustomed to working at large scale.

There are, of course, a myriad of "how to" acquisition guides for the program manager. Mitre, a DoD think tank contractor, has a decent acquisition guide written in 2014.

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Ride a dead horse

---

A paraphrase:

The code of tribal wisdom says that when you discover you are riding a dead horse, the best strategy is to dismount. In the [project office], we often try other strategies with dead horses, including the following:

• Buying a stronger whip;
• Changing riders;
• Saying things like ‘this is the way we’ve always ridden the horse;
• Appointing a committee to study the horse;
• Arranging a visit to other [PMOs] to see how they ride dead horses;
• Increasing the standards to ride dead horses;
• Declaring the horse is better, faster, cheaper dead; and finally 
• Harnessing the dead horses together for increased speed 

Thomas Penfield Jackson

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Distorted reality field

---

Imaginative innovation requires acceptance of a distorted reality field

Paraphrase of an observation and commentary by Walter Isaacson

"Distorted reality?
How does that fit with "we're all in this together"; committed scope-cost-schedule-quality; risk management; and "other people's money"?

When I think PMO, I think: Orthodox PM; rules; constraints; commitments; and all the rest
I'm not thinking "distorted reality"

But wait! How about "imagineering"? Disney has it; so could you! Just put the distorted realists off in their world, throw in money, delete for clarity (DFC) all bureaucracy; and sit back, but don't look down.

You might get the next big thing; you might get an Edsel.  

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

The essence of bureaucracy?

---

All projects of any scale are bureaucratic, meaning:

•  there is some pecking order, 
• some chain of decision making -- especially when there is "other people's money" funding the project -- and 
• some means for assigning responsibilities (followed, consequentially, by measurements and accountability)

And so, I was struck by this passage which asserts the "essence of bureaucracy": (my emphasis added)

"... with writing all of [the project] minutiae are written somewhere, and the assemblage of relevant documents defines the identity and power of [the project office].

Writing has thus enabled humans to organise entire societies in an algorithmic fashion. ...  In illiterate societies people make all calculations and decisions in their heads.

In literate societies people are organised into networks, so that each person is only a small step in a huge algorithm, and it is the algorithm as a whole that makes the important decisions.

This is the essence of bureaucracy."

"Homo Deus: A Brief History of Tomorrow" 

by Yuval Noah Harari

In other words: The algorithm made me do it! 

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Observation and imagination

---

... in a nutshell, [this] was Lenonardo's signature talent: the ability to convey, by marrying observation with imagination, "not only the works of nature but also infinite things nature never created"

Walter Isaacson in his history "Lenonardo da Vinci"

And, how many of us have Lenonardo's instincts of observation -- on the one hand -- which are then drivers for something unimagined?

• How would you handle a Lenonardo on your team? (Did I mention his total disregard for schedule; his somewhat arrogant approach to customers; and the fact he finished modestly few projects)
• How does your stuffy project office handle the imagination thing, one might ask? (I'm sure observations are just fine in the PMO)

In an earlier posting on Leonardo, I quoted another historian:

Alas! This man will do nothing at all, since he is thinking of the end before he has made a beginning. ... In his imagination, he frequently formed enterprises so difficult and so subtle that they could not be realized and worthily executed by human hands. His conceptions were varied to infinity

On the other hand, speaking of ".... conceptions varied to infinity". Is that all bad?

• Who knew we needed a smart phone before someone imagined it? 
• And, thousands of other examples over history, probably going back to the bow and arrow -- and before (See: imagining ancient pyramids, and yes, some of them were used for grain storage to buffer bad harvests)

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

A grand bargin: Agile in the enterprise

---

In my book, Project Management the Agile Way, now in its second edition, I make this statement in the chapter on contracts:

Firm Fixed Price (FFP) completion contracts are inappropriate for contracting agile.

I got an email from a reader challenging that assertion, to which I responded:

I always start by asserting that agile is a "fixed price" methodology. But, there is a big difference between a contract for your best effort at a fixed price, and a fixed price contract for working product, to wit: "fixed scope completion"

There's no problem whatsoever in conveying best effort at fixed price through a contract mechanism; it's quite another thing to convey fixed price for a working product of fixed scope.

Agile is a methodology that honors the  plan-driven case for strategic intent and business value; but agile is also a methodology that is tactically changeable -- thus emergent in character -- re interpretation of the plan.

Using the definition that strategic intent is the intended discriminating difference to be attained in the "far" future, a project can be chartered to develop the product drivers for that discriminating difference. Thus,  think of agile as iterative tactically, and plan-driven strategically.

The sponsor has control of the strategy; the project team has control of many of the tactics.

Re FFP specifically, I was aiming my arguments primarily at the public sector community, and particularly the US federal acquisition protocols. The public sector -- federal or otherwise -- usually goes to a great deal of trouble to carefully prescribe contract relationships, and the means to monitor and control scope, cost, and schedule.

In particular, in the federal sector, only the "contracting officer" -- who is a legal person usually -- has the authority to accept a change in the written description of scope, a change in the cost obligated, or a change in the delivery schedule and location. The CO ordinarily has one or more official "representatives" (CORs) or technical representatives (COTRS) that are empowered to "interpret" changes, etc,

In the commercial business domain, the concepts of contract protocols are usually much more relaxed, starting with the whole concept of a CO and COR -- many businesses simply don't have a CO at all... just an executive that is empowered to sign a PO or a contract. Thus, there are many flexibilities afforded in the private sector not available to public sector

When I say "fixed price", in effect I am saying "not cost reimbursable". Cost reimbursable is quite common in science and technology contracts in the public sector, but almost never in the "IT" sector, public or private. So, I find that many IT execs have little understanding why you might write a contract for a contractor to take your money and not pledge completion of the original scope.

Working from the perspective of "not cost reimbursable",  I make the point about a FFP completion contract as distinct from other forms of fixed price arrangements, like best effort. In my opinion, agile is not an appropriate methodology for a completion contract in the way in which I use the term, to wit: Pass me the money and I will "complete the work" you describe in the contract when we sign the deal.

However,  there are FP alternatives for a traditional completion effort, the best of the lot in my opinion being a FP framework within which each iteration being a separate and negotiated fixed scope and fixed price job order wherein the job order backlog is planned case by case.

However, even in such a JO arrangement, the customer is "not allowed" to trade or manage the backlog in such a way that the business case for the strategic value is compromised. The project narrative must be "stationary" (invariant to time or location of observation); although the JO nuts and bolts can be emergent.

Typically if the agile principle of persistent team structure is being followed, where the team metrics for throughput (velocity x time) are benchmarked, then the cost of a JO is almost the same every time -- plus or minus a SME or special tool -- and thus the "price" of the contract is "fixed" simply by limiting the number of JOs what will fit within the cost ceiling.

There are other factors which are vexing in the public sector in a FP contract arrangement:
1. Agile promotes a shift in allegiance from the specification being dominant to the customer needs/wants/priorities being dominate. Try telling a CO you are not going to honor the spec as the first precedence!

2. Following on from a shift in allegiance, what then is the contractual definition of "done". Is the project done when the money runs out (best effort); when the backlog is exhausted (all requirements satisfied); or the customer simply says "I've got what I want"? This debate drives the COs nuts.

3. How does a COTR verify and validate (V and V)? In the federal sector, V and V is almost a religion. But, what's to be validated? Typically, verify means everything that is supposed to be delivered got delivered; validated means it meets the quality standard of fitness for use. If the scope is continuously variable, what's to be verified? What do you tell the CO?

4.  I suggest a "grand bargain" between sponsor and PM (with customer's needs in the frame) wherein for a fixed investment and usually a fixed time frame the PM is charged with delivering the best value possible. Can the "grand bargain" be contracted?  Sure, we usually call it "best value"

Best value is defined as the maximum scope (feature/function/performance) possible that conforms to the customer's urgency/need/want as determined iteratively (somewhat on the fly). Thus requirements are allowed to be driven (dependent) by customer's direction of urgency/need/want and available cost and schedule (independent).

Where the customer doesn't usually get a vote is on the non-functional requirements, especially those required to maintain certifications (like SEI level or ISO), compliance to certain regulations (particularly in safety, or some finance (SOx)), or certain internal standards (engineering or architecture)

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog

Overfit v underfit

---

You've got data!

That's a good start. Now, working back to cause for these effects, what model fits the data? If you get the model right, you can forecast (gasp! estimate) what comes next.

You can make two errors, both of which could be costly, but one more than the other:

1. Underfit the data. Meaning: a "too tight" fit such that some data fits very well, and other data not so well. The danger here is that the "good fit" stuff may actually be only a selection of outliers and ill effects. Thus, the real causation is missed
2. Overfit the data. Meaning: a "too loose" fit such that too many ill effects and outliers are included and thus too many causations are possible and the model is too sloppy to be meaningful

Now, in practice, the "underfit" is most common. Why? Because with the tight fit it looks really good on a PowerPoint slide and thus wins the day in the briefing.

But, come reality, the underfit model breaks down, and the estimating naysayers say nay to estimating. Who can blame them?

... it may look superficially more impressive until then, claiming to make very accurate and newsworthy predictions and to represent an advance over previously applied techniques. This may make it easier to get the model published in an academic journal or to sell to a client, crowding out more honest models from the marketplace. But if the model is fitting noise, it has the potential to hurt the science."

"The Signal and the Noise: Why So Many Predictions Fail-but Some Don't" 

by Nate Silver

---

Read in the library at Square Peg Consulting about these books I've written
Buy them at any online book retailer!

Read my contribution to the Flashblog