Budget wisdom

---

"Priorities aren't redal unless budgets reflect them"

CIA Director Burns

Of course, Director Burn's assertion is spot-on and another version of "show me the money", or perhaps sticking with the intelligence domain: "follow the money".

This whole idea is the bane of strategic planning in which long-term plans outrun the budget authority and even outrun the budget planning, in other words: a floating apex, unsupported by a pyramid of budgets.

Nonetheless, lightening could strike. Having an idea on the shelf is not all bad. But if it's technology, it has a half life measured in a year or two. So, a constant dusting to keep current is required. Who knows: the money might show up.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

AI-squared ... a testing paradigm

---

AI-squared. What's that?

Is this something Project Managers need to know about?

Actually, yes, PMs need to know that there are entirely new test protocols coming that more or less challenge some system test paradigms that are at the heart of PM best practice.

AI-squared

That's using an AI device (program, app, etc.) to validate another AI device, sometimes a version difference of itself! Like GPT-2 validating -- or supervising, which is a term of art -- GPT-4. (Is that even feasible? Read on.)

As reported by Matteo Wong, all the AI firms, to include OpenAI, Microsoft, Google, and others are working on some version of "recursive self-improvement (Sam Altman), or as OpenAI researchers put it: the "alignment" problem which includes the "supervision problem", to use some of the industry jargon. 

From a project development viewpoint, these techniques are close to what we traditionally think of as verification that results comport with the prompt, and validation that results are accurate. 

But in the vernacular of model V&V, and particularly AI "models" like GPT-X, the words are 'alignment' and 'supervision'

• Alignment is the idea of not inventing new physics when asked for a solution. Whatever the model's answer to a prompt is, the prompted answer has to "align" with the known facts, or a departure has to be justified. One wonders if Einstein (relativity) and Planck (quantum theory) were properly "aligned" in their day. 
  
  
• 'Supervision is the act of conducting V&V on model results. The question arises: who is "smarter": the supervisor or the supervised? In the AI world, this is not trivial. In the traditional PM world, a lot of deference is paid to the 'grey beards' or very-senior tech staff as the font of trustworthy knowledge. This may be about to change.

And now: "Unlearning"!

After spending all that project money on training and testing, you are now told to have your project model "unlearn" stuff. Why?

Let's say you have an AI engine for kitchen recipes, apple pie, etc. What other recipes might it know about? Ones with fertilizer and diesel? Those are to be "unlearned"

One technique along this line is to have true professional experts in the domains to be forgotten ask nuanced questions (not training questions) to ascertain latent knowledge. If discovered, then the model is 'taught to forget'. Does this technique work? Some say yes.

 

What to think of his?

Obviously, my first thought was "mutual reinforcement" or positive feedback ... you don't want the checker reinforcing the errors of the checked.  Independence of the developers by the testers has been a pillar of best-practices project process since anyone can remember.

OpenAI has a partial answer to my thoughts in this interesting research paper.

But there is the other issue: so-called "weak supervision" described by the OpenAI reseachers. Human developers and checkers are categorized as "weak" supervisors of what AI devices can produce. 

Weakness arises by limited by time, by overwhelming complexity, and by enormous scope that is economically out of reach for human validation. And, humans are susceptible to biases and judgments that machines would not be. This has been the bane of project testing all along: humans are just not consistent or objective in every test situation, and perhaps from day to day.

Corollary: AI can be, or should be, a "strong supervisor" of other AI. Only more research will tell the tale on that one.

My second thought was: "Why do this (AI checking AI)? Why take a chance on reinforcement?" 

The answer comes back: Stronger supervision is imperative. Better timeliness, better scope, and improved consistency of testing, as compared to human checking, even with algorithmic support to the human. 

And of course, AI testing takes the labor cost out of the checking process for the device. And reduced labor cost could translate into few jobs for AI developers and checkers.

Is there enough data?

And now it's reported that most of the low hanging data sources have been exploited for AI training. 

Is it still possible to verify and validate ever more complex models like it was possible (to some degree) to validate what we have so far?

Unintelligible intelligence

Question: Is AI-squared enough, or does the exponent go higher as "supervision" requirements grow because more exotic and even less-understood AI capabilities come onto the scene?

• Will artificial intelligence be intelligible? 
• Will the so-called intelligence of machine devices be so advanced that even weak supervision -- by humans -- is not up to the task? 

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Senior Leadership

---

"I don’t think you can be good at these jobs unless you’re willing to lose them. You have to get your mind at a stage in your life and career where the best move to make could put yourself in jeopardy to losing your job, but it’s the best move to make"

Paul D. Ryan

In many respects Ryan's "put yourself out there" advice for successful senior leadership is a great discriminator between leadership and managership, and certainly is a "lean into it" risk attitude.

Who said compromise?
Possibly hidden behind the words, though Ryan didn't say it, is a willingness for pragmatic compromise that is not long-term in violation of principles. In other words: strategic consistency while also entertaining agile tactics.

We need managers also
But I'm the first to say it takes all kinds to make a project team, and everyone can't be on the edge, thereby putting stability, predictability, and reliability at stake, all the time, on everything. 

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

X-37B on the way

---

Some projects just look really swell. 

• Falcon Heavy with X-37B (mission 7) Orbital Test Vehicle on-board. MORE
• An award wining photograph, shot by professional photographer Pascal Fouquet
• From 14 miles out from the launch pad.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Heat Batteries

---

Does your project need green energy for development purposes, or is your project incorporating green energy in a deliverable?

Perhaps a Heat Battery fits your need.

As reported by the WSJ:

".... researchers are developing heat batteries—also called thermal batteries—that store renewable energy as heat and then release it on demand to power industrial processes.

Traditional batteries store and release power by moving lithium ions through a liquid from the cathode to the anode, and back again. They are great when space is at a premium, as is the case inside EVs. But they are relatively expensive and typically can only discharge energy for a few hours, limiting their industrial applications.

Heat batteries, on the other hand, work by passing current through a resistor to heat some type of material that can stay hot for days—such as bricks, rocks or molten salt. These materials can store energy generated from intermittent renewable sources as heat—and then release it on demand whenever it’s needed"

There are companies in various stages of readiness on heat batteries. Some actually delivering product, and others are still researching the opportunity. 

This may be a "hot" topic in the future!

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

From soda straws to 'constant stare'

---

Does your project have proprietary or other intellectual property that is, of necessity, our of doors?

Specialized antennas, telescopes, and other sensors?

Unique infrastructure or private facilities?

Proprietary ground or air or even space and underwater vehicles, or vehicle performance?

New, competitive installations?

In the past, you could keep the wraps on by simply hiding the location, or restricting access and observation from the ground.

There were a handful of earth-observing satellites, mostly run by governments, that had a 'soda-straw' look at any point on earth, and then often only for a short time. Revisit rates varied from a couple of hours to perhaps a geo-stationary stare until some other mission had to be satisfied.

With predictable orbits and observation parameters, the ground target could take countermeasures.

Then came drones, with long time-over-target durations, but nonetheless limited by fuel and other mission assignments. But drones are not altogether stealthy, as yet, and so there are countermeasures that can be employed by the target.

Constant stare: 

Now comes 'constant stare', the conception being 24x7 global real time observation of anywhere. Tried and true countermeasures go out the window.

To get to constant stare, perhaps thousands of satellites, about the size of a loaf of bread are to be deployed. And for the most part this capability is provided by civilian reconnaissance companies whose objective is to monetize this service.

A recent essay by David Zikusoka makes this observation:

"..... AI has enabled the teaming of humans and machines, with computer algorithms rapidly sifting through data and identifying relevant pieces of information for analysts. 

Private satellite-launching companies such as SpaceX and Rocket Lab have leveraged these technologies to build what are termed “megaconstellations,” in which hundreds of satellites work together to provide intelligence to the public, businesses, and nongovernmental organizations. These companies are updating open-source, planet-scale databases multiple times per day. Some of these companies can deliver fresh intelligence from nearly any point on the globe within 30 minutes of a request."

On the risk register

This stuff needs to get on the risk register.

So, first balloons, then the occasional overflight, more recently drones, and soon to come 'constant stare'. 

When constructing the risk register, and considering the time-sensitivity of proprietary project information, take into account that others may be observing, measuring, and analyzing right along side your project team.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Help coming: IT Risk Management

---

Risk Management in IT projects 

For years (a lot of years) IT companies have been paying bounties to hackers to find vulnerabilities in target IT systems and report them to bug fixers before they become a business hazard. This bounty system has worked for the most part, but it's a QC (find after the fact) rather than QA (quality built-in) approach, somewhat of necessity given the complexity of IT software systems. 

Enter AI agents

Now, of course, there is a new sheriff in town that aims more at QA than QC: AI bug detectors based on the large language models (LLM) that can be deployed to seek out the bug risks earlier in the development and beta cycles.

For a deep dive, you can read this paper from researchers at Cornell University.

But the idea is summarized by Daniel Miessler this way:

The way forward on automated hacking is this: 1) teams of agents, 2) extremely detailed capture of human tester thought processes, lots of real-world examples, and time. I suspect that in 2-5 years, agent-based web hacking will be able to get 90% of the bugs we normally see submitted in web bug bounties. But they’ll be faster. And the reports will be better. That last 10% will remain elusive until those agents are at AGI level.

Zero Trust

CISA, the nation's cyber-defense agency, is continuing its 'zero trust' IT systems imitative, now with an office dedicated to the program. Some of the program details are found here, including information about the Zero Trust Security Model.

 

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Redesigning the "Meeting"

---

From Connor Grant at the WSJ:

The traditional business meeting is changing; the 'pandemic' made me do it!

Here is Grant's reporting -- somewhat abridged -- on changes now in place and expected to come:

1. More office meeting rooms will have high-tech equipment such as holograms, virtual reality and other immersive technologies that allow remote workers to feel like they are in the same room as their in-office colleagues ....

2. Employers will conduct walk-and-talk meetings outside, reducing the amount of time spent looking at—or being distracted by—screens. 

3. Managers will "pregame" meetings by asking workers to add thoughts, ideas or feedback to a shared meeting document at least a week in advance. 

4. Some companies will have once-a-quarter retreats at hotels or co-working spaces

5. Businesses will use mixed-reality tools to supercharge premeeting preparation ... [and] receive real-time feedback when they practice presentations or conversations.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!

Some Big Words about the Risk Register

---

Every PMO plan includes some form of risk management, and a favorite way to communicate risk to your team, sponsors, and other stakeholders is the (ageless) risk register.

So much has been written about the ubiquitous risk register, it's a wonder there is anything more to be said. But here goes:

In simplest terms, the risk register is a matrix of rows and columns showing the elements of expected value:

• Rows identify the risk impact and give it some weight or value, which can be as simple as high, medium, or low. But if you have information -- or at least an informed guess -- about dollar value, then that's another way to weight the risk impact value.
  
  
• Columns identify the probability of the impact actually occurring. Again, with little calibrated information, an informed guess of high, medium, or low will get you started. 
  
  
• The field of column-row intersections is where the expected value is expressed. If you're just applying labels, then an intersection might be "high-medium" row by column. Statistically you can't calculate anything based on uncalibrated labels, but nonetheless the "low-low" are not usually actively managed, and thus the management workload is lessened.

But, there is more to be said (Big words start here)

Consider having more than one matrix, each matrix aligned with the nature of the risk and the quality of the information.

White noise register: One class of risks are the so-called "white noise" risks which are variously called stochastic or aleatory risks; they have three main characteristics:

1. They are utterly random in the moment, but not necessarily uniformly or bell shaped in their distributions.
2. They have a deterministic -- that is, not particularly random and not necessarily linear -- long-term trend or value. Regression methods can often times discover a "best fit" trend line.
3. Other than observe the randomness to get a feel for the long term trend and to sort the range of the "tails", or less frequently occurring values, there's not much you can do about the random effects of "white noise"

Aleatory risks are said to be "irreducible", meaning there is nothing about the nature of the risk that can be mitigated with more information. There are no information dependencies.

Epistemic risks are those with information dependencies. Epistemic risks could have their own risk register which identifies and characterizes the dependencies:

• Epistemic risks are "reducible" with more information, approaching -- in the limit -- something akin to a stochastic irreducible risk. 
• An epistemic risk register would identify the information-acquisition tasks necessary to manage the risks. 

Situationally sensitive Idiosyncratic risk register: Idiosyncratic risks are those that are a peculiar and unique subset of a more general class. Idiosyncratic risks are unique to a situation, and might behave differently and be managed differently if the situation changed.  And so the risk register would identify the situational dependency so that management actions might shift as the situation shifts.

Hypothesis or experiment driven risks are methodologically unique. When you think about it, a really large proportion of the risks attendant to projects fall into this category. 

With these types of risks we get into Bayesian methods of estimating and considering conditional risks where the risk is dependent on conditions and evidence which are updated as new observations and measurements are made.

These risks certainly belong on their own register with action plans in accord with the methodology below.  The general methodology goes like this:

• Hypothesize an outcome (risk event) and 
• Then make a first estimate of the probability of the hypothesized event, with and without conditions.
• Make observations or measurements to confirm the hypothesis
• If unconfirmed, adjust the estimate of conditions, and repeat until conditions are sufficiently defined to confirm the hypothesis
• If no conditions suffice, then the hypothesis is false. Adjust the hypothesis, and repeat all. 

Pseudo-chaotic risks: These are the one-off, or nearly so, very aperiodic upsets or events that are not stochastic in the sense of having a predictable observable distribution and calculable trend. Some are known knowns like unplanned absences of key personnel. 

Anti-fragile methods: Designing the project to be anti-fragile is one way to immunize the project from the pseudo-chaotic risks. See my posts on anti-fragile for more.

Bottom line: take advantage of the flexibility of a generic risk register to give yourself more specificity in what you are to manage.

---

Like this blog? You'll like my books also! Buy them at any online book retailer!