Musings on project management: Ooops! Right data; Wrong answer

Thursday, June 19, 2014

Ooops! Right data; Wrong answer

It is certainly true that when working with incomplete or uncertain data sets, even qualitative data sets, it's quite possible to draw a wrong inference from a correct data set.

That is: the data could be correct and true in all respects, but a false conclusion is drawn or a false cause is assigned in the cause-effect relationship.

Of course: how would you know you're drawing a wrong conclusion?

Really, only by experimentation or prototyping. If an experiment, developed from your (false)
conclusion yields inconsistent data that does not fit with the original data set, then you can reasonably conclude that a wrong inference has been drawn.

John Mandel, writing in "The Statistical Analysis of Experimental Data" (now available as an e-book) tell us:

..... inferences drawn by induction from incomplete information may be entirely wrong, even when the information given is unquestionably correct.
... the dependence of inductive inferences [is] not only on the correctness of the data, but also on their completeness.

A simple example is this one:

if one were given the ... pressure and volume of a fixed mass of gas, one might infer, by induction, that the pressure of a gas is proportional to its volume, a completely erroneous statement. The error is due, of course, to the fact that another important item of information was omitted, namely that each pair of measurements was obtained at a different temperature,

So, we must be cautious about how we reason with incomplete data. Mandel goes on: "A. Fisher, one of the founders of the modern science of statistics, has pointed to a basic and most important difference between the results of induction and deduction":

In the latter [deduction], conclusions based on partial information are always correct, despite the incompleteness of the premises, provided that this partial information is itself correct.
For example, the theorem that the sum of the angles of a plane triangle equals 180 degrees ... does not necessitate information as to whether the triangle is drawn on paper or on cardboard.... If information of this type is subsequently added, it cannot possibly alter the fact expressed by the theorem.
On the other hand, inferences drawn by induction from incomplete information may be entirely wrong, even when the information given is unquestionably

Wikipedia puts it this way:

While the conclusion of a deductive argument is supposed to be certain, the truth of the conclusion of an inductive argument is supposed to be probable [or, at least credible], based upon the evidence given