John Allen Paulos
Mr. Paulos, in writing this thought, was rephrasing Albert Einstein's familiar caution: "Not everything that can be counted counts, and not everything that counts can be counted."
John Paulos goes on to say that more often than not it's not the statistical tests applied to data observations, although it is important to choose wisely about what to observe [Dr Einstein's point], but rather the really important thing is what we do to prepare before data is gathered, and what we do with the conclusions and analysis once data is gathered and processed.
Part of the preparation is constructing a data dictionary that gives full explanation and definition to the data elements in the observation. Ambiguities of definition will invalidate or damage any credibility of subsequent analysis. Second, analysts should identify ambiguous cause-and-effect that might erroneously show functional relationships where they don't actually exist.
And, of course, once data is gathered, it's typical to aggregate data by common affinity. Affinity is a matter of judgment, and judgment is subject to many biases. So, now we see the opportunity for qualitative biases to color objective facts.
Of course, Einstein is telling us that sometimes the stuff that counts is not actually countable. This is an entry point for utility assignments. Utility is a tool for giving relative quantitative weights to qualitative properties so that some of our counting tools can be used. Having made this point, now cycle back to John Paulos's caution and begin again!
