"Uncertainty is an essential and nonnegotiable part of a forecast. .... sometimes an honest and accurate expression of the uncertainty is what has the potential to save [big things].... However, there is another reason to quantify the uncertainty carefully and explicitly. It is essential to scientific progress, especially under Bayes’s theorem."
"The Signal and the Noise: Why So Many Predictions Fail-but Some Don't"
by Nate Silver
Now, some say: "we don't estimate; we don't forecast".Of course, that's nonsense. Everyone estimates, if even only in their head
- How long will it take me to write this blog?
- How long will it take me to go to lunch?
- How long will it take me to do almost anything I can think of?
- Formulate an issue or question or hypothesis
- Make an early guess as to outcome
- Experiment to gather evidence as to whether or not the guess is reasonable
- Re-formulate based on evidence -- or lack thereof
- Repeat as necessary
"In science, one rarely sees all the data point toward one precise conclusion. Real data is noisy—even if the theory is perfect, the strength of the signal will vary. And under Bayes’s theorem, no theory is perfect. Rather, it is a work in progress, always subject to further refinement and testing. This is what scientific skepticism is all about."And, one last caution from author Silver -- which reinforces the ideas of the Bayes process and also makes the point -- often ignored or overlooked -- that there is often little enough data inside one-time projects to support textbook statistical approaches:
"As we have learned throughout this book, purely statistical approaches toward forecasting are ineffective at best when there is not a sufficient sample of data to work with."
Buy them at any online book retailer!
