What Do You Really Know?

 by Ira J. Rimson and Ludwig Benner, Jr.

Page:  1 | 2 | 3 | 4

Inputs to risk analysis include both actual historical occurrence data and estimated data. Actual data are derived from documentation of past events. Risk is often analyzed with logic trees showing the progression of coupled events and conditions as inputs leading to some outcome, with numerical probability values assigned to the coupled elements and outcomes based on actual occurrence data, or "expert" estimates based on estimated data. Our challenge: Are those risk inputs and resultant probability outputs accepted uncritically? Do analysts thoroughly examine the epistemology of the problem?

Paraphrasing Yogi Berra,

Good past data has a lot of good in it, but it's the bad side that's bad.⁷

How do analysts determine whether what they think they know — the data they use — belongs to the "good side" or the "bad side?" Were analysts to substitute the probability <P=1> for fractional probabilities in all logic tree analyses, would the outcome require further analysis to determine the real risk?⁸

More attention needs to be focused on answers to the questions "what do we need to know"? and "what is the validity of what we think we know?" Boorstin's "Illusion of Knowledge" is the product of assumptions multiplied by ignorance. One need merely compare the plethora of contradictory "scientific" studies on the effects of foods, medications, exercise, stress, caffeine, global warming and/or cooling, …ad infinitum, to realize the prevalence of illusory knowledge in our society.⁹ The "good" thing is that the probability that these common manifestations of uncommon ignorance might result in fatal or highly costly outcomes is no better than chance. The "bad" is that the probability of their occurrence is >P=0, and no analyst can predict accurately when that shoe will drop.

The author of a letter published in the most recent eJSS¹⁰ disagreed with our column in V. 43, #2, in which we described the COMAIR flight crew's take-off decision at Lexington, Kentucky, as an error in judging the criticality of the potential outcome(s) of their behavior choice. His arguments typify the absence of epistemological examination common in risk analyses:

I do disagree with a few of the details, such as their assertion that the crew of the airliner on the wrong runway "didn't recognize the potential criticality of the outcome." Of course the crew knew the potential criticality of trying to take off on a short runway, they just didn't know they were doing such a thing. They didn't know that the runway was far too short and decided to give it a go anyway. They were mistaken — they made an error, but not an error in judging the criticality of their decision.

« previous page     next page »