What Do You Really Know?

 by Ira J. Rimson and Ludwig Benner, Jr.

Page:  1 | 2 | 3 | 4

The greatest obstacle to discovering the shape of the Earth, the continents and the oceans was not ignorance, but the Illusion of Knowledge.
— Daniel Boorstin, Librarian of the U.S. Congress, Emeritus

In his paper "What is Missing? A Philosophy of Risk Management,"¹ Michael Murphy² argues that high-cost risk management failures might be mitigated, were risk managers to give greater credence to the major traditional branches of philosophy. He listed 15 well-known cases of risk management failures that occurred in recent history, from the Chevrolet Corvair to the Walkerton, Ontario, e. coli outbreak, to Enron, and enumerated the influences of lack of consideration for philosophic precepts in their systems' ultimate collapse. Among the five major philosophical branches, two are represented in all cases on Murphy's list: Metaphysics and Logic. Metaphysics is comprised of Ontology (what exists in reality vs. what does not; i.e., reality vs. perception) and Epistemology (the theory of knowledge, its limits and the validity of what we know). Logic defines the validity of inference and reason, which overlies the entirety of system development. We have chosen to concentrate this column on the issue of epistemology because, in our experience, it is the philosophical discipline most commonly violated by risk managers and analysts. It is also likely the easiest to interject into current system safety practices.

Recent risk-management failures have continued to demonstrate the need to expand traditional analytic methodologies to include "what did we know?"; "what did we need to know?"; and "how accurate and valid were our data and the assumptions we derived from them?" We have emphasized in prior columns that humans are the principal — and least predictable — variables in systems' operation. The risks inherent in inanimate systems are calculable. The infinite intricacies of human behaviors bias and change those risks beyond capability for reliable mathematical calculation. Such simplistic explanations as "It was an error," or "We didn't anticipate that [X] would respond that way," are useless for establishing any kind of behavioral change that might preclude recurrence.

Epistemology is the philosophy of knowledge, and is useful for determining what we need to know, how much we really know compared to what we need to know,³ and the degree of uncertainty of what we think we know.

The London School of Economics' Risk and Regulation Magazine recently published a paper by Nicholas Taleb and Avital Pilpel titled "Epistemology and Risk Management," in which they argue:

…[T]he production of a risk "measure" must be subjected to the question "how do you know what you claim to know" .... Claims regarding risk cannot be made without any rigorously established supervision of their validity. There is a need for skeptical inquiries concerning how a risk measure was obtained and how an opinion was formed.⁴

We believe that absence of skeptical inquiry is a critical inadequacy of current system safety practice. Skepticism is particularly significant when risk analyses are derived from historic data that analysts assume to be accurate predictors of future events that might bear little, if any, resemblance to the assumed risk inputs. Analysts don't often delve into the details of their assumed "precursors," least of all into behavior of human players in the system. Most system safety analyses relegate human operator behavior to a binary mode: either the operator(s) perform according to preprogrammed expectation — doing what they are supposed to — or they don't — not doing what they are supposed to. Analysts rarely consider the infinite variety of alternative responses that human behavior can instantiate in the system.⁵

next page »