Qualitative or Quantitative Analysis? by Clifton A. Ericson II   System safety analysts are often in a quandary about whether to use a qualitative or quantitative analysis technique. Understanding which analysis type to use, and when, often seems more of an art than a science. The qualitative/quantitative factor is one of the basic attributes of a hazard analysis technique. Most hazard analyses are performed to identify hazards and then determine the mishap risk of the hazard, where mishap risk is defined as Risk=Probability X Severity. The probability risk factor refers to the probability of the mishap actually occurring given the latent hazard conditions, and the severity risk factor refers to the damage and/or loss resulting from the mishap after it occurs. To determine the risk of an identified hazard, a risk characterization methodology must be utilized for the probability and severity parameters. Both quantitative and qualitative risk characterization methods have been developed for use in the system safety discipline. Both approaches are useful, but each contains unique advantages and disadvantages. A qualitative analysis is a non-mathematical process that reviews all factors affecting mishap risk against a predetermined set of parameters. Qualitative analysis involves the use of qualitative criteria in the analysis and provides a qualitative result. Typically, this approach uses categories to separate different parameters, with qualitative definitions that establish the ranges for each category. Engineering judgments are made as to which category something might fit. This approach has the characteristic of being subjective, but it allows more generalization and is therefore less restricting. Arbitrary categories have been established in MIL-STD-882 that provide qualitative measures for the most reasonable likelihood of occurrence of a mishap and for the outcome severity of a mishap. For example, if the safety analyst assesses that an event will occur frequently, it is assigned an index level “A”; or, if it occurs occasionally, it is given an index level “C.” This qualitative index value is then used in qualitative risk calculations and assessments.   "Don't ever confuse mathematical model results with reality." Quantitative analysis involves the use of numerical or quantitative data in the analysis, and provides a quantitative result. This approach has the characteristic of possibly being more objective and more accurate. It should be noted, however, that quantitative results can be biased by the validity and accuracy of the input numbers. For this reason, quantitative results should not be viewed as an exact number but as an estimate, with a range of variability depending upon the quality of the data. In a quantitative analysis, mathematical theories and models are used to calculate mishap risk factors. It is important to recognize that models are the analyst’s view of a system and not the actual system itself. Don’t ever confuse mathematical model results with reality. A probability guarantees nothing — it is an estimate from a model that provides relative information for decision making. Table 1 identifies some of the attributes that can be used to judge the strengths and weaknesses of qualitative and quantitative approaches. Table 1 — Differences Between Qualitative and Quantitative Methods.   Attribute Qualitative Quantitative 1 Numerical Results No Yes 2 Cost Lower Higher 3 Subjective/Objective Subjective Objective 4 Difficulty Lower Higher 5 Complexity Lower Higher 6 Data Less Detailed More Detailed 7 Technical Expertise Lower Higher 8 Time Required Lower Higher 9 Tools Required Seldom Usually 10 Accuracy Lower Higher   The system safety discipline primarily uses the qualitative risk characterization approach for a majority of safety work, based on the advantages it provides. This approach has been recommended in MIL-STD-882 since the original version in 1969. One reason why system safety typically applies the qualitative method is that for a large system with many hazards, it can become cost prohibitive to quantitatively model, analyze and predict the risk of each and every hazard. In addition, low-risk hazards do not require the refinement provided by quantitative analysis. It may be necessary to conduct a quantitative analysis only on a select few high-consequence hazards. Experience over the years has proven that qualitative methods are very effective, and in most cases provide decision-making capability comparable to quantitative analysis.   "It may be necessary to conduct a quantitative analysis only on a select few high-consequence hazards." Qualitative risk characterization provides a very practical and effective approach when cost and time are concerns, and/or when there is very little supporting data available. The key to developing a qualitative approach is in carefully defining severity and mishap probability categories. Quantitative risk characterization provides a useful approach when greater accuracy is required for decision making. Occasionally, a numerical design requirement must be met, and the only way to provide evidence that it is satisfied is through quantitative analysis. Probabilistic Risk Assessment (PRA) is a quantitative analysis that estimates the probability factor of mishap risk. For high-consequence systems it is often necessary to conduct a PRA to determine all of the causal factors for a given mishap, and their total probability of causing the mishap to occur. Scientific theory teaches that when something can be measured (quantitatively), more is known about it, and therefore numerical results provide more value. This is generally true; however, the strict use of quantitative methods must be tempered by utility and model accuracy. Qualitative judgments can provide useful results while involving less time and expense. In a risk assessment, precise numerical accuracy is not always necessary. Mishap risks are not easily estimated using probability and statistics when the hazard causal factors are not yet well understood (such as in preliminary design). Qualitative measures provide a useful and valid judgment at much less expense than quantitative measures, and they can be obtained much earlier in the system development lifecycle. It makes sense to first evaluate all identified hazards qualitatively, and then conduct a quantitative analysis for high-risk hazards to gain more precise knowledge. In any evaluation of mishap risk assessment, the question of risk-measurement methods and acceptability parameters arises. There is always the danger that system safety analysts and managers will become so enamored of probability and statistics that simpler and more meaningful engineering processes will be ignored. Before embarking in a particular direction, be sure that the limitations and principles of both approaches are well understood, as well as the actual analysis needs. Please send me your comments. I would like to know what your thoughts are and if you agree or disagree with these ideas. Regards, Clif