In this article, we delve into the critical aspects of controlling subjectivity within the realm of risk-based decision making (RBDM). We explore the utilization of expert knowledge and probabilities as key tools to enhance the objectivity of such processes. This discussion is particularly pertinent in the context of quality risk management (QRM) and the challenges it faces concerning subjectivity, as highlighted in the International Council for Harmonisation’s (ICH) guidelines.
ICH Q9 and the Challenge of Subjectivity ICH Q9, which addresses quality risk management, has long grappled with the issue of subjectivity. The success of any risk management program hinges on the ability to recognize and mitigate the subjectivity that can seep into expert-based judgments during the development and application of risk tools. To address this, ICH Q9 (R1) has introduced a section dedicated to “Managing and Minimizing Subjectivity (5.3),” along with an expanded discussion on the concept of formality in QRM. In this article, we explore various approaches to mitigating subjectivity, with a particular focus on expert elicitation.
Controlling Subjectivity in RBDM Effective strategies for controlling subjectivity in RBDM include the following:
- Raising Awareness: Providing training on probability, expert judgments, heuristics, biases, and the potential impact of subjectivity is essential. This helps stakeholders recognize and address subjectivity in their decision-making processes.
- Formal Calibration: Implementing formal calibration methods, such as Cooke’s classical method, helps in eliciting subjective probabilities from individual experts. These calibrated probabilities can then be used to make informed decisions.
- Group Elicitation: Conducting subjective probability elicitation in group settings, facilitated by techniques like the Delphi method, value-focused thinking, or multi-criteria decision making (MCDM), can further reduce subjectivity by promoting discussion and consensus among experts.
Elicitation of Expert Knowledge and Probabilities Expert elicitation is a crucial component of managing subjectivity in RBDM. It involves extracting valuable insights and quantifiable probabilities from domain experts. Here, we distinguish between two types of expert elicitation:
- Elicitation of Expert Knowledge: This involves qualitatively gathering expert insights to define decision-making objectives, model structure, key risk factors, and appropriate QRM tools. Multi-criteria decision analysis (MCDA) techniques can be employed to structure complex decision hierarchies.
- Structured Expert Judgments (SEJ): SEJ is a more formal process, primarily used in high-impact RBDM. It entails quantitatively eliciting expert judgments of probabilities. Methods like Cooke’s classical model guide experts in providing calibrated estimates of probabilities, enhancing the reliability of risk assessments.
When Is Expert Elicitation Needed? Expert elicitation is deemed necessary in QRM, particularly for complex risk assessments. Even seemingly straightforward risk tools, such as risk matrices and failure mode and effects analysis (FMEA), benefit from expert input. Structured thinking facilitated by expert elicitation ensures the development of sustainable risk tools.
Principles for Expert Elicitation Adhering to principles for expert elicitation adds value to the process. These principles encompass transparency, empirical control, neutrality, and fairness. They ensure that the elicitation process is open to peer review, subject to quality controls, unbiased, and respectful of experts’ contributions.
Limitations and Resources While expert elicitation holds promise for enhancing RBDM, limitations include resource constraints, expertise availability, resistance from experts, and potential misuse. Resources for expert elicitation can be found in a variety of fields, including multi-criteria decision analysis (MCDA) and structured expert judgment (SEJ). The choice of framework depends on the composition of the decision-making team and the specific needs of the risk assessment.
Controlling subjectivity in risk-based decision making is vital for effective quality risk management. Expert knowledge and probabilities play a significant role in mitigating subjectivity. Whether through qualitative expert elicitation or structured expert judgments, these approaches contribute to more objective and reliable risk assessments. Embracing the principles of transparency, empirical control, neutrality, and fairness further enhances the value of expert elicitation in RBDM.
Your go-to for supply chain report news updates: The Supply Chain Report. For international trade tools, see ADAMftd.com.
#RiskBasedDecisionMaking #QualityRiskManagement #SubjectivityControl #ExpertElicitation #ICHQ9 #RiskAssessment #DecisionMakingTools #ProbabilityElicitation #FormalCalibration #ExpertKnowledge #StructuredJudgments #DelphiMethod #MultiCriteriaDecisionMaking #ManagingSubjectivity #QRMTools
