Dataset Information

Use (and abuse) of expert elicitation in support of decision making for public policy.

ABSTRACT: The elicitation of scientific and technical judgments from experts, in the form of subjective probability distributions, can be a valuable addition to other forms of evidence in support of public policy decision making. This paper explores when it is sensible to perform such elicitation and how that can best be done. A number of key issues are discussed, including topics on which there are, and are not, experts who have knowledge that provides a basis for making informed predictive judgments; the inadequacy of only using qualitative uncertainty language; the role of cognitive heuristics and of overconfidence; the choice of experts; the development, refinement, and iterative testing of elicitation protocols that are designed to help experts to consider systematically all relevant knowledge when they make their judgments; the treatment of uncertainty about model functional form; diversity of expert opinion; and when it does or does not make sense to combine judgments from different experts. Although it may be tempting to view expert elicitation as a low-cost, low-effort alternative to conducting serious research and analysis, it is neither. Rather, expert elicitation should build on and use the best available research and analysis and be undertaken only when, given those, the state of knowledge will remain insufficient to support timely informed assessment and decision making.

SUBMITTER: Morgan MG

PROVIDER: S-EPMC4034232 | biostudies-literature | 2014 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Use (and abuse) of expert elicitation in support of decision making for public policy.

Morgan M Granger MG

Proceedings of the National Academy of Sciences of the United States of America 20140512 20

The elicitation of scientific and technical judgments from experts, in the form of subjective probability distributions, can be a valuable addition to other forms of evidence in support of public policy decision making. This paper explores when it is sensible to perform such elicitation and how that can best be done. A number of key issues are discussed, including topics on which there are, and are not, experts who have knowledge that provides a basis for making informed predictive judgments; th ...[more]

PMID: 24821779

Similar Datasets

Project description:BackgroundPublic health professionals are increasingly expected to engage in evidence-informed decision making to inform practice and policy decisions. Evidence-informed decision making involves the use of research evidence along with expertise, existing public health resources, knowledge about community health issues, the local context and community, and the political climate. The National Collaborating Centre for Methods and Tools has identified a seven step process for evidence-informed decision making. Tools have been developed to support public health professionals as they work through each of these steps. This paper provides an overview of tools used in three Canadian public health departments involved in a study to develop capacity for evidence-informed decision making.MethodsAs part of a knowledge translation and exchange intervention, a Knowledge Broker worked with public health professionals to identify and apply tools for use with each of the steps of evidence-informed decision making. The Knowledge Broker maintained a reflective journal and interviews were conducted with a purposive sample of decision makers and public health professionals. This paper presents qualitative analysis of the perceived usefulness and usability of the tools.ResultsTools were used in the health departments to assist in: question identification and clarification; searching for the best available research evidence; assessing the research evidence for quality through critical appraisal; deciphering the 'actionable message(s)' from the research evidence; tailoring messages to the local context to ensure their relevance and suitability; deciding whether and planning how to implement research evidence in the local context; and evaluating the effectiveness of implementation efforts. Decision makers provided descriptions of how the tools were used within the health departments and made suggestions for improvement. Overall, the tools were perceived as valuable for advancing and sustaining evidence-informed decision making.ConclusionTools are available to support the process of evidence-informed decision making among public health professionals. The usability and usefulness of these tools for advancing and sustaining evidence-informed decision making are discussed, including recommendations for the tools' application in other public health settings beyond this study. Knowledge and awareness of these tools may assist other health professionals in their efforts to implement evidence-informed practice.

Project description:BackgroundThe EU 6th Framework Program (FP)-funded Health and Environment Network (HENVINET) aimed to support informed policy making by facilitating the availability of relevant knowledge on different environmental health issues. An approach was developed by which scientific agreement, disagreement, and knowledge gaps could be efficiently identified, and expert advice prepared in a way that is usable for policy makers. There were two aims of the project: 1) to apply the tool to a relevant issue; the potential health impacts of the widely used plasticizers, phthalates, and 2) to evaluate the method and the tool by asking both scientific experts and the target audience, namely policy makers and stakeholders, for their opinions.MethodsThe tool consisted of an expert consultation in several steps on the issue of phthalates in environmental health. A diagram depicting the cause-effect chain, from the production and use of phthalates to potential health impacts, was prepared based on existing reviews. This was used as a basis for an online questionnaire, through which experts in the field were consulted. The results of this first round of consultation laid the foundation for a new questionnaire answered by an expert panel that, subsequently, also discussed approaches and results in a workshop. One major task of the expert panel was to pinpoint priorities from the cause-effect chain according to their impact on the extent of potential health risks and their relevance for reducing uncertainty. The results were condensed into a policy brief that was sent to policy makers and stakeholders for their evaluation.ResultsThe experts agreed about the substantial knowledge gaps within the field of phthalates. The top three priorities for further research and policy action were: 1) intrauterine exposure, 2) reproductive toxicology, and 3) exposure from medical devices. Although not all relevant information from the cause-effect chain is known for phthalates, most experts thought that there are enough indications to justify a precautionary approach and to restrict their general use. Although some of the experts expressed some scepticism about such a tool, most felt that important issues were highlighted.ConclusionsThe approach used was an efficient way at summarising priority knowledge gaps as a starting point for health risk assessment of compounds, based on their relevance for the risk assessment outcome. We conclude that this approach is useful for supporting policy makers with state-of-the-art scientific knowledge weighed by experts. The method can assist future evidence-based policy making.

Project description:BackgroundAlthough important syntheses and theoretical works exist in relation to understanding the organisational factors that facilitate research use, these contributions differ in their scope and object of study as well as their theoretical underpinnings. Therefore, from an exploratory angle, it may be useful to map out the current literature on organisational factors of research use in public health policy-making when revisiting existing theories and frameworks to gain further theoretical insights.MethodsHerein, a scoping review technique and thematic content analysis were used to bring together findings from both synthesised and empirical studies of different types to map out the organisational factors that facilitate research use in public health policy-making.ResultsA total of 14 reviews and 40 empirical studies were included in the analysis. These were thematically coded and the intra-organisational factors reported as enabling research use were examined. Five main categories of organisational factors that advance research use in policy organisations - (1) individual factors, (2) the management of research integration, (3) organisational systems and infrastructures of research use, (4) institutional structures and rules for policy-making, and (5) organisational characteristics - were derived as well as 18 subcategories and a total of 64 specific factors, where 27 factors were well supported by research.ConclusionsUsing a scoping review methodology, the intra-organisational factors influencing research use in policy-making (including individual factors) were systematically mapped and the theories applied in this area of research were assessed. The review findings confirm the importance of an intra-organisational perspective when exploring research use, showing that many organisational factors are critical facilitators of research use but also that many factors and mechanisms are understudied. The synthesis shows a lack of studies on politicians and the need for more theoretically founded research. Despite increased efforts to update the existing evidential and theoretical basis of research use, we still need frameworks that combine different approaches and theories to help us grasp the complex organisational mechanisms that facilitate research use in policy settings.

Dataset Information

Use (and abuse) of expert elicitation in support of decision making for public policy.

Publications

Use (and abuse) of expert elicitation in support of decision making for public policy.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets