Dataset Information

An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics.

ABSTRACT:

Background

The systematic review of economic evaluations plays a critical role in making well-informed decisions about competing healthcare interventions. The quality of these systematic reviews varies due to the lack of internationally recognized methodological evaluation standards.

Methods

Nine English and Chinese databases including the Cochrane Library, PubMed, EMbase (Ovid), NHS economic evaluation database (NHSEED) (Ovid), Health Technology Assessment (HTA) database, Chinese National Knowledge Infrastructure (CNKI), WangFang, VIP Chinese Science & Technology Periodicals (VIP) and Chinese Biomedical Literature Database (CBM) were searched. Two reviewers independently screened studies and extracted data. The methodological quality of the literature was measured with modified AMSTAR. Data were narrative synthesized.

Results

165 systematic reviews were included. The overall methodological quality of the literature was moderate according to the AMSTAR scale. In these articles, thirteen quality assessment tools and 32 author self-defined criteria were used. The three most widely used tools were the Drummond checklist (19.4%), the BMJ checklist (15.8%), the Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement (12.7%). Others included the Quality of Health Economic Studies (QHES), the Consensus on Health Economic Criteria (CHEC), the checklist of Center for Reviews and Dissemination (CRD), the Philips checklist, the World Health Organization (WHO) checklist, the checklist of Critical Appraisal Skills Program (CASP), the Pediatric Quality Appraisal Questionnaire (PQAQ), the Joanna Briggs Institute (JBI) checklist, Spanish and Chinese guidelines. The quantitative scales used in these literature were the QHES and PQAQ.

Conclusions

Evidence showed that pharmacoeconomic systematic reviews' methodology remained to be improved, and the quality assessment criteria were gradually unified. Multiple scales can be used in combination to evaluate the quality of economic research in different settings and types.

SUBMITTER: Min C

PROVIDER: S-EPMC7870091 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics.

Min Chen C Xue Mi M Haotian Fei F Jialian Li L Lingli Zhang Z

PloS one 20210208 2

<h4>Background</h4>The systematic review of economic evaluations plays a critical role in making well-informed decisions about competing healthcare interventions. The quality of these systematic reviews varies due to the lack of internationally recognized methodological evaluation standards.<h4>Methods</h4>Nine English and Chinese databases including the Cochrane Library, PubMed, EMbase (Ovid), NHS economic evaluation database (NHSEED) (Ovid), Health Technology Assessment (HTA) database, Chinese ...[more]

PMID: 33556056

Similar Datasets

Project description:BackgroundThe field of eHealth is growing rapidly and chaotically. Health care professionals need guidance on reviewing and assessing health-related smartphone apps to propose appropriate ones to their patients. However, to date, no framework or evaluation tool fulfills this purpose.ObjectiveBefore developing a tool to help health care professionals assess and recommend apps to their patients, we aimed to create an overview of published criteria to describe and evaluate health apps.MethodsWe conducted a systematic review to identify existing criteria for eHealth smartphone app evaluation. Relevant databases and trial registers were queried for articles. Articles were included that (1) described tools, guidelines, dimensions, or criteria to evaluate apps, (2) were available in full text, and (3) were written in English, French, German, Italian, Portuguese, or Spanish. We proposed a conceptual framework for app evaluation based on the dimensions reported in the selected articles. This was revised iteratively in discussion rounds with international stakeholders. The conceptual framework was used to synthesize the reported evaluation criteria. The list of criteria was discussed and refined by the research team.ResultsScreening of 1258 articles yielded 128 (10.17%) that met the inclusion criteria. Of these 128 articles, 30 (23.4%) reported the use of self-developed criteria and described their development processes incompletely. Although 43 evaluation instruments were used only once, 6 were used in multiple studies. Most articles (83/128, 64.8%) did not report following theoretical guidelines; those that did noted 37 theoretical frameworks. On the basis of the selected articles, we proposed a conceptual framework to explore 6 app evaluation dimensions: context, stakeholder involvement, features and requirements, development processes, implementation, and evaluation. After standardizing the definitions, we identified 205 distinct criteria. Through consensus, the research team relabeled 12 of these and added 11 more-mainly related to ethical, legal, and social aspects-resulting in 216 evaluation criteria. No criteria had to be moved between dimensions.ConclusionsThis study provides a comprehensive overview of criteria currently used in clinical practice to describe and evaluate apps. This is necessary as no reviewed criteria sets were inclusive, and none included consistent definitions and terminology. Although the resulting overview is impractical for use in clinical practice in its current form, it confirms the need to craft it into a purpose-built, theory-driven tool. Therefore, in a subsequent step, based on our current criteria set, we plan to construct an app evaluation tool with 2 parts: a short section (including 1-3 questions/dimension) to quickly disqualify clearly unsuitable apps and a longer one to investigate more likely candidates in closer detail. We will use a Delphi consensus-building process and develop a user manual to prepare for this undertaking.Trial registrationPROSPERO International Prospective Register of Systematic Reviews CRD42021227064; https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021227064.

Project description:While single indicators measure a specific aspect of quality (e.g. timely support during labour), users of these indicators, such as patients, providers and policy-makers, are typically interested in some broader construct (e.g. quality of maternity care) whose measurement requires a set of indicators. However, guidance on desirable properties of indicator sets is lacking. Based on the premise that a set of valid indicators does not guarantee a valid set of indicators, the aim of this review is 2-fold: First, we introduce content validity as a desirable property of indicator sets and review the extent to which studies in the peer-reviewed health care quality literature address this criterion. Second, to obtain a complete inventory of criteria, we examine what additional criteria of quality indicator sets were used so far. We searched the databases Web of Science, Medline, Cinahl and PsycInfo from inception to May 2021 and the reference lists of included studies. English- or German-language, peer-reviewed studies concerned with desirable characteristics of quality indicator sets were included. Applying qualitative content analysis, two authors independently coded the articles using a structured coding scheme and discussed conflicting codes until consensus was reached. Of 366 studies screened, 62 were included in the review. Eighty-five per cent (53/62) of studies addressed at least one of the component criteria of content validity (content coverage, proportional representation and contamination) and 15% (9/62) addressed all component criteria. Studies used various content domains to structure the targeted construct (e.g. quality dimensions, elements of the care pathway and policy priorities), providing a framework to assess content validity. The review revealed four additional substantive criteria for indicator sets: cost of measurement (21% [13/62] of the included studies), prioritization of 'essential' indicators (21% [13/62]), avoidance of redundancy (13% [8/62]) and size of the set (15% [9/62]). Additionally, four procedural criteria were identified: stakeholder involvement (69% [43/62]), using a conceptual framework (44% [27/62]), defining the purpose of measurement (26% [16/62]) and transparency of the development process (8% [5/62]). The concept of content validity and its component criteria help assessing whether conclusions based on a set of indicators are valid conclusions about the targeted construct. To develop a valid indicator set, careful definition of the targeted construct including its (sub-)domains is paramount. Developers of quality indicators should specify the purpose of measurement and consider trade-offs with other criteria for indicator sets whose application may reduce content validity (e.g. costs of measurement) in light thereof.

Project description:In the literature, there is widespread consensus that methods in plastic research need improvement. Current limitations in quality assurance and harmonization prevent progress in our understanding of the true effects of microplastic in the environment. Following the recent development of quality assessment methods for studies reporting concentrations in biota and water samples, we propose a method to assess the quality of microplastic effect studies. We reviewed 105 microplastic effect studies with aquatic biota, provided a systematic overview of their characteristics, developed 20 quality criteria in four main criteria categories (particle characterization, experimental design, applicability in risk assessment, and ecological relevance), propose a protocol for future effect studies with particles, and, finally, used all the information to define the weight of evidence with respect to demonstrated effect mechanisms. On average, studies scored 44.6% (range 20-77.5%) of the maximum score. No study scored positively on all criteria, reconfirming the urgent need for better quality assurance. Most urgent recommendations for improvement relate to avoiding and verifying background contamination, and to improving the environmental relevance of exposure conditions. The majority of the studies (86.7%) evaluated on particle characteristics properly, nonetheless it should be underlined that by failing to provide characteristics of the particles, an entire experiment can become irreproducible. Studies addressed environmentally realistic polymer types fairly well; however, there was a mismatch between sizes tested and those targeted when analyzing microplastic in environmental samples. In far too many instances, studies suggest and speculate mechanisms that are poorly supported by the design and reporting of data in the study. This represents a problem for decision-makers and needs to be minimized in future research. In their papers, authors frame 10 effects mechanisms as "suggested", whereas 7 of them are framed as "demonstrated". When accounting for the quality of the studies according to our assessment, three of these mechanisms remained. These are inhibition of food assimilation and/or decreased nutritional value of food, internal physical damage, and external physical damage. We recommend that risk assessment addresses these mechanisms with higher priority.

Project description:BackgroundNational cardiac registries are increasingly used for informing health policy, improving the quality and cost-effectiveness of patient care, clinical research, and monitoring the safety of novel treatments. However, the quality of registries is variable. We aimed to assess the characteristics and quality of national cardiac registries across all subspecialties of cardiac care.MethodsPublications relating to national cardiac registries across six cardiac subspecialty domains were identified by searching MEDLINE and the Google advanced search function with 26 438 citations and 4812 full-text articles reviewed.ResultsA total of 155 registries, representing 49 countries, were included in the study. Of these, 45 related to coronary disease or percutaneous coronary intervention, 28 related to devices, arrhythmia, and electrophysiology, 24 related to heart failure, transplant, and mechanical support, 21 related to structural heart disease, 21 related to congenital heart disease, and 16 related to cardiac surgery. Enrollment was procedure-based in 60% and disease-based in 40%. A total of 73.10 million patients were estimated to have been enrolled in cardiac registries. Quality scoring was performed using a validated registry grading system, with registries performing best in the use of explicit variable definitions and worst in assessment of data reliability. Higher quality scores were associated with government funding, mandated enrollment, linkage to other registries, and outcome risk adjustment. Quality scores and number of registries within a country were positively correlated with each other and with measures of national economic output, health expenditure, and urbanization.ConclusionsThere has been remarkable growth in the uptake of national cardiac registries across the last few decades. However, the quality of processes used to ensure data completeness and accuracy remain variable and few countries have integrated registries covering multiple subspecialty domains. Clinicians, funders, and health policymakers should be encouraged to focus on the range, quality, and integration of these registries. Registration: URL: https://www.crd.york.ac.uk/prospero; Unique identifier: CRD42020204224.

Project description:Background Medical specialty certification exams are high-stakes summative assessments used to determine which doctors have the necessary skills, knowledge, and attitudes to treat patients independently. Such exams are crucial for patient safety, candidates’ career progression and accountability to the public, yet vary significantly among medical specialties and countries. It is therefore of paramount importance that the quality of specialty certification exams is studied in the scientific literature. Methods In this systematic literature review we used the PICOS framework and searched for papers concerning medical specialty certification exams published in English between 2000 and 2020 in seven databases using a diverse set of search term variations. Papers were screened by two researchers independently and scored regarding their methodological quality and relevance to this review. Finally, they were categorized by country, medical specialty and the following seven Ottawa Criteria of good assessment: validity, reliability, equivalence, feasibility, acceptability, catalytic and educational effect. Results After removal of duplicates, 2852 papers were screened for inclusion, of which 66 met all relevant criteria. Over 43 different exams and more than 28 different specialties from 18 jurisdictions were studied. Around 77% of all eligible papers were based in English-speaking countries, with 55% of publications centered on just the UK and USA. General Practice was the most frequently studied specialty among certification exams with the UK General Practice exam having been particularly broadly analyzed. Papers received an average of 4.2/6 points on the quality score. Eligible studies analyzed 2.1/7 Ottawa Criteria on average, with the most frequently studied criteria being reliability, validity, and acceptability. Conclusions The present systematic review shows a growing number of studies analyzing medical specialty certification exams over time, encompassing a wider range of medical specialties, countries, and Ottawa Criteria. Due to their reliance on multiple assessment methods and data-points, aspects of programmatic assessment suggest a promising way forward in the development of medical specialty certification exams which fulfill all seven Ottawa Criteria. Further research is needed to confirm these results, particularly analyses of examinations held outside the Anglosphere as well as studies analyzing entire certification exams or comparing multiple examination methods. Supplementary Information The online version contains supplementary material available at 10.1186/s12909-023-04600-x.

Dataset Information

An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics.

Background

Methods

Results

Conclusions

Publications

An overview of the characteristics and quality assessment criteria in systematic review of pharmacoeconomics.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets