Dataset Information

Quality of cost evaluations of physician continuous professional development: Systematic review of reporting and methods.

ABSTRACT:

Introduction

We sought to evaluate the reporting and methodological quality of cost evaluations of physician continuing professional development (CPD).

Methods

We conducted a systematic review, searching MEDLINE, Embase, PsycInfo, and the Cochrane Database for studies comparing the cost of physician CPD (last update 23 April 2020). Two reviewers, working independently, screened all articles for inclusion. Two reviewers extracted information on reporting quality using the Consolidated Health Economic Evaluation Reporting Standards (CHEERS), and on methodological quality using the Medical Education Research Study Quality Instrument (MERSQI) and a published reference case.

Results

Of 3338 potentially eligible studies, 62 were included. Operational definitions of methodological and reporting quality elements were iteratively revised. Articles reported mean (SD) 43% (20%) of CHEERS elements for the Title/Abstract, 56% (34%) for Introduction, 66% (19%) for Methods, 61% (17%) for Results, and 66% (30%) for Discussion, with overall reporting index 292 (83) (maximum 500). Valuation methods were reported infrequently (resource selection 10 of 62 [16%], resource quantitation 10 [16%], pricing 26 [42%]), as were descriptions/discussion of the physicians trained (42 [68%]), training setting (42 [68%]), training intervention (40 [65%]), sensitivity analyses of uncertainty (9 [15%]), and generalizability (30 [48%]). MERSQI scores ranged from 6.0 to 16.0 (mean 11.2 [2.4]). Changes over time in reporting index (initial 241 [105], final 321 [52]) and MERSQI scores (initial 9.8 [2.7], final 11.9 [1.9]) were not statistically significant (p ≥ 0.08).

Discussion

Methods and reporting of HPE cost evaluations fall short of current standards. Gaps exist in the valuation, analysis, and contextualization of cost outcomes.

SUBMITTER: Cook DA

PROVIDER: S-EPMC9240125 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceThe economic impact of continuous professional development (CPD) education is incompletely understood.ObjectiveTo systematically identify and synthesize published research examining the costs associated with physician CPD for drug prescribing.Evidence reviewMEDLINE, Embase, PsycInfo, and the Cochrane Database were searched from inception to April 23, 2020, for comparative studies that evaluated the cost of CPD focused on drug prescribing. Two reviewers independently screened all articles for inclusion and reviewed all included articles to extract data on participants, educational interventions, study designs, and outcomes (costs and effectiveness). Results were synthesized for educational costs, health care costs, and cost-effectiveness.FindingsOf 3338 articles screened, 38 were included in this analysis. These studies included at least 15 659 health care professionals and 1 963 197 patients. Twelve studies reported on educational costs, ranging from $281 to $183 554 (median, $15 664). When economic outcomes were evaluated, 31 of 33 studies (94%) comparing CPD with no intervention found that CPD was associated with reduced health care costs (drug costs), ranging from $4731 to $6 912 000 (median, $79 373). Four studies found reduced drug costs for 1-on-1 outreach compared with other CPD approaches. Regarding cost-effectiveness, among 5 studies that compared CPD with no intervention, the incremental cost-effectiveness ratio for a 10% improvement in prescribing ranged from $15 390 to $437 027 to train all program participants. Four comparisons of alternative CPD approaches found that 1-on-1 educational outreach was more effective but more expensive than group education or mailed materials (incremental cost-effectiveness ratio, $18-$4105 per physician trained).Conclusions and relevanceIn this systematic review, CPD for drug prescribing was associated with reduced health care (drug) costs. The educational costs and cost-effectiveness of CPD varied widely. Several CPD instructional approaches (including educational outreach) were more effective but more costly than comparators.

Project description:Background:Many authors choose to work with professional medical writers when reporting the results of clinical trials. We conducted a systematic review to examine the relationship between professional medical writing support (PMWS) and the quality, ethics and timeliness of publications reporting clinical trials. Methods:Using terms related to 'medical writer' and 'observational study', we searched MEDLINE and Embase (no date limits), as well as abstracts and posters from meetings of the International Society for Medical Publication Professionals (ISMPP; 2014-2018). We also hand-searched the journals Medical Writing and The Write Stuff (2014-2018) and the bibliographies of studies identified in the electronic searches. We screened the results to identify studies that compared the quality, ethics and timeliness of clinical trial publications written with and without declared PMWS. Results:Our searches identified 97 potentially relevant studies, of which 89 were excluded during screening and full paper review. The remaining eight studies compared 849 publications with PMWS with 2073 articles developed without such support. In these eight studies, PMWS was shown to be associated with increased adherence to Consolidated Standards of Reporting Trials (CONSORT) guidelines (in 3/3 studies in which this was assessed), publication in journals with an impact factor (one study), a higher quality of written English (one study), and a lower likelihood of reporting non-pre-specified outcomes (one study). PMWS was not associated with increased adherence to CONSORT for Abstracts guidelines (one study) or with the impact of published articles (mean number of citations per year, mean number of article views per year and Altmetric score; one study). In studies that assessed timeliness of publication, PMWS was associated with a reduced time from last patient visit in clinical trials to primary publication (one study), whereas time from submission to acceptance showed inconsistent results (two studies). Conclusions:This systematic review of eight observational studies suggests that PMWS is positively associated with measures of overall quality of reporting of clinical trials and may improve the timeliness of publication.

Project description:Background and objectivesThe aim was to systematically review whether the reporting and analysis of trial-based cost-effectiveness evaluations in the field of obstetrics and gynaecology comply with guidelines and recommendations, and whether this has improved over time.Data sources and selection criteriaA literature search was performed in MEDLINE, the NHS Economic Evaluation Database (NHS EED) and the Health Technology Assessment (HTA) database to identify trial-based cost-effectiveness evaluations in obstetrics and gynaecology published between January 1, 2000 and May 16, 2017. Studies performed in middle- and low-income countries and studies related to prevention, midwifery, and reproduction were excluded.Data collection and analysisReporting quality was assessed using the Consolidated Health Economic Evaluation Reporting Standard (CHEERS) statement (a modified version with 21 items, as we focused on trial-based cost-effectiveness evaluations) and the statistical quality was assessed using a literature-based list of criteria (8 items). Exploratory regression analyses were performed to assess the association between reporting and statistical quality scores and publication year.ResultsThe electronic search resulted in 5482 potentially eligible studies. Forty-five studies fulfilled the inclusion criteria, 22 in obstetrics and 23 in gynaecology. Twenty-seven (60%) studies did not adhere to 50% (n = 10) or more of the reporting quality items and 32 studies (71%) did not meet 50% (n = 4) or more of the statistical quality items. As for the statistical quality, no study used the appropriate method to assess cost differences, no advanced methods were used to deal with missing data, and clustering of data was ignored in all studies. No significant improvements over time were found in reporting or statistical quality in gynaecology, whereas in obstetrics a significant improvement in reporting and statistical quality was found over time.LimitationsThe focus of this review was on trial-based cost-effectiveness evaluations in obstetrics and gynaecology, so further research is needed to explore whether results from this review are generalizable to other medical disciplines.Conclusions and implications of key findingsThe reporting and analysis of trial-based cost-effectiveness evaluations in gynaecology and obstetrics is generally poor. Since this can result in biased results, incorrect conclusions, and inappropriate healthcare decisions, there is an urgent need for improvement in the methods of cost-effectiveness evaluations in this field.

Project description:ImportanceUS hospitals report data on many health care quality metrics to government and independent health care rating organizations, but the annual cost to acute care hospitals of measuring and reporting quality metric data, independent of resources spent on quality interventions, is not well known.ObjectiveTo evaluate externally reported inpatient quality metrics for adult patients and estimate the cost of data collection and reporting, independent of quality-improvement efforts.Design, setting, and participantsRetrospective time-driven activity-based costing study at the Johns Hopkins Hospital (Baltimore, Maryland) with hospital personnel involved in quality metric reporting processes interviewed between January 1, 2019, and June 30, 2019, about quality reporting activities in the 2018 calendar year.Main outcomes and measuresOutcomes included the number of metrics, annual person-hours per metric type, and annual personnel cost per metric type.ResultsA total of 162 unique metrics were identified, of which 96 (59.3%) were claims-based, 107 (66.0%) were outcome metrics, and 101 (62.3%) were related to patient safety. Preparing and reporting data for these metrics required an estimated 108 478 person-hours, with an estimated personnel cost of $5 038 218.28 (2022 USD) plus an additional $602 730.66 in vendor fees. Claims-based (96 metrics; $37 553.58 per metric per year) and chart-abstracted (26 metrics; $33 871.30 per metric per year) metrics used the most resources per metric, while electronic metrics consumed far less (4 metrics; $1901.58 per metric per year).Conclusions and relevanceSignificant resources are expended exclusively for quality reporting, and some methods of quality assessment are far more expensive than others. Claims-based metrics were unexpectedly found to be the most resource intensive of all metric types. Policy makers should consider reducing the number of metrics and shifting to electronic metrics, when possible, to optimize resources spent in the overall pursuit of higher quality.

Project description:Background: Schizophrenia is a severe and complex disease with substantial economic and social burdens. Despite multiple treatment choices, adverse events, and impaired social functions are still challenges in clinical therapy. Pharmacoeconomic evaluations could provide evidence to help decision makers improve the utilization of scarce resources. However, there remains some challenges especially in modeling due to uncertainties in progression of schizophrenia. There are limited summaries about the overall methodologies of schizophrenia economic evaluations. Objective: The aim of this study is to review the existing economic evaluations of antipsychotics for the treatment of schizophrenia and summarize the evidence and methods applied. Methods: An electronic literature search was performed in PubMed, Web of Science, EBSCO host, The Cochrane Library and ScienceDirect from January 2014 to December 2020. Search terms included "schizophrenia," "schizophrenic," "pharmacoeconomic," "economic evaluation," "cost-effectiveness," and "cost-utility." The Literature was screened and extracted by two researchers independently and assessed with the Quality of Health Economic Studies (QHES) List and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) Statement. Results: A total of 25 studies were included in the review. The regions included Europe, North America, Asia and Africa. Most of the studies chose second-generation antipsychotics as comparators and integrated treatment sequences. Time horizons varied from 1 year to lifetime. The healthcare sector was the most common perspective, accordingly, most of the evaluations considered only direct medical costs. The Markov model and decision tree model were the most common choices. Adverse events, compliance and persistence were considered important parameters. Quality-adjusted life-years were the major outcomes applied to the economic evaluations. All utilities for health states and adverse events were collected from published literature. All of the studies applied uncertainty analysis to explore the robustness of the results. The quality of the studies was generally satisfactory. However, improvements were needed in the choice of time horizons, the measurements of outcomes and the descriptions of assumptions. Conclusions: This study highlights the methodology of economic evaluation of schizophrenia. Recommendations for modeling method and future study are provided.

Project description:BackgroundEfforts to improve the quality, safety, and efficiency of health care provision have often focused on changing approaches to the way services are organized and delivered. Continuous quality improvement (CQI), an approach used extensively in industrial and manufacturing sectors, has been used in the health sector. Despite the attention given to CQI, uncertainties remain as to its effectiveness given the complex and diverse nature of health systems. This review assesses the effectiveness of CQI across different health care settings, investigating the importance of different components of the approach.MethodsWe searched 11 electronic databases: MEDLINE, CINAHL, EMBASE, AMED, Academic Search Complete, HMIC, Web of Science, PsycINFO, Cochrane Central Register of Controlled Trials, LISTA, and NHS EED to February 2019. Also, we searched reference lists of included studies and systematic reviews, as well as checking published protocols for linked papers. We selected randomized controlled trials (RCTs) within health care settings involving teams of health professionals, evaluating the effectiveness of CQI. Comparators included current usual practice or different strategies to manage organizational change. Outcomes were health care professional performance or patient outcomes. Studies were published in English.ResultsTwenty-eight RCTs assessed the effectiveness of different approaches to CQI with a non-CQI comparator in various settings, with interventions differing in terms of the approaches used, their duration, meetings held, people involved, and training provided. All RCTs were considered at risk of bias, undermining their results. Findings suggested that the benefits of CQI compared to a non-CQI comparator on clinical process, patient, and other outcomes were limited, with less than half of RCTs showing any effect. Where benefits were evident, it was usually on clinical process measures, with the model used (i.e., Plan-Do-Study-Act, Model of Improvement), the meeting type (i.e., involving leaders discussing implementation) and their frequency (i.e., weekly) having an effect. None considered socio-economic health inequalities.ConclusionsCurrent evidence suggests the benefits of CQI in improving health care are uncertain, reflecting both the poor quality of evaluations and the complexities of health services themselves. Further mixed-methods evaluations are needed to understand how the health service can use this proven approach.Trial registrationProtocol registered on PROSPERO (CRD42018088309).

Project description:BackgroundPharmaceutical costs are the fastest-growing health-care expense in most developed countries. Higher drug costs have been shown to negatively impact patient outcomes. Studies suggest that doctors have a poor understanding of pharmaceutical costs, but the data are variable and there is no consistent pattern in awareness. We designed this systematic review to investigate doctors' knowledge of the relative and absolute costs of medications and to determine the factors that influence awareness.Methods and findingsOur search strategy included The Cochrane Library, EconoLit, EMBASE, and MEDLINE as well as reference lists and contact with authors who had published two or more articles on the topic or who had published within 10 y of the commencement of our review. Studies were included if: either doctors, trainees (interns or residents), or medical students were surveyed; there were more than ten survey respondents; cost of pharmaceuticals was estimated; results were expressed quantitatively; there was a clear description of how authors defined "accurate estimates"; and there was a description of how the true cost was determined. Two authors reviewed each article for eligibility and extracted data independently. Cost accuracy outcomes were summarized, but data were not combined in meta-analysis because of extensive heterogeneity. Qualitative data related to physicians and drug costs were also extracted. The final analysis included 24 articles. Cost accuracy was low; 31% of estimates were within 20% or 25% of the true cost, and fewer than 50% were accurate by any definition of cost accuracy. Methodological weaknesses were common, and studies of low methodological quality showed better cost awareness. The most important factor influencing the pattern and accuracy of estimation was the true cost of therapy. High-cost drugs were estimated more accurately than inexpensive ones (74% versus 31%, Chi-square p < 0.001). Doctors consistently overestimated the cost of inexpensive products and underestimated the cost of expensive ones (binomial test, 89/101, p < 0.001). When asked, doctors indicated that they want cost information and feel it would improve their prescribing but that it is not accessible.ConclusionsDoctors' ignorance of costs, combined with their tendency to underestimate the price of expensive drugs and overestimate the price of inexpensive ones, demonstrate a lack of appreciation of the large difference in cost between inexpensive and expensive drugs. This discrepancy in turn could have profound implications for overall drug expenditures. Much more focus is required in the education of physicians about costs and the access to cost information. Future research should focus on the accessibility and reliability of medical cost information and whether the provision of this information is used by doctors and makes a difference to physician prescribing. Additionally, future work should strive for higher methodological standards to avoid the biases we found in the current literature, including attention to the method of assessing accuracy that allows larger absolute estimation ranges for expensive drugs.

Project description:There is a growing concern both inside and outside the scientific community over the lack of reproducibility of experiments. The depth and detail of reported methods are critical to the reproducibility of findings, but also for making it possible to compare and integrate data from different studies. In this study, we evaluated in detail the methods reporting in a comprehensive set of trypanosomiasis experiments that should enable valid reproduction, integration and comparison of research findings. We evaluated a subset of other parasitic (Leishmania, Toxoplasma, Plasmodium, Trichuris and Schistosoma) and non-parasitic (Mycobacterium) experimental infections in order to compare the quality of method reporting more generally. A systematic review using PubMed (2000-2012) of all publications describing gene expression in cells and animals infected with Trypanosoma spp was undertaken based on PRISMA guidelines; 23 papers were identified and included. We defined a checklist of essential parameters that should be reported and have scored the number of those parameters that are reported for each publication. Bibliometric parameters (impact factor, citations and h-index) were used to look for association between Journal and Author status and the quality of method reporting. Trichuriasis experiments achieved the highest scores and included the only paper to score 100% in all criteria. The mean of scores achieved by Trypanosoma articles through the checklist was 65.5% (range 32-90%). Bibliometric parameters were not correlated with the quality of method reporting (Spearman's rank correlation coefficient <-0.5; p>0.05). Our results indicate that the quality of methods reporting in experimental parasitology is a cause for concern and it has not improved over time, despite there being evidence that most of the assessed parameters do influence the results. We propose that our set of parameters be used as guidelines to improve the quality of the reporting of experimental infection models as a pre-requisite for integrating and comparing sets of data.