Use of an institutional template for annual program evaluation and improvement: benefits for program participation and performance.
Ontology highlight
ABSTRACT:
SUBMITTER: Andolsek KM
PROVIDER: S-EPMC2930309 | biostudies-other | 2010 Jun
REPOSITORIES: biostudies-other
Similar Datasets
Project description:BackgroundThe Accreditation Council for Graduate Medical Education (ACGME) requires sponsoring institutions to demonstrate effective oversight through an annual institutional review (AIR). The ACGME only requires 3 elements to be reported, and it is up to the discretion of the designated institutional official (DIO) whether other supporting information should be included. This leads to uncertainty and inconsistency for DIOs as they decide what to report.ObjectiveWe surveyed DIOs in an effort to provide national data on key performance indicators and other relevant components of the AIR process.MethodsIn July 2019, we conducted a national survey of 847 DIOs. The survey had 16 questions that explored basic institutional demographics, timelines, and processes for the AIR and key performance indicators. Written answers were grouped by similar responses, and we performed descriptive statistics on all variables to assess distributions of responses. We also explored associations between variables using cross-tabulation and chi-square statistics.ResultsA total of 267 DIOs responded to the survey (32% response rate). There were 7 institutional performance measures that achieved over 50% consensus. These reviews required the majority of DIOs (62%, 167 of 267) 5 to 20 hours to complete. Less than one-third of sponsoring institutions reported diversity data. The majority of DIOs (68%, 182 of 267) felt the AIR process added substantial value.ConclusionsThis survey reports key performance measures and processes included by DIOs in the AIR. Our results show a wide range of institutional responses though consensus was achieved on 7 key performance measures.
| S-EPMC7901609 | biostudies-literature
Project description:The Accreditation Council for Graduate Medical Education (ACGME) requires programs to engage annually in program evaluation and improvement.We assessed the value of creating educational competency committees (ECCs) that use successful elements of 2 established processes-institutional special reviews and institutional oversight of annual program evaluations.The ECCs used a template to review programs' annual program evaluations. Results were aggregated into an institutional dashboard. We calculated the costs, sensitivity, specificity, and predictive value by comparing programs required to have a special review with those that had ACGME citations, requests for a progress report, or a data-prompted site visit. We assessed the value for professional development through a participant survey.Thirty-two ECCs involving more than 100 individuals reviewed 237 annual program evaluations over a 3-year period. The ECCs required less time than internal reviews. The ECCs rated 2 to 8 programs (2.4%-9.8%) as "noncompliant." One to 13 programs (1.2%-14.6%) had opportunities for improvement identified. Institutional improvements were recognized using the dashboard. Zero to 13 programs (0%-16%) were required to have special reviews. The sensitivity of the decision to have a special review was 83% to 100%; specificity was 89% to 93%; and negative predictive value was 99% to 100%. The total cost was $280 per program. Of the ECC members, 86% to 95% reported their participation enhanced their professional development, and 60% to 95% believed the ECC benefited their program.Educational competency committees facilitated the identification of institution-wide needs, highlighted innovation and best practices, and enhanced professional development. The cost, sensitivity, specificity, and predictive value indicated good value.
| S-EPMC4936857 | biostudies-other
Project description:ObjectiveTo evaluate whether reporting of hospital performance was associated with a change in quality indicators in Italian hospitals.Data sources/study settingNationwide Hospital Information System for 2006-2009.Study designWe performed a pre-post evaluation in Lazio (before and after disclosure of the Regional Outcome Evaluation Program P.Re.Val.E.) and a comparative evaluation versus Italian regions without comparable programs. We analyzed risk-adjusted proportions of percutaneous coronary intervention (PCI), hip fractures operated on within 48 hours, and cesarean deliveries.Data collection/extraction methodsUsing standardized ICD-9-CM coding algorithms, we selected 381,053 acute myocardial infarction patients, 250,712 hip fractures, and 1,736,970 women who had given birth.Principal findingsIn Lazio PCI within 48 hours changed from 22.49 to 29.43 percent following reporting of the P.Re.Val.E results (relative increase, 31 percent; p < .001). In the other regions this proportion increased from 22.48 to 27.09 percent during the same time period (relative increase, 21 percent; p < .001). Hip fractures operated on within 48 hours increased from 11.73 to 15.78 percent (relative increase, 34 percent; p < .001) in Lazio, and not in other regions (29.36 to 28.57 percent). Cesarean deliveries did not decrease in Lazio (34.57-35.30 percent), and only slightly decreased in the other regions (30.49-28.11 percent).ConclusionsReporting of performance data may have a positive but limited impact on quality improvement. The evaluation of quality indicators remains paramount for public accountability.
| S-EPMC3513610 | biostudies-literature
Project description:BACKGROUND: The Accreditation Council for Graduate Medical Education (ACGME) requires residency programs to meet and demonstrate outcomes across 6 competencies. Measuring residents' competency in practice-based learning and improvement (PBLI) is particularly challenging. PURPOSE: We developed an educational tool to meet ACGME requirements for PBLI. The PBLI template helped programs document quality improvement (QI) projects and supported increased scholarly activity surrounding PBLI learning. METHODS: We reviewed program requirements for 43 residency and fellowship programs and identified specific PBLI requirements for QI activities. We also examined ACGME Program Information Form responses on PBLI core competency questions surrounding QI projects for program sites visited in 2008-2009. Data were integrated by a multidisciplinary committee to develop a peer-protected PBLI template guiding programs through process, documentation, and evaluation of QI projects. All steps were reviewed and approved through our GME Committee structure. RESULTS: An electronic template, companion checklist, and evaluation form were developed using identified project characteristics to guide programs through the PBLI process and facilitate documentation and evaluation of the process. During a 24 month period, 27 programs have completed PBLI projects, and 15 have reviewed the template with their education committees, but have not initiated projects using the template. DISCUSSION: The development of the tool generated program leaders' support because the tool enhanced the ability to meet program-specific objectives. The peer-protected status of this document for confidentiality and from discovery has been beneficial for program usage. The document aggregates data on PBLI and QI initiatives, offers opportunities to increase scholarship in QI, and meets the ACGME goal of linking measures to outcomes important to meeting accreditation requirements at the program and institutional level.
| S-EPMC3399615 | biostudies-other
Project description:BackgroundThe Donation Improvement Program (DIP) is intended to increase organ donation at hospitals. The program includes education for healthcare professionals of participating hospitals about each step of donation as well as evaluation. The DIP consists of medical record review (MRR) and a Hospital Attitude Survey (HAS). The purpose of this study was to evaluate the DIP results for the last 7 years.MethodsFor MRR, we analyzed 58,385 cases of mortality from 77 hospitals between 2012 and 2018. The HAS data for the degree of education experience, competence, and knowledge related to brain death (BD) and donation were analyzed from 23 DIP-participating hospitals in 2012 and 51 DIP-participating hospitals in 2015 and 2018 each.ResultsThe recognition rate of potential BD was 24.9%, 61.3%, and 73.2%, and donation rate was 7.5%, 11.7%, and 15.8% at 6 months before, 1?2 years after, and 4?5 years after the agreement, respectively. Hospital staff with the necessary competence or knowledge to explain BD constituted 44.0% in 2012, while this increased to 62.8% in 2018.ConclusionsThe DIP could increase the recognition of BD and the organ donation rate, and positively affect the attitudes of healthcare professionals toward organ donation. Spreading the DIP to all hospitals is urgent to increase organ donation.
| S-EPMC9235343 | biostudies-literature
Project description:Aims:To explore the potential influence of the Stanford Chronic Disease Self-Management Program (CDSMP) on social support in Parkinson disease (PD). Methods:This was a quasi-experimental mixed methods design. Volunteers with PD (n=27) and care partners (n=6) completed the CDSMP, questionnaires of social support and self-management outcomes, and an interview about social support in relation to CDSMP participation. PD participants (n=19) who did not participate in the CDSMP completed the questionnaires for quantitative comparison purposes. Results:Regarding the quantitative data, there were no significant effects of CDSMP participation on social support questionnaire scores; however, there were some positive correlations between changes in social support and changes in self-management outcomes from pre- to post-CDSMP participation. Three qualitative themes emerged from the interviews: lack of perceived change in amount and quality of social support, positive impact on existing social networks, and benefit from participating in a supportive PD community. Conclusions:Although participants did not acknowledge major changes in social support, there were some social support-related benefits of CDSMP participation for PD participants and care partners. These findings provide a starting point for more in-depth studies of social support and self-management in this population.
| S-EPMC5711466 | biostudies-literature
Project description:This study aimed to evaluate the effect of the formative program evaluation on the continuous improvement of a clinical training program for Lao health professionals. The training program was conducted 4 times consecutively for total 48 health professionals, and the formative program evaluation was carried out during the whole process. To evaluate the satisfaction and the transfer of the trainees, the questionnaire survey, the focus group interview, and the trainees' medical records were used. After the end of each batch of the program, the evaluation data were analyzed, and its results were shared with the training management committee and the trainers, who, based on the results, reached a consensus on how to improve the program. The evaluation results and the comparison of them among the four batches of the program showed that there was a continuous increase of the satisfaction and the transfer of the trainees, especially in the early period of the program. The formative program evaluation which was conducted during the whole process of the clinical training program had a positive effect on the improvement of the program, especially in the early phase, by increasing the satisfaction and transfer of the trainees.
| S-EPMC4689817 | biostudies-other
Project description:BackgroundAsthma is the leading chronic condition in adolescents. Nonetheless, research efforts to address adolescent asthma morbidity by applying a developmentally appropriate self-management intervention have been limited. Recently a peer-led asthma self-management program has been developed and implemented for adolescents. The purpose of the study was to assess the acceptability of a peer-led asthma program for adolescents based on participant feedback and to examine the program's impact on asthma outcomes in peer leaders.MethodsAdolescents with persistent asthma (n = 91, 13-17 years) participated in a camp-based asthma program led by peer leaders (the intervention group) or healthcare professionals (the control group), and completed a set of program evaluation forms. Peer leaders (n = 14, 16-20 years) completed study questionnaires measuring asthma knowledge, self-efficacy, asthma control, and quality of life at baseline, immediately after camp, and 3, 6, and 9 months post-camp.ResultsThe peer-led group reported more positive experience with the program than the adult-led group (P = .01, Cohen's d = 0.53), particularly in its usefulness in dealing with asthma. Peer leaders were rated by participants highly on knowledge (98%), attitudes (93-96%), personal skills (91%), and perceived similarities (80-86%). Significant improvements were found in peer leaders' knowledge (P < .001, η(2) = 0.87), self-efficacy (P < .001, η(2) = 0.74), asthma control (P = .01, η(2) = 0.66), and quality of life in emotional functioning (P = .002, η(2) = 0.82) and activity (P = .003, η(2) = 0.62).ConclusionsA peer-led asthma self-management program was successfully implemented and well received by adolescent learners. Asthma outcomes in peer leaders appear to have improved as a result of the program, although caution should be exercised in determining clinical meaningfulness, given the modest changes in mean scores. ClinicalTrials.gov registration NCT01161225.
| S-EPMC3521097 | biostudies-literature
Project description:BACKGROUND: With the use of medicines being a broad and extensive part of health management, mechanisms to ensure quality use of medicines are essential. Drug usage evaluation (DUE) is an evidence-based quality improvement methodology, designed to improve the quality, safety and cost-effectiveness of drug use. The purpose of this paper is to describe a national DUE methodology used to improve health care delivery across the continuum through multi-faceted intervention involving audit and feedback, academic detailing and system change, and a qualitative assessment of the methodology, as illustrated by the Acute Postoperative Pain Management (APOP) project. METHODS: An established methodology, consisting of a baseline audit of inpatient medical records, structured patient interviews and general practitioner surveys, followed by an educational intervention and follow-up audit, is used. Australian hospitals, including private, public, metropolitan and regional, are invited to participate on a voluntary basis. De-identified data collected by hospitals are collated and evaluated nationally to provide descriptive comparative analyses. Hospitals benchmark their practices against state and national results to facilitate change. The educational intervention consists of academic detailing, group education, audit and feedback, point-of-prescribing prompts and system changes. A repeat data collection is undertaken to assess changes in practice.An online qualitative survey was undertaken to evaluate the APOP program. Qualitative assessment of hospitals' perceptions of the effectiveness of the overall DUE methodology and changes in procedure/prescribing/policy/clinical practice which resulted from participation were elicited. RESULTS: 62 hospitals participated in the APOP project. Among 23 respondents to the evaluation survey, 18 (78%) reported improvements in the documentation of pain scores at their hospital. 15 (65%) strongly agreed or agreed that participation in APOP directly resulted in increased prescribing of multimodal analgesia for pain relief in postoperative patients. CONCLUSIONS: This national DUE program has facilitated the engagement and participation of a number of acute health care facilities to address issues relating to quality use of medicine. This approach has been perceived to be effective in helping them achieve improvements in patient care.
| S-EPMC3182891 | biostudies-other
Project description:Many template-based modeling (TBM) methods have been developed over the recent years that allow for protein structure prediction and for the study of structure-function relationships for proteins. One major problem all TBM algorithms face, however, is their unsatisfactory performance when proteins under consideration are low-homology. To improve the performance of TBM methods for such targets, a novel model evaluation method was developed here, and named MEFTop. Our novel method focuses on evaluating the topology by using two novel groups of features. These novel features included secondary structure element (SSE) contact information and 3-dimensional topology information. By combining MEFTop algorithm with FR-t5, a threading program developed by our group, we found that this modified TBM program, which was named FR-t5-M, exhibited significant improvements in predictive abilities for low-homology protein targets. We further showed that the MEFTop could be a generalized method to improve threading programs for low-homology protein targets. The softwares (FR-t5-M and MEFTop) are available to non-commercial users at our website: http://jianglab.ibp.ac.cn/lims/FRt5M/FRt5M.html.
| S-EPMC3935967 | biostudies-literature