Assessment of Learning, for Learning: Operationalizing Milestones Data for Program-Level Improvement.
Ontology highlight
ABSTRACT:
SUBMITTER: Heath JK
PROVIDER: S-EPMC8078066 | biostudies-literature | 2021 Apr
REPOSITORIES: biostudies-literature
Publications
Assessment of Learning, for Learning: Operationalizing Milestones Data for Program-Level Improvement.
Heath Janae K JK Edgar Laura L Guralnick Susan S
Journal of graduate medical education 20210423 2 Suppl
Similar Datasets
Project description:BACKGROUND: The internal medicine milestones were developed to advance outcomes-based residency training and will play an important role in the next accreditation system. INNOVATION: As an element of our program's participation in the internal medicine educational innovations project, we implemented a milestones-based evaluation process in our general medicine and pulmonary-critical care rotations on July 1, 2010. MEASURES: Outcomes assessed included survey-rated acceptability to participating faculty, residents, and clinical competency committee members. RESULTS: Faculty and residents agreed that the milestones promoted a common understanding of what knowledge, skills, and attitudes should be displayed at particular points in residents' professional development and enhanced evaluators' ability to provide specific performance feedback. Most residents and faculty members agreed that the milestones promoted fairness and uniformity in the evaluation process. Clinical competency committee members agreed the milestones improved the quality of information available for deliberations and resulted in more uniform promotion standards. Faculty rated the use of too many milestones per form/tool at a mean of 7.3 (where 1 was minimally problematic, and 10 was maximally problematic) and the potential for evaluator fatigue (mean, 8.2) as the most significant challenges to the use of milestones. Eight of 12 faculty members would recommend milestones in other programs; 4 were uncertain. CONCLUSIONS: Despite logistical challenges, educators and trainees found that milestones promoted a common understanding of what knowledge, skills and attitudes should be displayed at particular stages of training; permitted greater specificity in performance feedback; and enhanced uniformity and fairness in promotion decisions.
| S-EPMC3613298 | biostudies-other
Project description: Not available
| S-EPMC5476376 | biostudies-literature
Project description:ObjectiveThe objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.MethodTwo approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.ResultsSearch strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.ConclusionThe results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.
| S-EPMC3016654 | biostudies-literature
Project description:The Accreditation Council for Graduate Medical Education requires that trainees show progressive milestone attainment in the practice-based learning and systems-based practice competencies. As part of the Clinical Learning Environment Review, sponsoring hospitals must educate trainees in health care quality improvement, provide them with specialty-specific quality data, and ensure trainee participation in quality improvement activities and committees. Subspecialty-specific quality improvement curricula in nephrology training programs have not been reported, although considerable curricular and assessment material exists for specialty residencies, including tools for assessing trainee and faculty competence. Nephrology-specific didactic material exists to assist nephrology fellows and faculty mentors in designing and implementing quality improvement projects. Nephrology is notable among internal medicine subspecialties for the emphasis placed on adherence to quality thresholds-specifically for chronic RRT shown by the Centers for Medicare and Medicaid Services Quality Incentive Program. We have developed a nephrology-specific curriculum that meets Accreditation Council for Graduate Medical Education and Clinical Learning Environment Review requirements, acknowledges regulatory quality improvement requirements, integrates with ongoing divisional quality improvement activities, and has improved clinical care and the training program. In addition to didactic training in quality improvement, we track trainee compliance with Kidney Disease Improving Global Outcomes CKD and ESRD quality indicators (emphasizing Quality Improvement Program indicators), and fellows collaborate on a yearly multidisciplinary quality improvement project. Over the past 6 years, each fellowship class has, on the basis of a successful quality improvement project, shown milestone achievement in Systems-Based Practice and Practice-Based Learning. Fellow quality improvement projects have improved nephrology clinical care within the institution and introduced new educational and assessment tools to the training program. All have been opportunities for quality improvement scholarship. The curriculum prepares fellows to apply quality improvement principals in independent clinical practice-while showing milestone advancement and divisional compliance with Clinical Learning Environment Review requirements.
| S-EPMC5293331 | biostudies-literature
Project description:BackgroundDepression is common in the palliative care setting and impacts outcomes. Operationalized screening is unusual in palliative care.Local problemLack of operationalized depression screening at two ambulatory palliative care sites.MethodsA fellow-driven quality improvement initiative to implement operationalized depression screening using the patient health questionnaire-2 (PHQ-2). The primary measure was rate of EMR-documented depression screening. Secondary measures were clinician perspectives on the feasibility and acceptability of implementing the PHQ-2.InterventionThe intervention is a clinic-wide implementation of PHQ-2 screening supported by note templates, brief clinician training, referral resources for clinicians, and opportunities for indirect psychiatric consultation.ResultsOperationalized depression screening rates increased from 2% to 38%. All clinicians felt incorporation of depression screening was useful and feasible.ConclusionsOperationalized depression screening is feasible in ambulatory palliative care workflow, though optimization through having screening be completed prior to clinician visit might improve uptake.
| S-EPMC9790031 | biostudies-literature
Project description:Learning goals and objectives are a key part of instruction, informing curricular design, assessment, and learning. These goals and objectives are also applied at the programmatic level, with program learning outcomes (PLOs) providing insight into the skills that undergraduate biology programs intend for their students to master. PLOs are mandated by all major higher education accreditation agencies and play integral roles in programmatic assessment. Despite their importance, however, there have not been any prior attempts to characterize PLOs across undergraduate biology programs in the United States. Our study reveals that many programs may not be using PLOs to communicate learning goals with students. We also identify key themes across these PLOs and differences in skills listed between institution types. For example, some Vision & Change core competencies (e.g., interdisciplinary nature of science; connecting science to society; quantitative reasoning) are highlighted by a low percentage of programs, while others are shared more frequently between programs. Similarly, we find that biology programs at 4-year institutions likely emphasize PLOs relating to computational skills and research more than at 2-year institutions. We conclude by discussing implications for how to best use PLOs to support student learning, assessment, and curricular improvements.
| S-EPMC10074271 | biostudies-literature
Project description:BackgroundEvery hospital manager aims to build harmonious, mutually beneficial, and steady-state departments. Therefore, it is important to explore a hospital department development assessment model based on objective hospital data.ObjectiveThis study aims to use a novel machine learning algorithm to identify key evaluation indexes for hospital departments, offering insights for strategic planning and resource allocation in hospital management.MethodsData related to the development of a hospital department over the past 3 years were extracted from various hospital information systems. The resulting data set was mined using neural machine algorithms to assess the possible role of hospital departments in the development of a hospital. A questionnaire was used to consult senior experts familiar with the hospital to assess the actual work in each hospital department and the impact of each department's development on overall hospital discipline. We used the results from this questionnaire to verify the accuracy of the departmental risk scores calculated by the machine learning algorithm.ResultsDeep machine learning was performed and modeled on the hospital system training data set. The model successfully leveraged the hospital's training data set to learn, predict, and evaluate the working and development of hospital departments. A comparison of the questionnaire results with the risk ranking set from the departments machine learning algorithm using the cosine similarity algorithm and Pearson correlation analysis showed a good match. This indicates that the department development assessment model and risk score based on the objective data of hospital systems are relatively accurate and objective.ConclusionsThis study demonstrated that our machine learning algorithm provides an accurate and objective assessment model for hospital department development. The strong alignment of the model's risk assessments with expert opinions, validated through statistical analysis, highlights its reliability and potential to guide strategic hospital management decisions.
| S-EPMC11411220 | biostudies-literature
Project description:BackgroundThere is a focus on integrating quality improvement with medical education and advancement of the American College of Graduate Medical Education (ACGME) core competencies.ObjectiveTo determine if audits of patients with unexpected admission to the medical intensive care unit using a self-assessment tool and a focused Morbidity and Mortality (M&M) conference improves patient care.DesignCharts from patients transferred from the general medical floor (GMF) to the medical intensive care unit (ICU) were reviewed by a multidisciplinary team. Physician and nursing self-assessment tools and a targeted monthly M&M conference were part of the educational component.ParticipantsPhysicians and nurses participated in root cause analysis.MeasuresRecords of all patients transferred from a general medical floor (GMF) to the ICU were audited. One hundred ninety-four cases were reviewed over a 10-month period.ResultsNew policies regarding vital signs and house staff escalation of care were initiated. The percentage of calls for patients who met medical emergency response team/critical care consult criteria increased from 53% to 73%, nurse notification of a change in a patient's condition increased from 65% to 100%, nursing documentation of the change in the patients condition and follow-up actions increased from 65% percent to a high of 90%, the number of cardiac arrests on a GMF decreased from 3.1/1,000 discharges to 0.6/1,000 discharges (p = 0.002), and deaths on the Medicine Service decreased from 34/1,000 discharges to 24/1,000 discharges (p = 0.024).ConclusionWe describe an audit-based program that involves nurses, house staff, a self-assessment tool and a focused M&M conference. The program resulted in significant policy changes, more rapid assessment of unstable patients and improved hospital outcomes.
| S-EPMC2955460 | biostudies-literature
Project description:BackgroundSuccessful implementation of pay-for-quality (P4Q) programs mostly depends upon a valid, timely, and reliable data about quality measures generated by providers, and interpreted by payers. The aim of this study was to establish a data reporting method for P4Q program through an action research.MethodsQualitative method was used to align theory with action through a three-cycle action research. The study was conducted in September 15, 2015 to March 15, 2017, in East-Azerbaijan, Iran. The purposeful sampling was used to select participants. The participants included healthcare providers, staff in district health centers (DHC), experts, and managers in the provincial primary health center (PPHC). Data was collected by interviews, focus group discussions, and expert panels. Content analysis was used to synthesize the data. In each step, decisions about data reporting methods were made through a consensus of expert panel members.ResultsThe most important dimensions of data reporting method were data entry and accuracy, data reporting, data analysis and interpretations, the flexibility of method, and training. By establishment of an online data reporting system for the P4Q program, a major improvement was observed in the documentation of performance data, the satisfaction of health care providers and staff (e.g. either in DHCs or PPHC), improvement of the P4Q program and acceptance of the P4Q program by providers. Following the present study, the online system was expanded in Iran's public health system for data collection and estimating the amount of incentive payments in P4Q program. Moreover, more improvements were achieved by linking the system to EMRs and also, providing automated feedback to providers about their own performance.ConclusionsA web-based computerized system with the capability of linking medical record and also its ability to provide feedback to healthcare providers was identified as an appropriate method of data reporting in the P4Q program from the viewpoints of participants in this study.
| S-EPMC6128004 | biostudies-literature
Project description:Indigenous peoples have cultivated biodiverse agroecosystems since time immemorial. The rise of metagenomics and high-throughput sequencing technologies in biodiversity studies has rapidly expanded the scale of data collection from these lands. A respectful approach to the data life cycle grounded in the sovereignty of indigenous communities is imperative to not perpetuate harm. In this paper, we operationalize an indigenous data sovereignty (IDS) framework to outline realistic considerations for genomic data that span data collection, governance, and communication. As a case study for this framework, we use arthropod genomic data collected from diversified and simplified farm sites close to and far from natural habitats within a historic Kānaka 'Ōiwi (Indigenous Hawaiian) agroecosystem. Diversified sites had the highest Operational Taxonomic Unit (OTU) richness for native and introduced arthropods. There may be a significant spillover effect between forest and farm sites, as farm sites near a natural habitat had higher OTU richness than those farther away. We also provide evidence that management factors such as the number of Polynesian crops cultivated may drive arthropod community composition. Through this case study, we emphasize the context-dependent opportunities and challenges for operationalizing IDS by utilizing participatory research methods, expanding novel data management tools through the Local Contexts Hub, and developing and nurturing community partnerships-all while highlighting the potential of agroecosystems for arthropod conservation. Overall, the workflow and the example presented here can help researchers take tangible steps to achieve IDS, which often seems elusive with the expanding use of genomic data.
| S-EPMC11696483 | biostudies-literature