Dataset Information

Measuring faculty reflection on adverse patient events: development and initial validation of a case-based learning system.

ABSTRACT:

Background

Critical reflection by faculty physicians on adverse patient events is important for changing physician's behaviors. However, there is little research regarding physician reflection on quality improvement (QI).

Objective

To develop and validate a computerized case-based learning system (CBLS) to measure faculty physicians' reflections on adverse patient events.

Design

Prospective validation study.

Participants

Staff physicians in the Department of Medicine at Mayo Clinic Rochester.

Main measures

The CBLS was developed by Mayo Clinic information technology, medical education, and QI specialists. The reflection questionnaire, adapted from a previously validated instrument, contained eight items structured on five-point scales. Three cases, representing actual adverse events, were developed based on the most common error types: systems, medication, and diagnostic. In 2009, all Mayo Clinic hospital medicine, non-interventional cardiology, and pulmonary faculty were invited to participate. Faculty reviewed each case, determined the next management step, rated case generalizability and relevance, and completed the reflection questionnaire. Factor analysis and internal consistency reliability were calculated. Associations between reflection scores and characteristics of faculty and patient cases were determined.

Key results

Forty-four faculty completed 107 case reflections. The CBLS was rated as average to excellent in 95 of 104 (91.3%) completed satisfaction surveys. Factor analysis revealed two levels of reflection: Minimal and High. Internal consistency reliability was very good (overall Cronbach's ?=0.77). Item mean scores ranged from 2.89 to 3.73 on a five-point scale. The overall reflection score was 3.41 (standard deviation 0.64). Reflection scores were positively associated with case generalizability (p=0.001), and case relevance (p=0.02).

Conclusions

The CBLS is a valid method for stratifying faculty physicians' levels of reflection on adverse patient events. Reflection scores are associated with case generalizability and relevance, indicating that reflection improves with pertinent patient encounters. We anticipate that this instrument will be useful in future research on QI among low versus high-reflecting physicians.

SUBMITTER: Wittich CM

PROVIDER: S-EPMC3043183 | biostudies-literature | 2011 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measuring faculty reflection on adverse patient events: development and initial validation of a case-based learning system.

Wittich Christopher M CM Lopez-Jimenez Francisco F Decker Lindsay K LK Szostek Jason H JH Mandrekar Jayawant N JN Morgenthaler Timothy I TI Beckman Thomas J TJ

Journal of general internal medicine 20101027 3

<h4>Background</h4>Critical reflection by faculty physicians on adverse patient events is important for changing physician's behaviors. However, there is little research regarding physician reflection on quality improvement (QI).<h4>Objective</h4>To develop and validate a computerized case-based learning system (CBLS) to measure faculty physicians' reflections on adverse patient events.<h4>Design</h4>Prospective validation study.<h4>Participants</h4>Staff physicians in the Department of Medicine ...[more]

PMID: 20978863

Similar Datasets

Project description:BackgroundProblem-based learning (PBL) is a pedagogy involving self-directed learning in small groups around case problems. Group function is important to PBL outcomes, but there is currently poor scaffolding around key self-reflective practices that necessarily precedes students' and tutors' attempts to improve group function. This study aims to create a structured, literature-based and stakeholder-informed tool to help anchor reflective practices on group function. This article reports on the development process and perceived utility of this tool.MethodsTool development unfolded in four steps: 1) a literature review was conducted to identify existent evaluation tools for group function in PBL, 2) literature findings informed the development of this new tool, 3) a group of PBL experts were consulted for engagement with and feedback of the tool, 4) four focus groups of stakeholders (medical students and tutors with lived PBL experiences) commented on the tool's constructs, language, and perceived utility. The tool underwent two rounds of revisions, informed by the feedback from experts and stakeholders.ResultsNineteen scales relating to group function assessment were identified in the literature, lending 18 constructs that mapped into four dimensions: Learning Climate, Facilitation and Process, Engagement and Interactivity, and Evaluation and Group Improvement. Feedback from experts informed the addition of missing items. Focus group discussions allowed further fine-tuning of the organization and language of the tool. The final tool contains 17 descriptive items under the four dimensions. Users are asked to rate each dimension holistically on a 7-point Likert scale and provide open comments. Researchers, faculty, and students highlighted three functions the tool could perform: (1) create space, structure, and language for feedback processes, (2) act as a reference, resource, or memory aid, and (3) serve as a written record for longitudinal benchmarking. They commented that the tool may be particularly helpful for inexperienced and poor-functioning groups, and indicated some practical implementation considerations.ConclusionA four-dimension tool to assist group function reflection in PBL was produced. Its constructs were well supported by literature and experts. Faculty and student stakeholders acknowledged the utility of this tool in addressing an acknowledged gap in group function reflection in PBL.

Project description:PurposeReflective capacity is "the ability to understand critical analysis of knowledge and experience to achieve deeper meaning." In medicine, there is little provision for post-graduate medical education to teach deliberate reflection. The feasibility, scoring characteristics, reliability, validation, and adaptability of a modified previously validated instrument was examined for its usefulness assessing reflective capacity in residents as a step toward developing interventions for improvement.MethodsThird-year residents and fellows from four anesthesia training programs were administered a slightly modified version of the Reflection Evaluation for Learners' Enhanced Competencies Tool (REFLECT) in a prospective, observational study at the end of the 2019 academic year. Six written vignettes of imperfect anesthesia situations were created. Subjects recorded their perspectives on two randomly assigned vignettes. Responses were scored using a 5-element rubric; average scores were analyzed for psychometric properties. An independent self-report assessment method, the Cognitive Behavior Survey: Residency Level (rCBS) was used to examine construct validity. Internal consistency (ICR, Cronbach's alpha) and interrater reliability (weighted kappa) were examined. Pearson correlations were used between the two measures of reflective capacity.Results46/136 invited subjects completed 2/6 randomly assigned vignettes. Interrater agreement was high (k = 0.85). The overall average REFLECT score was 1.8 (1-4 scale) with good distribution across the range of scores. ICR for both the REFLECT score (mean 1.8, sd 0.5; α = 0.92) and the reflection scale of the rCBS (mean 4.5, sd 1.1; α = 0.94) were excellent. There was a significant correlation between REFLECT score and the rCBS reflection scale (r = .44, p < 0.01).ConclusionsThis study demonstrates feasibility, reliability, and sufficiently robust psychometric properties of a modified REFLECT rubric to assess graduate medical trainees' reflective capacity and established construct/convergent validity to an independent measure. The instrument has the potential to assess the effectiveness of interventions intended to improve reflective capacity.

Project description:BACKGROUND:Existing studies have explored many aspects of medical students' experiences of patient death and propose the importance of faculty support for coping. However, UK-based literature on this subject and research concerning learning through reflection as part of coping are relatively limited. This study, through the lens of reflection, aims to explore students' experiences with patient death in a UK context. These include coping strategies, support from faculty following patient death and the relationship between these experiences and learning. Our research questions were: How do medical students cope with and learn from their experiences?How does support from ward staff and the medical school help them cope with and learn from these experiences?How can students best be supported following patient death? METHODS:We employed narrative inquiry to explore how medical students made sense of their experiences of patient death. Twelve students participated in our study via an online narrative questionnaire. Thematic analysis and complementary narrative analysis of an exemplar were applied to address our research aim. RESULTS:Coping strategies comprised internal and external strategies. Internal strategies included (1) re-interpretation of the death into a meaningful experience including lessons learned; (2) normalization; (3) staying busy and (4) enduring negative emotions. External strategies included speaking to someone, which was found to influence normalization, and lessons learned. Both satisfactory and unsatisfactory support from ward staff was identified. Satisfactory support was characterized by the inclusion of emotional and professional support. Unsatisfactory support was often characterized by a lack of emotional support. Narrative analysis further demonstrated how the experience with patient death was re-interpreted meaningfully. Students suggested that support should be structured, active, sensitive, and include peers and near-peers. CONCLUSION:Many coping strategies, internal and external, were employed in students' experiences with patient death. Student reflections, enhanced by support from ward staff, were shown to be important for learning from patient death. We encourage faculty to have regular sessions in which medical students can reflect on the death incident and discuss appropriately with others, including peers and near-peers.

Project description:BackgroundHemodialysis (HD) therapy is an indispensable tool used in critical care management. Patients undergoing HD are at risk for intradialytic adverse events, ranging from muscle cramps to cardiac arrest. So far, there is no effective HD device-integrated algorithm to assist medical staff in response to these adverse events a step earlier during HD.ObjectiveWe aimed to develop machine learning algorithms to predict intradialytic adverse events in an unbiased manner.MethodsThree-month dialysis and physiological time-series data were collected from all patients who underwent maintenance HD therapy at a tertiary care referral center. Dialysis data were collected automatically by HD devices, and physiological data were recorded by medical staff. Intradialytic adverse events were documented by medical staff according to patient complaints. Features extracted from the time series data sets by linear and differential analyses were used for machine learning to predict adverse events during HD.ResultsTime series dialysis data were collected during the 4-hour HD session in 108 patients who underwent maintenance HD therapy. There were a total of 4221 HD sessions, 406 of which involved at least one intradialytic adverse event. Models were built by classification algorithms and evaluated by four-fold cross-validation. The developed algorithm predicted overall intradialytic adverse events, with an area under the curve (AUC) of 0.83, sensitivity of 0.53, and specificity of 0.96. The algorithm also predicted muscle cramps, with an AUC of 0.85, and blood pressure elevation, with an AUC of 0.93. In addition, the model built based on ultrafiltration-unrelated features predicted all types of adverse events, with an AUC of 0.81, indicating that ultrafiltration-unrelated factors also contribute to the onset of adverse events.ConclusionsOur results demonstrated that algorithms combining linear and differential analyses with two-class classification machine learning can predict intradialytic adverse events in quasi-real time with high AUCs. Such a methodology implemented with local cloud computation and real-time optimization by personalized HD data could warn clinicians to take timely actions in advance.

Project description:BackgroundIntegrating the patient's perspective has become an increasingly important component of adverse event reporting. The National Cancer Institute has developed a Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™). This instrument has been translated into German and linguistically validated; however, its quantitative measurement properties have not been evaluated.Patients and methodsA German language survey that included 31 PRO-CTCAE items, as well as the EORTC QLQ-C30 and the Oral Mucositis Daily Questionnaire (OMDQ), was distributed at 10 cancer treatment settings in Germany and Austria. Item quality was assessed by analysis of acceptability and comprehensibility. Reliability was evaluated by using Cronbach's' alpha and validity by principal components analysis (PCA), multitrait-multimethod matrix (MTMM) and known groups validity techniques.ResultsOf 660 surveys distributed to the study centres, 271 were returned (return rate 41%), and data from 262 were available for analysis. Participants' median age was 59.7 years, and 69.5% of the patients were female. Analysis of item quality supported the comprehensibility of the 31 PRO-CTCAE items. Reliability was very good; Cronbach's' alpha correlation coefficients were >0.9 for almost all item clusters. Construct validity of the PRO-CTCAE core item set was shown by identifying 10 conceptually meaningful item clusters via PCA. Moreover, construct validity was confirmed by the MTMM: monotrait-heteromethod comparison showed 100% high correlation, whereas heterotrait-monomethod comparison indicated 0% high correlation. Known groups validity was supported; PRO-CTCAE scores were significantly lower for those with impaired versus preserved health-related quality of life.ConclusionA set of 31 items drawn from the German PRO-CTCAE item library demonstrated favourable measurement properties. These findings add to the body of evidence that PRO-CTCAE provides a rigorous method to capture patient self-reports of symptomatic toxicity for use in cancer clinical trials.

Dataset Information

Measuring faculty reflection on adverse patient events: development and initial validation of a case-based learning system.

Background

Objective

Design

Participants

Main measures

Key results

Conclusions

Publications

Measuring faculty reflection on adverse patient events: development and initial validation of a case-based learning system.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets