Dataset Information

Internal Medicine Resident Engagement with a Laboratory Utilization Dashboard: Mixed Methods Study.

ABSTRACT: The objective of this study was to measure internal medicine resident engagement with an electronic medical record-based dashboard providing feedback on their use of routine laboratory tests relative to service averages. From January 2016 to June 2016, residents were e-mailed a snapshot of their personalized dashboard, a link to the online dashboard, and text summarizing the resident and service utilization averages. We measured resident engagement using e-mail read-receipts and web-based tracking. We also conducted 3 hour-long focus groups with residents. Using grounded theory approach, the transcripts were analyzed for common themes focusing on barriers and facilitators of dashboard use. Among 80 residents, 74% opened the e-mail containing a link to the dashboard and 21% accessed the dashboard itself. We did not observe a statistically significant difference in routine laboratory ordering by dashboard use, although residents who opened the link to the dashboard ordered 0.26 fewer labs per doctor-patient-day than those who did not (95% confidence interval, -0.77 to 0.25; ? = 0 .31). While they raised several concerns, focus group participants had positive attitudes toward receiving individualized feedback delivered in real time.

SUBMITTER: Kurtzman G

PROVIDER: S-EPMC5803096 | biostudies-literature | 2017 Sep

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Internal Medicine Resident Engagement with a Laboratory Utilization Dashboard: Mixed Methods Study.

Kurtzman Gregory G Dine Jessica J Epstein Andrew A Gitelman Yevgenly Y Leri Damien D Patel Miltesh S MS Ryskina Kyra K

Journal of hospital medicine 20170901 9

The objective of this study was to measure internal medicine resident engagement with an electronic medical record-based dashboard providing feedback on their use of routine laboratory tests relative to service averages. From January 2016 to June 2016, residents were e-mailed a snapshot of their personalized dashboard, a link to the online dashboard, and text summarizing the resident and service utilization averages. We measured resident engagement using e-mail read-receipts and web-based tracki ...[more]

PMID: 28914280

Similar Datasets

Project description:ObjectiveTo understand emergency department (ED) physicians' use of electronic documentation in order to identify usability and workflow considerations for the design of future ED information system (EDIS) physician documentation modules.MethodsWe invited emergency medicine resident physicians to participate in a mixed methods study using task analysis and qualitative interviews. Participants completed a simulated, standardized patient encounter in a medical simulation center while documenting in the test environment of a currently used EDIS. We recorded the time on task, type and sequence of tasks performed by the participants (including tasks performed in parallel). We then conducted semi-structured interviews with each participant. We analyzed these qualitative data using the constant comparative method to generate themes.ResultsEight resident physicians participated. The simulation session averaged 17 minutes and participants spent 11 minutes on average on tasks that included electronic documentation. Participants performed tasks in parallel, such as history taking and electronic documentation. Five of the 8 participants performed a similar workflow sequence during the first part of the session while the remaining three used different workflows. Three themes characterize electronic documentation: (1) physicians report that location and timing of documentation varies based on patient acuity and workload, (2) physicians report a need for features that support improved efficiency; and (3) physicians like viewing available patient data but struggle with integration of the EDIS with other information sources.ConclusionWe confirmed that physicians spend much of their time on documentation (65%) during an ED patient visit. Further, we found that resident physicians did not all use the same workflow and approach even when presented with an identical standardized patient scenario. Future EHR design should consider these varied workflows while trying to optimize efficiency, such as improving integration of clinical data. These findings should be tested quantitatively in a larger, representative study.

Project description:BACKGROUND:Feedback is essential for improving the skills of continuing medical education (CME) presenters. However, there has been little research on improving the quality of feedback to CME presenters. OBJECTIVES:To validate an instrument for generating balanced and behavior-specific feedback from a national cross-section of participants to presenters at a large internal medicine CME course. DESIGN, SETTING, AND PARTICIPANTS:A prospective, randomized validation study with qualitative data analysis that included all 317 participants at a Mayo Clinic internal medicine CME course in 2009. MEASUREMENTS:An 8-item (5-point Likert scales) CME faculty assessment enhanced study form (ESF) was designed based on literature and expert review. Course participants were randomized to a standard form, a generic study form (GSF), or the ESF. The dimensionality of instrument scores was determined using factor analysis to account for clustered data. Internal consistency and interrater reliabilities were calculated. Associations between overall feedback scores and presenter and presentation variables were identified using generalized estimating equations to account for multiple observations within talk and speaker combinations. Two raters reached consensus on qualitative themes and independently analyzed narrative entries for evidence of balanced and behavior-specific comments. RESULTS:Factor analysis of 5,241 evaluations revealed a uni-dimensional model for measuring CME presenter feedback. Overall internal consistency (Cronbach alpha?=?0.94) and internal consistency reliability (ICC range 0.88-0.95) were excellent. Feedback scores were associated with presenters' academic ranks (mean score): Instructor (4.12), Assistant Professor (4.38), Associate Professor (4.56), Professor (4.70) (p?=?0.046). Qualitative analysis revealed that the ESF generated the highest numbers of balanced comments (GSF?=?11, ESF?=?26; p?=?0.01) and behavior-specific comments (GSF?=?64, ESF?=?104; p?=?0.001). CONCLUSIONS:We describe a practical and validated method for generating balanced and behavior-specific feedback for CME presenters in internal medicine. Our simple method for prompting course participants to give balanced and behavior-specific comments may ultimately provide CME presenters with feedback for improving their presentations.

Project description:There are wide variations in maternal-newborn care practices and outcomes across Ontario. To help institutions and care providers learn about their own performance, the Better Outcomes Registry & Network (BORN) Ontario has implemented an audit and feedback system, the Maternal-Newborn Dashboard (MND), for all hospitals providing maternal-newborn care. The dashboard provides (1) near real-time feedback, with site-specific and peer comparison data about six key performance indicators; (2) a visual display of evidence-practice gaps related to the indicators; and (3) benchmarks to provide direction for practice change. This study aims to evaluate the effects of the dashboard, dashboard attributes, contextual factors, and facilitation/support needs that influence the use of this audit and feedback system to improve performance. The objectives of this study are to (1) evaluate the effect of implementing the dashboard across Ontario; (2) explore factors that potentially explain differences in the use of the MND among hospitals; (3) measure factors potentially associated with differential effectiveness of the MND; and (4) identify factors that predict differences in hospital performance.A mixed methods design includes (1) an interrupted time series analysis to evaluate the effect of the intervention on six indicators, (2) key informant interviews with a purposeful sample of directors/managers from up to 20 maternal-newborn care hospitals to explore factors that influence the use of the dashboard, (3) a provincial survey of obstetrical directors/managers from all maternal-newborn hospitals in the province to measure factors that influence the use of the dashboard, and (4) a multivariable generalized linear mixed effects regression analysis of the indicators at each hospital to quantitatively evaluate the change in practice following implementation of the dashboard and to identify factors most predictive of use.Study results will provide essential data to develop knowledge translation strategies for facilitating practice change, which can be further evaluated through a future cluster randomized trial.

Project description:BackgroundPreoperative medical consultations add expense and burden for patients and the impact of these consults on patient outcomes is conflicting. Previous work suggests that 10-40% of preoperative medical consult recommendations are not followed. This limits measurement of the effect of perioperative medical consultation on patient outcomes and represents a quality gap, given the patient time and healthcare cost associated with consultation. We aimed to measure, characterize, and understand reasons for missed recommendations from preoperative medical consultation.MethodsThis explanatory, sequential mixed-methods study used chart audits followed by semi-structured interviews. Chart audit of consecutive patients seen in preoperative medical clinic were reviewed to measure the proportion and characterize the type of recommendations that were not completed ("missed"). This phase informed the interview participants and questions. The interview guide was developed using the Consolidated Framework for Implementation Research and the Theoretical Domains Framework. Template analysis was used to understand drivers and barriers of missed recommendations RESULTS: Chart audit included 255 patients (n=161, 63.1% female) seen in preadmission clinic between April 1 and April 30, 2019. 55.7% of patients had all recommendations followed (n=142). Postoperative anticoagulation management and postoperative cardiac biomarker surveillance recommendations were least commonly followed (50.0%, n=28, and 68.9%, n=82, respectively). Eighteen surgical team members were interviewed. Missed recommendations were both unintentional and intentional, and the key drivers differed by these categories. Unintentionally missed recommendations occurred due to individual-level factors (drivers: knowledge of the consultation note, lack of routine for reviewing the consultation note, and competing demands on time) and systems-level factors (driver: lack of role clarity). Intentionally missed recommendations occurred due to user error due (drivers: lack of knowledge of guidelines or evidence) and appropriate modifications (driver: need to adapt a preoperative plan for a complicated postoperative course).ConclusionsOnly 55.7% of consult notes had all recommendations followed, suggesting a quality gap in perioperative medical care. Qualitative data suggests multiple drivers of missed recommendations that should be targeted to improve the efficiency of care for these patients.

Project description:BackgroundFeedback is a critical element of graduate medical education. Narrative comments on evaluation forms are a source of feedback for residents. As a shared mental model for performance, milestone-based evaluations may impact narrative comments and resident perception of feedback.ObjectiveTo determine if milestone-based evaluations impacted the quality of faculty members' narrative comments on evaluations and, as an extension, residents' perception of feedback.DesignConcurrent mixed methods study, including qualitative analysis of narrative comments and survey of resident perception of feedback.ParticipantsSeventy internal medicine residents and their faculty evaluators at the University of Utah.ApproachFaculty narrative comments from 248 evaluations pre- and post-milestone implementation were analyzed for quality and Accreditation Council for Graduate Medical Education competency by area of strength and area for improvement. Seventy residents were surveyed regarding quality of feedback pre- and post-milestone implementation.Key resultsQualitative analysis of narrative comments revealed nearly all evaluations pre- and post-milestone implementation included comments about areas of strength but were frequently vague and not related to competencies. Few evaluations included narrative comments on areas for improvement, but these were of higher quality compared to areas of strength (p < 0.001). Overall resident perception of quality of narrative comments was low and did not change following milestone implementation (p = 0.562) for the 86% of residents (N = 60/70) who completed the pre- and post-surveys.ConclusionsThe quality of narrative comments was poor, and there was no evidence of improved quality following introduction of milestone-based evaluations. Comments on areas for improvement were of higher quality than areas of strength, suggesting an area for targeted intervention. Residents' perception of feedback quality did not change following implementation of milestone-based evaluations, suggesting that in the post-milestone era, internal medicine educators need to utilize additional interventions to improve quality of feedback.

Dataset Information

Internal Medicine Resident Engagement with a Laboratory Utilization Dashboard: Mixed Methods Study.

Publications

Internal Medicine Resident Engagement with a Laboratory Utilization Dashboard: Mixed Methods Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets