Implementation of a Web-Based Tool With Text Message Prompts to Improve End-of-Shift Assessments for Emergency Medicine Residents.
Ontology highlight
ABSTRACT: Background
End-of-shift assessments (ESA) can provide representative data on medical trainee performance but do not occur routinely and are not documented systematically.Objective
To evaluate the implementation of a web-based tool with text message prompts to assist mobile ESA (mESA) in an emergency medicine (EM) residency program.Methods
mESA used timed text messages to prompt faculty/trainees to expect in-person qualitative ESA in a milestone content area and for the faculty to record descriptive performance data through a web-based platform. We assessed implementation between January 2018 and November 2019 using the RE-AIM framework (reach, effectiveness, adoption, implementation, and maintenance).Results
Reach: 96 faculty and 79 trainees participated in the mESA program. Effectiveness: From surveys, approximately 72% of faculty and 58% of trainees reported increases in providing and receiving ESA feedback after program implementation. From ESA submissions, trainees reported receiving in-person feedback on 90% of shifts. Residency leadership confirmed perceived utility of the mESA program. Adoption: mESA prompts were sent on 7792 unique shifts across 4 EDs, all days of week, and different times of day. Faculty electronically submitted ESA feedback on 45% of shifts. Implementation quality: No technological errors occurred. Maintenance: Completion of in-person ESA feedback and electronic submission of feedback by faculty was stable over time.Conclusions
We found mixed evidence in support of using a web-based tool with text message prompts for mESA for EM trainees.
SUBMITTER: Tobias A
PROVIDER: S-EPMC7771609 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BackgroundUse of a person's name in a text message has been shown to be effective in instigating behaviour change. We evaluated the effectiveness of a personalised text message (including the recipient's name) versus a standardised text message for prompting a response from trial participants to complete and return postal follow-up questionnaires.MethodsUsing a randomised study within a trial (SWAT) embedded within the host GRASP (Getting it Right: Addressing Shoulder Pain) trial, participants who provided a mobile telephone number were randomised (1:1) by a central computer system to receive either (1) a personalised text message which included their name or (2) a standard text message. Text messages were sent by the trial office on the same day as the 6-month GRASP follow-up questionnaire. The primary outcome was questionnaire response rate, defined as the proportion of 6-month GRASP follow-up questionnaires returned by participants. Secondary outcomes included time to response, the proportion of participants sent a reminder follow-up questionnaire, and cost.ResultsBetween March 2017 and May 2019 (recruitment period for GRASP trial), 618 participants were randomised to a personalised (n = 309) or standard (n = 309) text message and all were included in the analysis. The overall questionnaire response rate was 87% (n = 537/618); 90% (n = 277/309) of participants responded in the personalised text message group compared to 84% (n = 260/309) in the standard text message group (relative risk (RR) 1.07; 95% CI 1.00 to 1.13). Participants randomised to receive the personalised text message were more likely to return their initial postal questionnaire than those who received the standard text message (n = 185/309; 60% vs. n = 160/309; 52%) (RR 1.16; 95% CI 1.00 to 1.33); this represents an absolute percentage difference between intervention groups of 8%. Post hoc subgroup analysis showed that males under 65 years were the group most likely to return their initial questionnaire if they received a personalised text message.ConclusionOverall, participants who received a personalised text message were more likely to return their questionnaire than those who received the standard text message.Trial registrationGRASP Trial ISRCTN16539266 ; SWAT Repository ID 35.
| S-EPMC8320189 | biostudies-literature
Project description:BackgroundEngagement with digital health interventions (DHIs) may be regarded as a prerequisite for the intervention to achieve positive health or behavior change outcomes. One method employed to promote engagement is the use of prompts such as emails and text messages. However, little is known about the characteristics of prompts that promote engagement. This study explored the association between the content and delivery mode of prompts and the users' engagement with HeLP-Diabetes (Healthy Living for People with type 2 Diabetes), a DHI that aimed to promote self-management in adults with type 2 diabetes.ObjectiveThe objective of this study was to identify the characteristics of prompts, specifically the content and delivery mode, which were associated with increased engagement.MethodsThis was a mixed-methods study. Email and text message prompts were sent to the registered users of HeLP-Diabetes. Use of the intervention was recorded and examined to identify which email and text message prompts were associated with subsequent visits to the DHI. Characteristics of prompts that were identified as particularly effective or ineffective were explored through think-aloud interviews with the participants.ResultsOf a total of 39 email prompts, 49% (19/39) prompts showed a significant association with subsequent visits to the DHI. However, none of the text message prompts were associated with subsequent visits to the DHI. Furthermore, think-aloud interviews were carried out with 6 experienced participants with type 2 diabetes. The findings suggest that these participants preferred email prompts that were clear, relatively short, and empowering; used nondirective advice; included health professional references; were visually appealing; and contained news and updates.ConclusionsThe findings of this study contribute to the existing evidence supporting the role of email prompts in promoting and maintaining engagement with DHIs. This study described the content of prompts that may be engaging. However, the results should be interpreted with caution, as prompts may be context-specific interventions and the results may not be generalizable across other DHIs or other types of interventions targeting self-management of type 2 diabetes.
| S-EPMC5566257 | biostudies-literature
Project description:BackgroundGender-related disparities persist in medicine and medical education. Prior work has found differences in medical education assessments based on gender.ObjectiveWe hypothesized that gender bias would be mitigated in a simulation-based assessment.MethodsWe conducted a retrospective cohort study of emergency medicine residents at a single, urban residency program. Beginning in spring 2013, residents participated in mandatory individual simulation assessments. Twelve simulated cases were included in this study. Rating forms mapped milestone language to specific observable behaviors. A Bayesian regression was used to evaluate the effect of resident and rater gender on assessment scores. Both 95% credible intervals (CrIs) and a Region of Practical Equivalence approach were used to evaluate the results.ResultsParticipants included 48 faculty raters (25 men [52%]) and 102 residents (47 men [46%]). The difference in scores between male and female residents (M = -0.58, 95% CrI -3.31-2.11), and male and female raters (M = 2.87, 95% CrI -0.43-6.30) was small and 95% CrIs overlapped with 0. The 95% CrI for the interaction between resident and rater gender also overlapped with 0 (M = 0.41, 95% CrI -3.71-4.23).ConclusionsIn a scripted and controlled system of assessments, there were no differences in scores due to resident or rater gender.
| S-EPMC6108359 | biostudies-literature
Project description:BackgroundThe emergency medicine milestones were developed to provide more objective resident assessment than current methods. However, little is known about the best practices for applying the milestones in resident assessment.ObjectiveWe examined the utility of end-of-shift evaluations (ESEs) constructed using the milestones in resident assessment.MethodsWe developed 14 daily ESEs, each of which included 9 or 10 emergency medicine milestones. Postgraduate year (PGY)-1 and PGY-2 residents were assessed on milestone levels 1 through 3; PGY-3 and PGY-4 residents were assessed on levels 3 through 5. Each milestone was rated on a nominal scale (yes, no, or not applicable). The Clinical Competency Committee combined the ESE data with data from other assessments to determine each resident's proficiency level for the emergency medicine subcompetencies. We used descriptive statistics to summarize resident ESEs and milestone levels. We analyzed differences in ESE score across PGY levels using t tests and analyses of variance.ResultsFaculty completed 763 ESEs on 33 residents with a range of 2 to 54 (median=22) ESEs per resident. Faculty rarely (8%, 372 of 4633) rated a resident as not achieving a milestone on the ESEs. Analyses of variance revealed that ESE scores on level 3 milestones did not differ significantly by PGY level. There was poor agreement between ESE scores and Clinical Competency Committee ratings.ConclusionsThe ESEs constructed using the milestones resulted in grade or milestone inflation. Our results do not support using milestones as a stand-alone assessment tool.
| S-EPMC4512788 | biostudies-literature
Project description:ObjectiveTo explore whether a text message-based relapse prevention intervention (Preventing and Interrupting Early Relapse [PIER]1) is acceptable to treatment-seeking adults with opioid use disorder (OUD) after Emergency Department (ED) discharge using mixed-methods design.MethodsAdults seeking care in an urban ED for OUD (n = 20; mean age 22; 55% female; 75% white race) completed a baseline survey, and were invited to enroll in PIER1, which was delivered in 7-day blocks, with the option to re-enroll at the end of each block, up to 4 blocks. PIER1 included a morning "push" message focused on positive thinking, adaptive coping feedback tailored to twice-daily assessments of craving severity and contextual correlates of craving, and end-of-day feedback on daily opioid use and goal commitment. Participants were asked to complete a follow-up phone interview after the first 7 days of PIER1. Transcripts were thematically coded.ResultsSeventeen out of 20 participants enrolled in PIER1. In the first 7 days, response rates to text-message assessments averaged 30%. Ten out of 17 participants re-enrolled after 7 days. Main themes from follow-up interviews (n = 9) included ease of use, social connection, and self-empowerment. Participants desired more personalized support and the ability to communicate through text messaging with another person about their struggles. Event-level data suggest that higher craving severity increased risk of opioid lapses.ConclusionsIn this mixed-methods intervention development study, we found conflicting evidence supporting an automated text-message intervention providing relapse prevention support for treatment-seeking individuals with OUD discharged from the ED. Qualitative feedback suggests that PIER1 could be useful and acceptability enhanced through personalized human support.
| S-EPMC5659903 | biostudies-literature
Project description:Prospective enrollment of research subjects in the fast-paced emergency department (ED) is challenging. We sought to develop a software application to increase real-time clinical trial enrollment during an ED visit. The Prospective Intelligence System for Clinical Emergency Services (PISCES) scans the electronic health record during ED encounters for preselected clinical characteristics of potentially eligible study participants and notifies the treating physician via mobile phone text alerts. PISCES alerts began 3 months into a cluster randomized trial of an electronic health record-based risk stratification tool for pediatric abdominal pain in 11 Northern California EDs. We compared aggregate enrollment before (2577 eligible patients, October 2016 to December 2016) and after (12 049 eligible patients, January 2017 to January 2018) PISCES implementation. Enrollment increased from 10.8% to 21.1% following PISCES implementations (P < .001). PISCES significantly increased study enrollment and can serve as a valuable tool to assist prospective research enrollment in the ED.
| S-EPMC6798557 | biostudies-literature
Project description:BackgroundImproving retention within randomised controlled trials is important. The effectiveness of different strategies can be assessed using a Study Within A Trial (SWAT). Previous research has shown personalised text message reminders improve clinic attendance rates; however, the results are mixed on improving postal questionnaire return. This SWAT aims to assess whether personalised text message reminders improve completion rates for scheduled telephone follow-ups.MethodsThis SWAT is a two-arm, multi-centre randomised controlled trial with equal allocation. The host trial was the Melatonin for Anxiety prior to General anaesthesia In Children trial (ISRCTN 18296119), where the child's caregiver was to answer a scheduled telephone follow-up 14 days post-surgery; participants for the SWAT were therefore the caregiver. Text messages were sent 24-48 h before the scheduled call and the personalised version contained the first name of the caregiver which was omitted in the non-personalised version. The primary outcome was questionnaire completion rate, defined as the proportion of caregivers successfully contacted, and completed any of the questionnaires, over the telephone within the follow-up window (day 14 + 7 days).ResultsThe SWAT included 100 of the 110 (91%) participants randomised into the host trial. Randomisation within the SWAT was equal between non-personalised (n = 50) and personalised (n = 50) interventions. The overall questionnaire response rate was 73% with a difference between the two interventions of 68% in the non-personalised text message arm and 78% in the personalised text message arm. The adjusted absolute risk difference was 7.1% (95% confidence interval = -10.2%, 24.4%). There was no difference in either the time to response or the number of contact attempts between the two interventions.ConclusionsThere is some evidence that personalised text messages could be effective at increasing response rates when data is collected via telephone and in a population of caregivers for paediatric trial participants. However, similar SWATs have shown mixed results. Given the low-cost and low risks associated with personalising text message reminders, this SWAT could be implemented easily in other RCTs scheduling telephone follow-up appointments.Trial registrationISRCTN 18296119 , SWAT 35 (MRC Northern Ireland Network for Trials Methodology Network).
| S-EPMC10848401 | biostudies-literature
Project description:To date, the practice of global emergency medicine (GEM) has involved being "on the ground" supporting in-country training of local learners, conducting research, and providing clinical care. This face-to-face interaction has been understood as critically important for developing partnerships and building trust. The COVID-19 pandemic has brought significant uncertainty worldwide, including international travel restrictions of indeterminate permanence. Following the 2020 Society for Academic Emergency Medicine meeting, the Global Emergency Medicine Academy (GEMA) sought to enhance collective understanding of best practices in GEM training with a focus on multidirectional education and remote collaboration in the setting of COVID-19. GEMA members led an initiative to outline thematic areas deemed most pertinent to the continued implementation of impactful GEM programming within the physical and technologic confines of a pandemic. Eighteen GEM practitioners were divided into four workgroups to focus on the following themes: advances in technology, valuation, climate impacts, skill translation, research/scholastic projects, and future challenges. Several opportunities were identified: broadened availability of technology such as video conferencing, Internet, and smartphones; online learning; reduced costs of cloud storage and printing; reduced carbon footprint; and strengthened local leadership. Skills and knowledge bases of GEM practitioners, including practicing in resource-poor settings and allocation of scarce resources, are translatable domestically. The COVID-19 pandemic has accelerated a paradigm shift in the practice of GEM, identifying a previously underrecognized potential to both strengthen partnerships and increase accessibility. This time of change has provided an opportunity to enhance multidirectional education and remote collaboration to improve global health equity.
| S-EPMC7821062 | biostudies-literature
Project description:Study objectiveOpportunistic brief in-person emergency department (ED) interventions can be effective at reducing hazardous alcohol use in young adult drinkers, but require resources frequently unavailable. Mobile telephone text messaging (short message service [SMS]) could sustainably deliver behavioral support to young adult patients, but efficacy remains unknown. We report 3-month outcome data of a randomized controlled trial testing a novel SMS-delivered intervention in hazardous-drinking young adults.MethodsWe randomized 765 young adult ED patients who screened positive for past hazardous alcohol use to one of 3 groups: SMS assessments+feedback (SA+F) intervention who were asked to respond to drinking-related queries and received real-time feedback through SMS each Thursday and Sunday for 12 weeks (n=384), SMS assessments (SA) who were asked to respond to alcohol consumption queries each Sunday but did not receive any feedback (N=196), and a control group who did not participate in any SMS (n=185). Primary outcomes were self-reported number of binge drinking days and number of drinks per drinking day in the past 30 days, collected by Web-based timeline follow-back method and analyzed with regression models. Secondary outcomes were the proportion of participants with weekend binge episodes and most drinks consumed per drinking occasion during 12 weekends, collected by SMS.ResultsWith Web-based data, there were decreases in the number of self-reported binge drinking days from baseline to 3 months in the SA+F group (-0.51 [95% confidence interval {CI} -0.10 to -0.95]), whereas there were increases in the SA group (0.90 [95% CI 0.23 to 1.6]) and the control group (0.41 [95% CI -0.20 to 1.0]). There were also decreases in the number of self-reported drinks per drinking day from baseline to 3 months in the SA+F group (-0.31 [95% CI -0.07 to -0.55]), whereas there were increases in the SA group (0.10 [95% CI -0.27 to 0.47]) and the control group (0.39 [95% CI 0.06 to 0.72]). With SMS data, there was a lower mean proportion of SA+F participants reporting a weekend binge during 12 weeks (30.5% [95% CI 25% to 36%) compared with the SA participants (47.7% [95% CI 40% to 56%]). There was also a lower mean drinks consumed per weekend during 12 weeks in the SA+F group (3.2 [95% CI 2.6 to 3.7]) compared to the SA group (4.8 [95% CI 4.0 to 5.6]).ConclusionA text message intervention can produce small reductions in self-reported binge drinking and the number of drinks consumed per drinking day in hazardous-drinking young adults after ED discharge.
| S-EPMC4254153 | biostudies-literature
Project description:BackgroundEarly and excessive alcohol use is a significant threat to healthy development. Evidence supports the effectiveness of electronic alcohol interventions for young drinkers. However, effects are typically small and studies targeting under 18-year-olds are scarce. This trial is the first to evaluate the effectiveness of a single-session, brief, motivational, web-based intervention (ProWISE) plus weekly text-message-initiated individualised prompts (TIPs) in reducing alcohol consumption and alcohol-related harm among children and adolescents aged ≥ 12 years. TIPs are designed to decrease risky alcohol use by reaching youth in the contexts of their everyday lives and by providing individualised feedback on drinking intentions, actual drinking and succession in achieving personal goals for low-risk drinking or abstinence.Methods/designThe trial is part of the multicentre consortium ProHEAD testing e-interventions for mental health problems in children and adolescents. Participants in grades 6-13 aged ≥ 12 years will be recruited in schools which participate in ProHEAD (target N = 15,000). Main criterion for inclusion in the ProWISE-TIP trial is a positive screening for at-risk alcohol use in the CRAFFT-d questionnaire (target n = 1076). In a multicentre, four-arm, randomised controlled design the following groups will be compared: (A) web-based intervention plus TIPs for 12 weeks; (B) web-based intervention plus text-message-initiated assessment of alcohol consumption for 12 weeks; (C) web-based intervention only; and (D) alcohol-related psychoeducation. TIPs will be delivered shortly before and after high-risk situations for excessive alcohol use and will be tailored to age, gender, drinking motives and alcohol consumption. Study participants will be followed up at three, six and nine months in the ProWISE-TIP trial and at one and two years in the ProHEAD consortium. Primary outcome is alcohol use in the past 30 days at nine months after enrolment. Secondary outcomes are alcohol-related problems, co-occurring substance use, health service utilisation, mental health problems and quality of life.DiscussionTrial results will generate important evidence on how to enhance effectiveness of single-session, web-based alcohol interventions for youth. The ProWISE-TIP intervention, if effective, can be used as a stand-alone alcohol intervention or as an add-on to school-based or community-based alcohol prevention programs.Trial registrationGerman Clinical Trials Register, DRKS00014606 Registered on 20 April 2018.
| S-EPMC6341631 | biostudies-literature