Comprehension and engagement in survey interviews with virtual agents.
Ontology highlight
ABSTRACT:
SUBMITTER: Conrad FG
PROVIDER: S-EPMC4611966 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:IntroductionVirtual interviews (VI) are now a permanent part of pediatric emergency medicine (PEM) recruitment, especially given the cost and equity advantages. Yet inability to visit programs in person can impact decision-making, leading applicants to apply to more programs. Moreover, the cost advantages of VI may encourage applicants to apply to programs farther away than they might otherwise have been willing or able to travel. This could create unnecessary strain on programs. We conducted this study to determine whether PEM fellowship applicants would apply to a larger number of programs and in different geographic patterns with VI (2020 and 2021) as compared to in-person interviews (2018 and 2019).MethodsWe conducted an anonymous national survey of all PEM fellows comparing two cohorts: current fellows who interviewed inperson (applied in 2018/2019) and fellows who underwent VIs in 2020/2021 (current fellows and those recently matched in 2021). The study took place in March-April 2022. Questions focused on geographic considerations during interviews and the match. We used descriptive statistics, chi-square and t-tests for analysis.ResultsOverall response rate was 42% (231/550); 32% (n = 74) interviewed in person and 68% (n = 157) virtually. Fellows applied to a median of 4/6 geographic regions (interquartile range 2, 5). Most applied for fellowship both in the same region as residency (216, 93%) and outside (192, 83%). Only the Pacific region saw a statistically significant increase in applicants during VI (59.9% vs 43.2%, P = 0.02). There was no statistical difference in the number of programs applied to during in-person vs VI (mean difference (95% confidence interval 0.72, -2.8 - 4.2). A majority matched in their preferred state both during VI (60.4%) and in-person interviews (65.7%). The difference was not statistically significant (P = 0.45).ConclusionWhile more PEM fellowship applicants applied outside the geographic area where their residency was and to the Pacific region, there was no overall increase in the number of programs or geographic areas PEM applicants applied to during VI as compared to in-person interview seasons. As this was the first two years of VI, ongoing data collection will further identify trends and the impactof VI.
| S-EPMC11000550 | biostudies-literature
Project description:PurposeTo determine whether respondents share researchers' understandings of concepts and questions frequently used in the assessment of usual physical activity (PA) behavior.MethodsAs part of On the Move, a study aimed at reducing measurement error in self-reported physical activity (PA), we conducted cognitive interviews with 19 men and 21 women, ages 45-65, regarding their responses to the PA questionnaires used in two large, population-based studies, Life After Cancer Epidemiology and California Men's Health study. One questionnaire asks about the frequency, the duration, and the perceived intensity of a range of specific activities in several different domains over the past 12 months. The second questionnaire asks about frequency and duration of specific, mostly recreational activities, grouped by intensity (i.e., moderate or vigorous) over the past 3 months. We used verbal probing techniques to allow respondents to describe their thought processes as they completed the questionnaires. All interviews were tape-recorded and transcribed, and the transcripts were then analyzed using standard qualitative methods.ResultsCognitive interviews demonstrated that a sizable number of respondents understood "intensity" in terms of emotional or psychological intensity rather than physical effort. As a result, the perceived intensity with which a participant reported doing a specific activity often bore little relationship to the MET value of that activity. Additionally, participants often counted the same activity more than once, overestimated work-related PA, and understood activities that were grouped together in a single category to be definitive lists rather than examples.ConclusionCognitive interviews revealed significant gaps between respondents' interpretations of some PA questions and researchers' assumptions about what those questions were intended to measure. Some sources of measurement error in self-reported PA may be minimized by additional research that focuses on the cognitive processes required to respond to PA questionnaires.
| S-EPMC2694667 | biostudies-literature
Project description:Background Residency selection in the United States relied on in-person interviews for many decades. The COVID-19 pandemic and recommendations from the Coalition for Physician Accountability (COPA) required programs to implement virtual interviews for the 2020–2021 residency selection cycle. Although virtual interviews may become the norm in the future, there is scant data at the institutional level to inform how to best approach this process. Objective To describe the perceptions of applicants to several residency programs at one institution on the importance of virtual recruitment features and assess the impact on their overall ranking decisions. Methods Applicants who interviewed for 12 medical and surgical residency programs during the 2020–2021 cycle at the University of California San Francisco were invited to participate in an anonymous survey in March 2021, after all interviews were completed. A survey consisting of 26 questions was administered to applicants on features that are important during interviews and the impact on their ranking decisions scored on a 5-point Likert scale. Results Of the 1422 participating applicants, 303 (21%) completed the survey. The most important feature for applicants during the interview day was getting a feel of the program (92%). Conversations with residents (91%) and faculty (79%) were also highly rated. Respondents reported morale and happiness of residents (71%) as an extremely important factor in their overall ranking decision . Conclusion Programs should consider prioritizing features that aid in alignment with getting to know residents and faculty and provide a sense of morale over emphasis on the institutional and location features. Supplementary Information The online version contains supplementary material available at 10.1007/s44186-022-00004-5.
| S-EPMC8906359 | biostudies-literature
Project description:Background and aimsAdvanced endoscopy (AE) fellowship interviews during the 2020 application cycle were held virtually due to COVID-19 pandemic. This study aimed to characterize the application experience with virtual interviews.MethodsApplicants and programs participating in the 2020 AE fellowship match were prospectively sent separate 17-question surveys. Responses were collected using Research Electronic Data Capture.ResultsIn total, 63/104 applicants (60.6%) matched and 64/71 (90.1%) positions filled within the match. The survey was completed by 37/87 (42.5%) applicants and 71 interviewers from 43/65 (66.2%) participating programs. Most applicants (64.7%) preferred a 1:1 applicant:faculty ratio and a 20-minute interview (73.5%), which was used by programs 50.7% and 47.8%, respectively. The majority of applicants (82.4%) and interviewers (76.8%) reported overall satisfaction with virtual interviews. The biggest limitations for applicants were getting a "feel" for the program and endoscopy unit. In the future, 41.2% of applicants preferred only virtual interviews compared to 7.2% of interviewers (P< 0.01). Conversely, 14.7% of applicants preferred only in-person interviews compared to 39.1% of interviewers (P= 0.01). Half of the interviewers (50.7%) would negatively view applicants who chose a virtual option over an in-person interview if both were offered.ConclusionThe majority of AE fellowship applicants and programs reported overall satisfaction with virtual interviews during the COVID-19 pandemic. Given the potential benefits, virtual interviews may have an increasing role in future residency and fellowship applications. Programs that conduct virtual interviews need to be aware of its limitations and have measures to improve the applicant experience and mitigate potential biases of interviewers.
| S-EPMC8528488 | biostudies-literature
Project description:BackgroundDue to the coronavirus disease 2019 (COVID-19) pandemic, all Obstetrics and Gynecology fellowship interviews were held virtually for the 2020 fellowship match cycle. The aim of this study was to describe our initial experience with virtual Obstetrics and Gynecology fellowship interviews and evaluate its effectiveness in assessing candidates.MethodsThis was a cross-sectional survey study that included all interviewing attending physicians and fellows from five Obstetrics and Gynecology subspecialties at a single academic institution following the 2020-2021 fellowship interview season. The survey consisted of 19 questions aimed to evaluate each subspecialty's virtual interview process, including its feasibility and performance in evaluating applicants. The primary outcome was the subjective utility of virtual interviews. Secondary outcomes included a comparison of responses from fellows and attending physicians.ResultsThirty-six attendings and fellows completed the survey (36/53, 68% response rate). Interviewers felt applicants were able to convey themselves adequately during the virtual interview (92%) and the majority (70%) agreed that virtual interviews should be offered in future years. Attending physicians were more likely than fellows to state that the virtual interview process adequately assessed the candidates (Likert Scale Mean: 4.4 vs. 3.8, respectively, p = 0.02). Respondents highlighted decreased cost, time saved, and increased flexibility as benefits to the virtual interview process.ConclusionThe use of virtual interviews provides a favorable method for conducting fellowship interviews and should be considered for use in future application cycles. Most respondents were satisfied with the virtual interview process and found they were an effective tool for evaluating applicants.
| S-EPMC8387093 | biostudies-literature
Project description:BackgroundIndividuals who have acquired communication disorders often struggle to transfer the skills they learn during therapy sessions to real-life situations. Immersive virtual reality (VR) technology has the potential to create realistic communication environments that can be used both in clinical settings and for practice at home by individuals with communication disorders.ObjectiveThis research aims to enhance our understanding of the acceptance, usefulness, and usability of a VR application (SIM:Kitchen), designed for communication rehabilitation. Additionally, this research aims to identify the perceived barriers and benefits of using VR technology from the perspective of individuals with acquired communication disorders.MethodsSemistructured interviews and usability surveys were conducted with 10 individuals with acquired neurogenic communication disorders aged 46-81 (mean 58, SD 9.57) years after trialing an immersive VR application. The audio-recorded interviews were transcribed and analyzed to identify themes.ResultsThe quantitative data regarding the usability of the system associated with participants' immersion experience in the VR application were promising. Findings from semistructured interviews are discussed across five key thematic areas including (1) participant's attitude toward VR, (2) perceived usefulness of the VR system, (3) perceived ease of use of the VR system, (4) their willingness to continue using VR, and (5) the factors they perceived as challenges or facilitators to adopting this VR technology.ConclusionsOverall, participants in this study found the VR experience to be enjoyable and were impressed by the realism of the VR application designed for communication rehabilitation. This study highlighted personally relevant, immersive VR interventions with different levels of task difficulty that could enhance technology uptake in the context of communication rehabilitation. However, it is essential that VR hand controller technology is refined to be more naturalistic in movement and able to accommodate user capabilities.
| S-EPMC10646677 | biostudies-literature
Project description:BackgroundTravel restrictions amidst the COVID-19 pandemic reshaped interviewing for fellowships into a predominantly virtual process. How this impacts Obstetrics and Gynecology (OB/GYN) resident approaches to fellowship application and Match navigation is largely unknown.MethodsWe performed a cross-sectional survey study of fourth year OB/GYN residents in the United States who participated in at least one virtual fellowship interview in 2020. We collected information regarding demographics, application strategy, perceived strengths and weaknesses of virtual interviews, and confidence with rank list creation. Descriptive statistics were used for categorical variables and responses pre- and post-Match were compared using Fisher's exact test.ResultsSeventy-five out of an estimated 490 applicants (~ 15% response rate) completed the survey. Of the respondents, 65.3% felt they interviewed at more programs virtually than they would anticipate completing in person, but perceived less confidence in having the necessary information (n = 45, 60%) or understanding the culture of programs (n = 59, 78.7%) to create a rank list. Cost savings were the main benefit of virtual interviews (n = 50, 66.7%), and inability to get a true "feel" for a program was the biggest limitation (n = 43, 57.3%). A majority (46.7%) advocate for a future hybrid interview process.ConclusionsOB/GYN residents pursuing fellowship reported interviewing at more programs during the virtual season, but had less confidence with rank list creation. Cost savings benefits are weighed against difficulty getting a "feel" for programs virtually. Most would advocate for a future hybrid interview process.
| S-EPMC8789205 | biostudies-literature
Project description:BackgroundThe shift from in-person to virtual residency interviews may impact greenhouse gas emissions (GHGE) and costs but the direction and amount of this change is not yet clear.ObjectiveTo estimate GHGE and financial impacts of virtual interviews among applicants and programs.MethodsIn 2020-2021 we sent a postinterview survey to 1429 applicants from 7 residency programs and 1 clinical psychology program at 1 institution. The survey collected origin of travel and transit type if in-person interviews had been held and excluded responses if the applicant would not have participated in an in-person interview, or if travel type or original city was missing. We used the International Civil Aviation Organization calculator to estimate flight-related GHGE in metric tons of carbon dioxide equivalent (MTCO2e) and Google Maps to estimate ground travel, with a standard CO2e per mile. Flight, hotel, and airport taxi costs were estimated using Expedia.com, Hotels.com, Uber, and Lyft. We aggregated these data and calculated median and interquartile ranges (IQRs) for applicant GHGE and cost savings, and assumed no cost or GHGE from virtual interviews. We used Wilcoxon signed rank sum tests to compare in-person 2019-2020 and virtual 2020-2021 GME program interview budgets.ResultsThe survey response rate was 565, or 40% of applicants; 543 remained after the exclusion criteria were applied. Reduction in applicant travel due to virtual interviews led to median estimated GHGE savings of 0.47 (IQR 0.30-0.61) MTCO2e and $490 (IQR $392-$544) per applicant, per interview. Programs savings ranged from $7,615 to $33,670 for the interview season.ConclusionsVirtual interviews in 8 GME programs were associated with lower estimated GHGE and costs, for applicants and programs, compared with in-person interviews.
| S-EPMC9934836 | biostudies-literature
Project description:ObjectiveDue to the COVID pandemic, restrictions were put in place mandating that all residency interviews be transitioned to a virtual format. Canadian CCFP(EM) programs were among the first to embark on this universal virtual interview process for resident selection. Although there have been several recent publications suggesting best practice guidelines for virtual interviews in trainee selection, pragmatic experiences and opinions from Program Directors (PDs) are lacking. This study aimed to elicit the experiences and perspectives of CCFP(EM) PDs after being amongst the first to conduct universal virtual interviews in Canada.MethodsA 17-item online survey was created and distributed to all CCFP(EM) PDs (n = 17). It explored the virtual interview format employed, perceived advantages and disadvantages of a virtual configuration, confidence in determining a candidate's rank order, and PD preference for employing a virtual interview format in the future. It also elicited practical advice to conduct a smooth and successful virtual interview day.ResultsThe survey response rate was 76.5% (13/17). Nine respondents (69.2%) agreed that the virtual interview format enabled them to confidently determine a candidate's rank order. With respect to preference for future use of virtual interviews, 23.1% agreed, 38.5% disagreed and 38.5% neither agreed nor disagreed. Inductive thematic analysis of free text responses revealed themes related to virtual interview advantages (time, financial, and resource costs), disadvantages (difficulty promoting smaller programs, getting a 'feel' for candidates and assessing their interpersonal skills), and practical tips to facilitate virtual interview processes.ConclusionOnce restrictions are lifted, cost-saving advantages must be weighed against suggested disadvantages such as showcasing program strengths and assessing interpersonal skills in choosing between traditional and virtual formats. Should virtual interviews become a routine part of resident selection, the advice suggested in this study may be considered to help optimize a successful virtual interview process.
| S-EPMC9150630 | biostudies-literature
Project description:ObjectiveThe aim of this study was to assess applicant and fellowship director (FD) perspectives on virtual interviewing based on the 2019 to 2020 orthopaedic trauma fellowship interview experience and to develop recommendations for future application cycles.MethodsWeb-based surveys were distributed to all matched applicants and orthopaedic trauma FDs after the 2019 and 2020 orthopaedic trauma fellowship match. Thirty-one applicants and 23 FDs completed the survey-response rates of 34% and 38%, respectively.ResultsVirtual interviews were completed by 68% of applicants and 17% of FDs. Twenty-nine percent of applicants felt they were able to familiarize themselves with the culture of programs, and 38% of applicants were satisfied with their ability to present themselves. Most (62%) were comfortable ranking programs based on the virtual interview, but 38% reported the format influenced them to rank a program lower. Among all applicants, 77% preferred the in-person interview. Most FDs (75%) reported virtual interviews limited their ability to familiarize themselves with an applicant, and only 50% were comfortable ranking an applicant afterward. Still, 78% of FDs believe there is a role for virtual interviews in the fellowship match. Choosing a virtual interview may negatively affect applicants as 97% of applicants worry the choice conveys less interest to programs, while 43% of FDs would interpret it as less interest.ConclusionsVirtual interviews have multiple shortcomings but are technically feasible and provide reasonable information to applicants and FDs to complete the match process. Our recommendations, based on the perspectives of applicants and FDs, can guide their implementation.
| S-EPMC8568371 | biostudies-literature