Latent classes associated with the intention to use a symptom checker for self-triage
Ontology highlight
ABSTRACT: It is currently unknown which attitude-based profiles are associated with symptom checker use for self-triage. We sought to identify, among university students, attitude-based latent classes (population profiles) and the association between latent classes with the future use of symptom checkers for self-triage. Informed by the Technology Acceptance Model and a larger mixed methods study, a cross-sectional survey was developed and administered to students (aged between 18 and 34 years of age) at a University in Ontario. Latent class analysis (LCA) was used to identify attitude-based profiles that exist among the sample while general linear modeling was applied to identify the association between latent classes and future symptom checker use for self-triage. Of the 1,547 students who opened the survey link, 1,365 did not use a symptom checker in the past year and were thus identified as “non-users”. After removing missing data (remaining sample = n = 1,305), LCA revealed five attitude-based profiles: tech acceptors, tech rejectors, skeptics, tech seekers, and unsure acceptors. Tech acceptors and tech rejectors were the most and least prevalent classes, respectively. As compared to tech rejectors, tech seekers and unsure acceptors were the latent classes with the highest and lowest odds of future symptom checker use, respectively. After controlling for confounders, the effect of latent classes on symptom checker use remains significant (p-value < .0001) with the odds of future use in tech acceptors being 5.6 times higher than the odds of future symptom checker use in tech rejectors [CI: (3.458, 9.078); p-value < .0001]. Attitudes towards AI and symptom checker functionality result in different population profiles that have different odds of using symptom checkers for self-triage. Identifying a person’s or group’s membership to a population profile could help in developing and delivering tailored interventions aimed at maximizing use of validated symptom checkers.
Project description:BackgroundSymptom checkers are digital tools assisting laypersons in self-assessing the urgency and potential causes of their medical complaints. They are widely used but face concerns from both patients and health care professionals, especially regarding their accuracy. A 2015 landmark study substantiated these concerns using case vignettes to demonstrate that symptom checkers commonly err in their triage assessment.ObjectiveThis study aims to revisit the landmark index study to investigate whether and how symptom checkers' capabilities have evolved since 2015 and how they currently compare with laypersons' stand-alone triage appraisal.MethodsIn early 2020, we searched for smartphone and web-based applications providing triage advice. We evaluated these apps on the same 45 case vignettes as the index study. Using descriptive statistics, we compared our findings with those of the index study and with publicly available data on laypersons' triage capability.ResultsWe retrieved 22 symptom checkers providing triage advice. The median triage accuracy in 2020 (55.8%, IQR 15.1%) was close to that in 2015 (59.1%, IQR 15.5%). The apps in 2020 were less risk averse (odds 1.11:1, the ratio of overtriage errors to undertriage errors) than those in 2015 (odds 2.82:1), missing >40% of emergencies. Few apps outperformed laypersons in either deciding whether emergency care was required or whether self-care was sufficient. No apps outperformed the laypersons on both decisions.ConclusionsTriage performance of symptom checkers has, on average, not improved over the course of 5 years. It decreased in 2 use cases (advice on when emergency care is required and when no health care is needed for the moment). However, triage capability varies widely within the sample of symptom checkers. Whether it is beneficial to seek advice from symptom checkers depends on the app chosen and on the specific question to be answered. Future research should develop resources (eg, case vignette repositories) to audit the capabilities of symptom checkers continuously and independently and provide guidance on when and to whom they should be recommended.
Project description:BackgroundComputerized algorithms known as symptom checkers aim to help patients decide what to do should they have a new medical concern. However, despite widespread implementation, most studies on symptom checkers have involved simulated patients. Only limited evidence currently exists about symptom checker safety or accuracy when used by real patients. We developed a new prototype symptom checker and assessed its safety and accuracy in a prospective cohort of patients presenting to primary care and emergency departments with new medical concerns.MethodA prospective cohort study was done to assess the prototype's performance. The cohort consisted of adult patients (≥16 years old) who presented to hospital emergency departments and family physician clinics. Primary outcomes were safety and accuracy of triage recommendations to seek hospital care, seek primary care, or manage symptoms at home.ResultsData from 281 hospital patients and 300 clinic patients were collected and analyzed. Sensitivity to emergencies was 100% (10/10 encounters). Sensitivity to urgencies was 90% (73/81) and 97% (34/35) for hospital and primary care patients, respectively. The prototype was significantly more accurate than patients at triage (73% versus 58%, p<0.01). Compliance with triage recommendations in this cohort using this iteration of the symptom checker would have reduced hospital visits by 55% but cause potential harm in 2-3% from delay in care.InterpretationThe prototype symptom checker was superior to patients in deciding the most appropriate treatment setting for medical issues. This symptom checker could reduce a significant number of unnecessary hospital visits, with accuracy and safety outcomes comparable to existing data on telephone triage.
Project description:Digital and online symptom checkers are an increasingly adopted class of health technologies that enable patients to input their symptoms and biodata to produce a set of likely diagnoses and associated triage advice. However, concerns regarding the accuracy and safety of these symptom checkers have been raised. This systematic review evaluates the accuracy of symptom checkers in providing diagnoses and appropriate triage advice. MEDLINE and Web of Science were searched for studies that used either real or simulated patients to evaluate online or digital symptom checkers. The primary outcomes were the diagnostic and triage accuracy of the symptom checkers. The QUADAS-2 tool was used to assess study quality. Of the 177 studies retrieved, 10 studies met the inclusion criteria. Researchers evaluated the accuracy of symptom checkers using a variety of medical conditions, including ophthalmological conditions, inflammatory arthritides and HIV. A total of 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low across included studies (range: 19-37.9%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8-90.1%) was typically higher than diagnostic accuracy. Overall, the diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards adopting this class of technologies across numerous health systems, this study demonstrates that reliance upon symptom checkers could pose significant patient safety hazards. Large-scale primary studies, based upon real-world data, are warranted to demonstrate the adequate performance of these technologies in a manner that is non-inferior to current best practices. Moreover, an urgent assessment of how these systems are regulated and implemented is required.
Project description:BackgroundYoung adults often browse the internet for self-triage and diagnosis. More sophisticated digital platforms such as symptom checkers have recently become pervasive; however, little is known about their use.ObjectiveThe aim of this study was to understand young adults' (18-34 years old) perspectives on the use of the Google search engine versus a symptom checker, as well as to identify the barriers and enablers for using a symptom checker for self-triage and self-diagnosis.MethodsA qualitative descriptive case study research design was used. Semistructured interviews were conducted with 24 young adults enrolled in a university in Ontario, Canada. All participants were given a clinical vignette and were asked to use a symptom checker (WebMD Symptom Checker or Babylon Health) while thinking out loud, and were asked questions regarding their experience. Interviews were audio-recorded, transcribed, and imported into the NVivo software program. Inductive thematic analysis was conducted independently by two researchers.ResultsUsing the Google search engine was perceived to be faster and more customizable (ie, ability to enter symptoms freely in the search engine) than a symptom checker; however, a symptom checker was perceived to be useful for a more personalized assessment. After having used a symptom checker, most of the participants believed that the platform needed improvement in the areas of accuracy, security and privacy, and medical jargon used. Given these limitations, most participants believed that symptom checkers could be more useful for self-triage than for self-diagnosis. Interestingly, more than half of the participants were not aware of symptom checkers prior to this study and most believed that this lack of awareness about the existence of symptom checkers hindered their use.ConclusionsAwareness related to the existence of symptom checkers and their integration into the health care system are required to maximize benefits related to these platforms. Addressing the barriers identified in this study is likely to increase the acceptance and use of symptom checkers by young adults.
Project description:BackgroundSymptom checkers are clinical decision support apps for patients, used by tens of millions of people annually. They are designed to provide diagnostic and triage advice and assist users in seeking the appropriate level of care. Little evidence is available regarding their diagnostic and triage accuracy with direct use by patients for urgent conditions.ObjectiveThe aim of this study is to determine the diagnostic and triage accuracy and usability of a symptom checker in use by patients presenting to an emergency department (ED).MethodsWe recruited a convenience sample of English-speaking patients presenting for care in an urban ED. Each consenting patient used a leading symptom checker from Ada Health before the ED evaluation. Diagnostic accuracy was evaluated by comparing the symptom checker's diagnoses and those of 3 independent emergency physicians viewing the patient-entered symptom data, with the final diagnoses from the ED evaluation. The Ada diagnoses and triage were also critiqued by the independent physicians. The patients completed a usability survey based on the Technology Acceptance Model.ResultsA total of 40 (80%) of the 50 participants approached completed the symptom checker assessment and usability survey. Their mean age was 39.3 (SD 15.9; range 18-76) years, and they were 65% (26/40) female, 68% (27/40) White, 48% (19/40) Hispanic or Latino, and 13% (5/40) Black or African American. Some cases had missing data or a lack of a clear ED diagnosis; 75% (30/40) were included in the analysis of diagnosis, and 93% (37/40) for triage. The sensitivity for at least one of the final ED diagnoses by Ada (based on its top 5 diagnoses) was 70% (95% CI 54%-86%), close to the mean sensitivity for the 3 physicians (on their top 3 diagnoses) of 68.9%. The physicians rated the Ada triage decisions as 62% (23/37) fully agree and 24% (9/37) safe but too cautious. It was rated as unsafe and too risky in 22% (8/37) of cases by at least one physician, in 14% (5/37) of cases by at least two physicians, and in 5% (2/37) of cases by all 3 physicians. Usability was rated highly; participants agreed or strongly agreed with the 7 Technology Acceptance Model usability questions with a mean score of 84.6%, although "satisfaction" and "enjoyment" were rated low.ConclusionsThis study provides preliminary evidence that a symptom checker can provide acceptable usability and diagnostic accuracy for patients with various urgent conditions. A total of 14% (5/37) of symptom checker triage recommendations were deemed unsafe and too risky by at least two physicians based on the symptoms recorded, similar to the results of studies on telephone and nurse triage. Larger studies are needed of diagnosis and triage performance with direct patient use in different clinical environments.
Project description:BackgroundOnline symptom checkers are digital health solutions that provide a differential diagnosis based on a user's symptoms. During the coronavirus disease 2019 (COVID-19) pandemic, symptom checkers have become increasingly important due to physical distance constraints and reduced access to in-person medical consultations. Furthermore, various symptom checkers specialised in the assessment of COVID-19 infection have been produced.ObjectivesAssess the correlation between COVID-19 risk assessments from an online symptom checker and current trends in COVID-19 infections. Analyse whether those correlations are reflective of various country-wise quality of life measures. Lastly, determine whether the trends found in symptom checker assessments predict or lag relative to those of the COVID-19 infections.Materials and methodsIn this study, we compile the outcomes of COVID-19 risk assessments provided by the symptom checker Symptoma (www.symptoma.com) in 18 countries with suitably large user bases. We analyse this dataset's spatial and temporal features compared to the number of newly confirmed COVID-19 cases published by the respective countries.ResultsWe find an average correlation of 0.342 between the number of Symptoma users assessed to have a high risk of a COVID-19 infection and the official COVID-19 infection numbers. Further, we show a significant relationship between that correlation and the self-reported health of a country. Lastly, we find that the symptom checker is, on average, ahead (median +3 days) of the official infection numbers for most countries.ConclusionWe show that online symptom checkers can capture the national-level trends in coronavirus infections. As such, they provide a valuable and unique information source in policymaking against pandemics, unrestricted by conventional resources.
Project description:ObjectiveTo determine the diagnostic and triage accuracy of online symptom checkers (tools that use computer algorithms to help patients with self diagnosis or self triage).DesignAudit study.SettingPublicly available, free symptom checkers.Participants23 symptom checkers that were in English and provided advice across a range of conditions. 45 standardized patient vignettes were compiled and equally divided into three categories of triage urgency: emergent care required (for example, pulmonary embolism), non-emergent care reasonable (for example, otitis media), and self care reasonable (for example, viral upper respiratory tract infection).Main outcome measuresFor symptom checkers that provided a diagnosis, our main outcomes were whether the symptom checker listed the correct diagnosis first or within the first 20 potential diagnoses (n=770 standardized patient evaluations). For symptom checkers that provided a triage recommendation, our main outcomes were whether the symptom checker correctly recommended emergent care, non-emergent care, or self care (n=532 standardized patient evaluations).ResultsThe 23 symptom checkers provided the correct diagnosis first in 34% (95% confidence interval 31% to 37%) of standardized patient evaluations, listed the correct diagnosis within the top 20 diagnoses given in 58% (55% to 62%) of standardized patient evaluations, and provided the appropriate triage advice in 57% (52% to 61%) of standardized patient evaluations. Triage performance varied by urgency of condition, with appropriate triage advice provided in 80% (95% confidence interval 75% to 86%) of emergent cases, 55% (47% to 63%) of non-emergent cases, and 33% (26% to 40%) of self care cases (P<0.001). Performance on appropriate triage advice across the 23 individual symptom checkers ranged from 33% (95% confidence interval 19% to 48%) to 78% (64% to 91%) of standardized patient evaluations.ConclusionsSymptom checkers had deficits in both triage and diagnosis. Triage advice from symptom checkers is generally risk averse, encouraging users to seek care for conditions where self care is reasonable.
Project description:Stressful life events (SLEs) have been associated with an increased risk of heavy drinking, suggesting individuals may use alcohol to cope with negative life events. However, little research has explored the extent to which SLEs have different effects on later alcohol use based on one's current alcohol use pattern. We replicated prototypical patterns of alcohol use via latent class analysis at Waves 2, 3, and 4 of the National Longitudinal Study of Adolescent to Adult Health (n = 4,569). Latent transition analysis was then used to examine the extent to which SLEs influenced the likelihood of stability or change in class membership from adolescence to early adulthood. Results suggested that adolescents were more likely to transition into different patterns of alcohol use as they entered early adulthood but were more likely to retain the same drinking pattern once in early adulthood. Among those who typically abstained, experiencing SLEs was associated with greater odds of transitioning to heavier drinking or problematic patterns of alcohol use. However, among those who had heavy or problematic alcohol use patterns, SLEs were associated with greater odds of decreasing alcohol use to either heavy or abstaining levels. Results suggest those who previously abstained may begin to use alcohol as a coping mechanism following stressful events, whereas those who drank heavily may decrease or abstain from alcohol use following life stress as a means of enacting positive life changes. The results encourage further study into factors that differentiate changes in alcohol use among light drinkers following SLEs. (PsycINFO Database Record (c) 2018 APA, all rights reserved).
Project description:ImportanceBecause more patients are presenting with self-guided research of symptoms, it is important to assess the capabilities and limitations of these available health information tools.ObjectiveTo determine the accuracy of the most popular online symptom checker for ophthalmic diagnoses.Design, setting, and participantsIn a cross-sectional study, 42 validated clinical vignettes of ophthalmic symptoms were generated and distilled to their core presenting symptoms. Cases were entered into WebMD symptom checker by both medically trained and nonmedically trained personnel blinded to the diagnosis. Output from the symptom checker, including the number of symptoms, ranking and list of diagnoses, and triage urgency were recorded. The study was conducted on October 13, 2017. Analysis was performed between October 15, 2017, and April 30, 2018.Main outcomes and measuresAccuracy of the top 3 diagnoses generated by the online symptom checker.ResultsThe mean (SD) number of symptoms entered was 3.6 (1.6) (range, 1-8). The median (SD) number of diagnoses generated by the symptom checker was 26.8 (21.8) (range, 1-99). The primary diagnosis by the symptom checker was correct in 11 of 42 (26%; 95% CI, 12%-40%) cases. The correct diagnosis was included in the online symptom checker's top 3 diagnoses in 16 of 42 (38%; 95% CI, 25%-56%) cases. The correct diagnosis was not included in the symptom checker's list in 18 of 42 (43%; 95% CI, 32%-63%) cases. Triage urgency based on the top diagnosis was appropriate in 7 of 18 (39%; 95% CI, 14%-64%) emergent cases and 21 of 24 (88%; 95% CI, 73%-100%) nonemergent cases. Interuser variability for the correct diagnosis being in the top 3 listed was at least moderate (Cohen κ = 0.74; 95% CI, 0.54-0.95).Conclusions and relevanceThe most popular online symptom checker may arrive at the correct clinical diagnosis for ophthalmic conditions, but a substantial proportion of diagnoses may not be captured. These findings suggest that further research to reflect the real-life application of internet diagnostic resources is required.
Project description:Symptom checkers are increasingly used to assess new symptoms and navigate the health care system. The aim of this study was to compare the accuracy of an artificial intelligence (AI)-based symptom checker (Ada) and physicians regarding the presence/absence of an inflammatory rheumatic disease (IRD). In this survey study, German-speaking physicians with prior rheumatology working experience were asked to determine IRD presence/absence and suggest diagnoses for 20 different real-world patient vignettes, which included only basic health and symptom-related medical history. IRD detection rate and suggested diagnoses of participants and Ada were compared to the gold standard, the final rheumatologists' diagnosis, reported on the discharge summary report. A total of 132 vignettes were completed by 33 physicians (mean rheumatology working experience 8.8 (SD 7.1) years). Ada's diagnostic accuracy (IRD) was significantly higher compared to physicians (70 vs 54%, p = 0.002) according to top diagnosis. Ada listed the correct diagnosis more often compared to physicians (54 vs 32%, p < 0.001) as top diagnosis as well as among the top 3 diagnoses (59 vs 42%, p < 0.001). Work experience was not related to suggesting the correct diagnosis or IRD status. Confined to basic health and symptom-related medical history, the diagnostic accuracy of physicians was lower compared to an AI-based symptom checker. These results highlight the potential of using symptom checkers early during the patient journey and importance of access to complete and sufficient patient information to establish a correct diagnosis.