Dataset Information

Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study.

ABSTRACT: BACKGROUND:Medical coding is essential for standardized communication and integration of clinical data. The Unified Medical Language System by the National Library of Medicine is the largest clinical terminology system for medical coders and Natural Language Processing tools. However, the abundance of ambiguous codes leads to low rates of uniform coding among different coders. OBJECTIVE:The objective of our study was to measure uniform coding among different medical experts in terms of interrater reliability and analyze the effect on interrater reliability using an expert- and Web-based code suggestion system. METHODS:We conducted a quasi-experimental study in which 6 medical experts coded 602 medical items from structured quality assurance forms or free-text eligibility criteria of 20 different clinical trials. The medical item content was selected on the basis of mortality-leading diseases according to World Health Organization data. The intervention comprised using a semiautomatic code suggestion tool that is linked to a European information infrastructure providing a large medical text corpus of >300,000 medical form items with expert-assigned semantic codes. Krippendorff alpha (Kalpha) with bootstrap analysis was used for the interrater reliability analysis, and coding times were measured before and after the intervention. RESULTS:The intervention improved interrater reliability in structured quality assurance form items (from Kalpha=0.50, 95% CI 0.43-0.57 to Kalpha=0.62 95% CI 0.55-0.69) and free-text eligibility criteria (from Kalpha=0.19, 95% CI 0.14-0.24 to Kalpha=0.43, 95% CI 0.37-0.50) while preserving or slightly reducing the mean coding time per item for all 6 coders. Regardless of the intervention, precoordination and structured items were associated with significantly high interrater reliability, but the proportion of items that were precoordinated significantly increased after intervention (eligibility criteria: OR 4.92, 95% CI 2.78-8.72; quality assurance: OR 1.96, 95% CI 1.19-3.25). CONCLUSIONS:The Web-based code suggestion mechanism improved interrater reliability toward moderate or even substantial intercoder agreement. Precoordination and the use of structured versus free-text data elements are key drivers of higher interrater reliability.

SUBMITTER: Varghese J

PROVIDER: S-EPMC6231825 | biostudies-other | 2018 Oct

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study.

Varghese Julian J Sandmann Sarah S Dugas Martin M

Journal of medical Internet research 20181015 10

<h4>Background</h4>Medical coding is essential for standardized communication and integration of clinical data. The Unified Medical Language System by the National Library of Medicine is the largest clinical terminology system for medical coders and Natural Language Processing tools. However, the abundance of ambiguous codes leads to low rates of uniform coding among different coders.<h4>Objective</h4>The objective of our study was to measure uniform coding among different medical experts in ter ...[more]

PMID: 30322834

Similar Datasets

Project description:BACKGROUND:Apolipoprotein E (APOE) ?4 allele is a major genetic risk factor for Alzheimer disease and mild cognitive impairment (MCI). Computer-based training programs can improve cognitive performance in elderly populations. However, the effects of computer-based interventions on MCI APOE ?4 carriers have never been studied before. OBJECTIVE:The effects of different web-based interventions and the APOE isoform-specific differences in training outcomes are investigated. METHODS:Using a quasi-experimental study design, 202 participants with MCI aged 60 years and older took part in three different intervention programs (physical and cognitive [Long-Lasting Memories, or LLM], cognitive [Active Control, or AC], or physical intervention [Physical Training Control, or PTC]) via an innovative information and communication technologies exergaming platform. Participants in each interventional group were subdivided into APOE ?4 carriers and non-APOE ?4 carriers. All participants underwent an extensive neuropsychological evaluation before and after the training, blood tests, and brain imaging. RESULTS:All interventions resulted in multiple statistically significant cognitive benefits after the intervention. Verbal learning (California Verbal Learning Test: immediate recall test score-LLM: P=.04; AC: P<.001), working memory (digit span forward and backward test scores-AC: P=.03; PTC: P=.02 and P=.006, respectively), and long-term memory (California Verbal Learning Test: delayed recall test score-LLM: P=.02; AC: P=.002; and PTC: P=.02) were improved. There was no statistically significant difference among the intervention effects. APOE ?4 presence moderates intervention effects as the LLM intervention improved only their task-switching processing speed (Trail Making Test, Part B: P=.03) and the PTC intervention improved only the working memory (digit span backward: P=.03). No significant performance alteration was noted for the APOE ?4+ cognitive AC training group. CONCLUSIONS:None of the applied interventions could be identified as the optimal one; it is suggested, however, that combined cognitive and physical training and physical training via exergaming may be more effective for the high-risk MCI ?POE ?4+ subgroup.

Project description:BackgroundResilience is a person's mental ability to deal with challenging situations adaptively and is a crucial stress management skill. Psychological resilience and finding ways to cope in crises is a highly relevant topic considering the COVID-19 pandemic, which enforced quarantine, social distancing measures, and school closures worldwide. Parents and children are currently living with increased stress due to COVID-19. We need to respond with immediate ways to strengthen children's resilience. Internet-based cognitive behavioral therapy interventions for children's stress management overcome accessibility issues such as the inability to visit mental health experts owing to COVID-19 movement restrictions. An interactive learning environment was created, based on the preventive program "Friends," to overcome accessibility issues associated with delivering cognitive behavioral therapy-based interventions in formal and informal education settings.ObjectiveThis study aimed to examine the effectiveness of a web-based learning environment on resilience in (1) reducing anxiety symptoms and (2) increasing emotion recognition and recognition of stress management techniques among 9-10-year-old children. We also aimed to evaluate the learning environment's usability.MethodsA quasi-experimental pretest-posttest control group design was used. In total, 20 fourth graders in the experimental group interacted with the learning environment over 6 weekly 80-minute sessions. Further, 21 fourth graders constituted the control group. The main data sources were (1) a psychometric tool to measure children's anxiety symptoms, namely the Greek translation of the original Spence Children's Anxiety Scale, (2) 3 open-ended questions assessing emotion recognition and recognition of stress management techniques, and (3) the System Usability Scale to measure the usability of the learning environment.ResultsIn both groups, there was a small but nonsignificant postintervention reduction in reported anxiety symptoms, except for obsessive-compulsive disorder symptoms in the experimental group. A paired samples t test revealed that students' reported symptom scores of obsessive-compulsive disorder significantly decreased from 1.06 (SD 0.68) to 0.76 (SD 0.61) (t19= 5.16; P=.01). The experimental group revealed a significant increase in emotion recognition (t19=-6.99; P<.001), identification of somatic symptoms of stress (t19=-7.31; P<.001), and identification of stress management techniques (t19=-6.85; P<.001). The learning environment received a satisfactory usability score. The raw average system usability score was 76.75 (SD 8.28), which is in the 80th percentile rank and corresponds to grade B.ConclusionsThis study shows that interactive learning environments might deliver resilience interventions in an accessible and cost-effective manner in formal education, potentially even in distance-learning conditions owing to the COVID-19 pandemic. Interactive learning environments on resilience are also valuable tools for parents who can use them with their children at home, for informal learning, using mobile devices. As such, they could be a promising first-step, low-intensity intervention that children and the youth can easily access.

Project description:BackgroundDespite the availability of evidence-based guidelines for conservative treatment of osteoarthritis (OA), management is often confined to the use of analgesics and waiting for eventual total joint replacement. This suggests a gap in knowledge for persons with OA regarding the many different treatments available to them.ObjectiveOur objective was to evaluate outcomes after usage of a Web-based resource called My Joint Pain that contains tailored, evidence-based information and tools aimed to improve self-management of OA on self-management and change in knowledge.MethodsA quasi-experimental design was used to evaluate the My Joint Pain website intervention over a 12-month period. The intervention provided participants with general and user-specific information, monthly assessments with validated instruments, and progress-tracking tools. A nationwide convenience sample of 195 participants with self-assessed hip and/or knee OA completed both baseline and 12-month questionnaires (users: n=104; nonusers: n=91). The primary outcome measure was the Health Evaluation Impact Questionnaire (heiQ) to evaluate 8 different domains (health-directed activity, positive and active engagement in life, emotional distress, self-monitoring and insight, constructive attitudes and approaches, skill and technique acquisition, social integration and support, health service navigation) and the secondary outcome measure was the 17-item Osteoarthritis Quality Indicator (OAQI) questionnaire to evaluate the change in appropriateness of care received by participants. Independent t tests were used to compare changes between groups for the heiQ and chi-square tests to identify changes within and between groups from baseline to 12 months for each OAQI item.ResultsBaseline demographics between groups were similar for gender (152/195, 77.9% female), age (mean 60, SD 9 years) and body mass index (mean 31.1, SD 6.8 kg/m(2)). With the exception of health service navigation, mean effect sizes from all other heiQ domains showed a positive trend for My Joint Pain users compared to the nonusers, although the differences between groups did not reach statistical significance. Within-group changes also showed improvements among the users of the My Joint Pain website for self-management (absolute change score=15%, P=.03), lifestyle (absolute change score=16%, P=.02), and physical activity (absolute change score=11%, P=.04), with no significant improvements for the nonusers. Following 12 months of exposure to the website, there were significant improvements for users compared to nonusers in self-management (absolute change score 15% vs 2%, P=.001) and weight reduction (absolute change scores 3% vs -6%, P=.03) measured on the OAQI.ConclusionsThe My Joint Pain Web resource does not significantly improve overall heiQ, but does improve other important aspects of quality of care in people with hip and/or knee OA. Further work is required to improve engagement with the website and the quality of information delivered in order to provide a greater impact.

Project description:Objective: To explore the effect of real-world fear on risky decision-making under certainty and uncertainty. Methods: This quasi-experimental study enrolled non-psychology undergraduate volunteers aged between 17 and 20 years old from the Preventive Medical Institute medical school in Xi'an. Participants were randomly divided into two groups, and each group received a two-stage crossover design intervention (of a calm and fearful situation) and completed the tasks of risky decision-making under uncertainty (the balloon analog risk task: BART) and certainty (the Cambridge gambling task: CGT), respectively. The primary outcomes included the behavioral impulsivity measured by the BART value, and the speed of decision-making, the quality of decisions, the adventure index, behavioral impulsivity, and risk adjustment measured by CGT. The secondary outcome was the concentration of cortisol in the saliva. Results: A total of 60 questionnaires and data were obtained from 60 participants (28 males and 32 females, aged 19.55 ± 0.75). Compared with the calm situation, participants were more likely to have a lower BART value (p = 0.013), slower speed of decision-making (p < 0.05), and higher adventure index (p = 0.018) in the fearful situation. The quality of decisions (p = 0.189), behavioral impulsivity index (p = 0.182), and risk adjustment (p = 0.063) between subjects in the fearful and calm situations were comparable. Furthermore, the mean value of the adventure index of CGT in male subjects was significantly higher than that in female subjects (p < 0.05), and the cortisol concentration in saliva during the fearful situation was significantly higher compared to the calm situation (p < 0.05). Conclusion: Fear might reduce behavioral impulsivity under uncertainty, and increase the adventure index under certainty in risky decision-making. Risky behavior might be influenced by gender: under certainty in risky decision-making, men were more adventurous. Additionally, fear increased the secretion of cortisol in saliva.

Project description:BACKGROUND:Web-based interventions are promising tools for increasing the understanding of illness and treatment among patients with serious mental disorders. OBJECTIVE:This study aimed to test the feasibility and acceptability of a Web-based patient education intervention using a quasi-experimental cluster design to report feedback on patient education sessions and the website used and to report preliminary evidence of the intervention's impact on patients with schizophrenia spectrum disorder. METHODS:A single-blind, parallel, quasi-experimental cluster study over a 6-month period comparing Web-based education (n=33) with a nonequivalent control group (treatment as usual, n=24) for people with schizophrenia spectrum disorder was conducted. Participants (N=57) were recruited from one psychiatric hospital (6 wards). Feasibility was assessed by participants' commitment (refusal rate, dropout rate) to the study. Acceptability was assessed as participants' commitment to the intervention. Patient education sessions and website feedback were assessed by the patients and health care professionals. The preliminary impact of the sessions on patients' self-efficacy, self-esteem, illness cognition, and knowledge level was measured at baseline and follow-ups (8 weeks, 6 months) with self-rated questionnaires. RESULTS:The refusal rate among patients was high with no statistically significant difference (69% [74/107] in the intervention group, 76% [76/100] in the control group; P=.21). The same result was found for the dropout rates (48% [16/33] vs 58% [14/24]; P=.46). The acceptability of the intervention was good; 31 participants out of 33 (94%) completed all five sessions. Feedback on the intervention was mainly positive; three out of four subscales of session were rated above the midpoint of 4.0. Feedback on the website was also positive, with a grade of good for content (69%, 20/29 patients; 75%, 21/28 professionals), layout (62%, 18/29 patients; 61%, 17/28 professionals), and usability (62%, 18/29 patients; and 68%, 19/28 professionals). The patients using the intervention had significantly higher scores 6 months after the sessions in self-efficacy (baseline mean 26.12, SD 5.64 vs 6-month mean 29.24, SD 6.05; P=.003) and regarding knowledge level about schizophrenia (mean 11.39, SD 4.65 vs 6-month mean 15.06, SD 5.26; P=.002), and lower scores in the subscale of helplessness in illness cognition (mean 2.26, SD 0.96 vs 6-month mean 1.85, SD 0.59; P=.03). Differences from the control group were not significant. No differences were found in patients' self-esteem or other subscales in illness cognition. CONCLUSIONS:The patients were reluctant to participate in the study and tended to drop out before the follow-ups. Once they had participated, their acceptance of the intervention was high. A more effective recruitment strategy and monitoring method will be needed in future studies. To assess the impact of the intervention, a more rigorous study design with an adequately powered sample size will be used in cooperation with outpatient mental health services.

Project description:BackgroundCancer is a disease that predominantly affects older adults, and several organizations recommend the completion of a geriatric assessment to help with cancer treatment decision-making. Owing to a shortage of geriatric teams and the vast number of older adults diagnosed with cancer each year, a web-based geriatric assessment may improve access to geriatric assessment for older adults. We systematically reviewed the literature to obtain the latest evidence for the design of our web-based geriatric assessment tool Comprehensive Health Assessment for My Plan.ObjectiveThis review aimed to probe the following questions: what is the impact of providing health test results to older adults in a web-based environment without the presence of a health care provider for patient-centered outcomes, including satisfaction, perceived harm, empowerment, quality of life, and health care use (eg, hospitalization, physician visits, emergency room visits, and costs), and what recommendations do older adults and developers have for designing future apps or websites for older adults?MethodsThis systematic review was guided by the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-analysis) statement. Studies were limited to publications in English that examined a web-based tool that provided test results to older adults (aged ≥65 years) without the presence of a health care provider. A health sciences librarian performed the search on November 29, 2019, on the following electronic databases: MEDLINE, Embase, CINAHL, PsycINFO, and the Cochrane Library. The quality of the included studies was assessed using the Mixed Methods Appraisal Tool Version 2018. The findings are summarized narratively and in tabular format.ResultsA total of 26,898 titles and abstracts were screened by 2 independent reviewers, of which 94 studies were selected for a full-text review, and 9 studies were included in this review. There were only 2 randomized controlled trials of high quality that explored the effects of receiving health care results on the web via eHealth tools for older adults or provided evidence-based recommendations for designing such tools. Older adults were generally satisfied with receiving screening results via eHealth tools, and several studies suggested that receiving health screening results electronically improved participants' quality of life. However, user interfaces that were not designed with older adults in mind and older adults' lack of confidence in navigating eHealth tools proved challenging to eHealth uptake and use. All 9 studies included in this systematic review made recommendations on how to design eHealth tools that are intuitive and useful for older adults.ConclusionseHealth tools should incorporate specific elements to ensure usability for older adults. However, more research is required to fully elucidate the impact of receiving screening and results via eHealth tools without the presence of a health care provider for patient-centered outcomes in this target population.

Project description:BACKGROUND:The internet allows patients to easily look for health information. However, how Chinese patients with breast cancer use the internet has rarely been investigated, and there is a scarcity of information about the influence of internet use on survival. OBJECTIVE:This observational study aimed to investigate the details of online medical information searching by Chinese patients with breast cancer and to determine whether internet use has any survival benefits. METHODS:Patients who were diagnosed with invasive breast cancer at Peking Union Medical College Hospital between January 2014 and December 2015 were enrolled. We obtained information on their internet-searching behavior and gathered data from the patients' medical and follow-up records. The associations between internet use and other clinic-pathological factors were analyzed. A Cox proportional-hazards model and the Kaplan-Meier method were used for disease-free survival (DFS) analyses. RESULTS:A total of 973 patients with invasive breast cancer who underwent definitive surgery took part in the study. Among them, 477 cases (49.0%) performed web-based breast cancer information searching before the initial treatment. A multivariate logistic regression analysis suggested that web-based breast cancer information searching was significantly associated with younger age (odds ratio [OR] 0.95, 95% CI 0.94-0.97, P<.001), higher education level (OR 1.37, 95% CI 1.01-1.86, P=.04), and breast conserving surgery (OR 1.35, 95% CI 1.04-1.77, P=.03). Baidu (73.4%, 350/477) and WeChat (66.7%, 318/477) were the two most popular online information sources for breast cancer; however, only 44.9% (214/477) felt satisfied with the online information. In contrast to the nonweb searching group, the web-using patients who were satisfied with online information showed significantly improved DFS (hazard ratio 0.26; 95% CI 0.08-0.88, P=.03). CONCLUSIONS:The patients who were most likely to search the internet for breast cancer information were younger and well-educated, and they were more likely to have breast conserving therapy. Web-using patients who were satisfied with the internet information showed significantly improved DFS. Patients should browse credible websites offering accurate and updated information, and website developers should provide high-quality and easy-to-understand information to better meet the needs of patients with breast cancer.

Dataset Information

Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study.

Publications

Web-Based Information Infrastructure Increases the Interrater Reliability of Medical Coders: Quasi-Experimental Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets