Dataset Information

An application for classifying perceptions on my health bank in Taiwan using convolutional neural networks and web-based computerized adaptive testing: A development and usability study.

ABSTRACT:

Background

The classification of a respondent's opinions online into positive and negative classes using a minimal number of questions is gradually changing and helps turn techniques into practices. A survey incorporating convolutional neural networks (CNNs) into web-based computerized adaptive testing (CAT) was used to collect perceptions on My Health Bank (MHB) from users in Taiwan. This study designed an online module to accurately and efficiently turn a respondent's perceptions into positive and negative classes using CNNs and web-based CAT.

Methods

In all, 640 patients, family members, and caregivers with ages ranging from 20 to 70 years who were registered MHB users were invited to complete a 3-domain, 26-item, 5-category questionnaire asking about their perceptions on MHB (PMHB26) in 2019. The CNN algorithm and k-means clustering were used for dividing respondents into 2 classes of unsatisfied and satisfied classes and building a PMHB26 predictive model to estimate parameters. Exploratory factor analysis, the Rasch model, and descriptive statistics were used to examine the demographic characteristics and PMHB26 factors that were suitable for use in CNNs and Rasch multidimensional CAT (MCAT). An application was then designed to classify MHB perceptions.

Results

We found that 3 construct factors were extracted from PMHB26. The reliability of PMHB26 for each subscale beyond 0.94 was evident based on internal consistency and stability in the data. We further found the following: the accuracy of PMHB26 with CNN yields a higher accuracy rate (0.98) with an area under the curve of 0.98 (95% confidence interval, 0.97-0.99) based on the 391 returned questionnaires; and for the efficiency, approximately one-third of the items were not necessary to answer in reducing the respondents' burdens using Rasch MCAT.

Conclusions

The PMHB26 CNN model, combined with the Rasch online MCAT, is recommended for improving the accuracy and efficiency of classifying patients' perceptions of MHB utility. An application developed for helping respondents self-assess the MHB cocreation of value can be applied to other surveys in the future.

SUBMITTER: Hsu CF

PROVIDER: S-EPMC8718177 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundWorkplace bullying has been measured in many studies to investigate its effects on mental health issues. However, none have used web-based computerized adaptive testing (CAT) with bully classifications and convolutional neural networks (CNN) for reporting the extent of individual bullying in the workplace.ObjectiveThis study aims to build a model using CNN to develop an app for automatic detection and classification of nurse bullying-levels, incorporated with online Rasch computerized adaptive testing, to help assess nurse bullying at an earlier stage.MethodsWe recruited 960 nurses working in a Taiwan Ch-Mei hospital group to fill out the 22-item Negative Acts Questionnaire-Revised (NAQ-R) in August 2012. The k-mean and the CNN were used as unsupervised and supervised learnings, respectively, for: (1) dividing nurses into three classes (n=918, 29, and 13 with suspicious mild, moderate, and severe extent of being bullied, respectively); and (2) building a bully prediction model to estimate 69 different parameters. Finally, data were separated into training and testing sets in a proportion of 70:30, where the former was used to predict the latter. We calculated the sensitivity, specificity, and receiver operating characteristic curve (area under the curve [AUC]), along with the accuracy across studies for comparison. An app predicting the respondent bullying-level was developed, involving the model's 69 estimated parameters and the online Rasch CAT module as a website assessment.ResultsWe observed that: (1) the 22-item model yields higher accuracy rates for three categories, with an accuracy of 94% for the total 960 cases, and accuracies of 99% (AUC 0.99; 95% CI 0.99-1.00) and 83% (AUC 0.94; 95% CI 0.82-0.99) for the lower and upper groups (cutoff points at 49 and 66 points) based on the 947 cases and 42 cases, respectively; and (2) the 700-case training set, with 95% accuracy, predicts the 260-case testing set reaching an accuracy of 97. Thus, a NAQ-R app for nurses that predicts bullying-level was successfully developed and demonstrated in this study.ConclusionsThe 22-item CNN model, combined with the Rasch online CAT, is recommended for improving the accuracy of the nurse NAQ-R assessment. An app developed for helping nurses self-assess workplace bullying at an early stage is required for application in the future.

Project description:BackgroundSupporting patients to self-manage their chronic kidney disease (CKD) has been identified as a research priority by patients with CKD and those who care for them. Self-management has been shown to slow CKD progression and improve the quality of life of individuals living with the disease. Previous work has identified a need for a person-centered, theory-informed, web-based tool for CKD self-management that can be individualized to a patient's unique situation, priorities, and preferences. We addressed this gap using an integrated knowledge translation method and patient engagement principles.ObjectiveThe aim of this study is to conduct systematic co-design and usability testing of a web-based self-management prototype for adults with CKD (nondialysis and nontransplant) and their caregivers to enhance self-management support.MethodsA multistep, iterative system development cycle was used to co-design and test the My Kidneys My Health prototype. The 3-step process included creating website features and content using 2 sequential focus groups with patients with CKD and caregivers, heuristic testing using the 10 heuristic principles by Nielsen, and usability testing through in-person 60-minute interviews with patients with CKD and their caregivers. Patients with CKD, caregivers, clinicians, researchers, software developers, graphic designers, and policy makers were involved in all steps of this study.ResultsIn step 1, 18 participants (14 patients and 4 caregivers) attended one of the 2 sequential focus groups. The participants provided specific suggestions for simplifying navigation as well as suggestions to incorporate video, text, audio, interactive components, and visuals to convey information. A total of 5 reviewers completed the heuristic analysis (step 2), identifying items mainly related to navigation and functionality. Furthermore, 5 participants completed usability testing (step 3) and provided feedback on video production, navigation, features and functionality, and branding. Participants reported visiting the website repeatedly for the following features: personalized food tool, my health care provider question list, symptom guidance based on CKD severity, and medication advice. Usability was high, with a mean system usability score of 90 out of 100.ConclusionsThe My Kidneys My Health prototype is a systematically developed, multifaceted, web-based CKD self-management support tool guided by the theory and preferences of patients with CKD and their caregivers. The website is user friendly and provides features that improve user experience by tailoring the content and resources to their needs. A feasibility study will provide insights into the acceptability of and engagement with the prototype and identify preliminary patient-reported outcomes (eg, self-efficacy) as well as potential factors related to implementation. This work is relevant given the shift to virtual care during the current pandemic times and provides patients with support when in-person care is restricted.

Project description:BackgroundA glaucoma-specific health-related quality of life (HRQoL) item bank (IB) and computerized adaptive testing (CAT) system relevant to Asian populations is not currently available. We aimed to develop content for an IB focusing on HRQoL domains important to Asian people with glaucoma; and to compare the content coverage of our new instrument with established glaucoma-specific instruments.MethodsIn this qualitative study of glaucoma patients recruited from the Singapore National Eye Centre (November 2018-November 2019), items/domains were generated from: (1) glaucoma-specific questionnaires; (2) published articles; (3) focus groups/semi-structured interviews with glaucoma patients (n = 27); and (4) feedback from glaucoma experts. Data were analyzed using the constant comparative method. Items were systematically refined to a concise set, and pre-tested using cognitive interviews with 27 additional glaucoma patients.ResultsOf the 54 patients (mean ± standard deviation [SD] age 66.9 ± 9.8; 53.7% male), 67 (62.0%), 30 (27.8%), and 11 (10.2%) eyes had primary open angle glaucoma, angle closure glaucoma, and no glaucoma respectively. Eighteen (33.3%), 11 (20.4%), 8 (14.8%), 12 (22.2%), and 5 (9.3%) patients had no, mild, moderate, severe, or advanced/end-stage glaucoma (better eye), respectively. Initially, 311 items within nine HRQoL domains were identified: Visual Symptoms, Ocular Comfort Symptoms, Activity Limitation, Driving, Lighting, Mobility, Psychosocial, Glaucoma management, and Work; however, Driving and Visual Symptoms were subsequently removed during the refinement process. During cognitive interviews, 12, 23 and 10 items were added, dropped and modified, respectively.ConclusionFollowing a rigorous process, we developed a 221-item, 7-domain Asian glaucoma-specific IB. Once operationalised using CAT, this new instrument will enable precise, rapid, and comprehensive assessment of the HRQoL impact of glaucoma and associated treatment efficacy.