Dataset Information

Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study.

ABSTRACT: Many research domains still heavily rely on paper-based data collection procedures, despite numerous associated drawbacks. The QuestionSys framework is intended to empower researchers as well as clinicians without programming skills to develop their own smart mobile apps in order to collect data for their specific scenarios.In order to validate the feasibility of this model-driven, end-user programming approach, we conducted a study with 80 participants.Across 2 sessions (7 days between Session 1 and Session 2), participants had to model 10 data collection instruments (5 at each session) with the developed configurator component of the framework. In this context, performance measures like the time and operations needed as well as the resulting errors were evaluated. Participants were separated into two groups (ie, novices vs experts) based on prior knowledge in process modeling, which is one fundamental pillar of the QuestionSys framework.Statistical analysis (t tests) revealed that novices showed significant learning effects for errors (P=.04), operations (P<.001), and time (P<.001) from the first to the last use of the configurator. Experts showed significant learning effects for operations (P=.001) and time (P<.001), but not for errors as the experts' errors were already very low at the first modeling of the data collection instrument. Moreover, regarding the time and operations needed, novices got significantly better at the third modeling task than experts were at the first one (t tests; P<.001 for time and P=.002 for operations). Regarding errors, novices did not get significantly better at working with any of the 10 data collection instruments than experts were at the first modeling task, but novices' error rates for all 5 data collection instruments at Session 2 were not significantly different anymore from those of experts at the first modeling task. After 7 days of not using the configurator (from Session 1 to Session 2), the experts' learning effect at the end of Session 1 remained stable at the beginning of Session 2, but the novices' learning effect at the end of Session 1 showed a significant decay at the beginning of Session 2 regarding time and operations (t tests; P<.001 for time and P=.03 for operations).In conclusion, novices were able to use the configurator properly and showed fast (but unstable) learning effects, resulting in their performances becoming as good as those of experts (which were already good) after having little experience with the configurator. Following this, researchers and clinicians can use the QuestionSys configurator to develop data collection apps for smart mobile devices on their own.

SUBMITTER: Schobel J

PROVIDER: S-EPMC6045789 | biostudies-other | 2018 Jun

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study.

Schobel Johannes J Pryss Rüdiger R Probst Thomas T Schlee Winfried W Schickler Marc M Reichert Manfred M

JMIR mHealth and uHealth 20180629 6

<h4>Background</h4>Many research domains still heavily rely on paper-based data collection procedures, despite numerous associated drawbacks. The QuestionSys framework is intended to empower researchers as well as clinicians without programming skills to develop their own smart mobile apps in order to collect data for their specific scenarios.<h4>Objective</h4>In order to validate the feasibility of this model-driven, end-user programming approach, we conducted a study with 80 participants.<h4>M ...[more]

PMID: 29959107

Similar Datasets

Project description:BackgroundThere is currently limited evidence on the level and intensity of physical activity in individuals with hemophilia A. Mobile technologies can offer a rigorous and reliable alternative to support data collection processes but they are often associated with poor user retention. The lack of longitudinal continuity in their use can be partly attributed to the insufficient consideration of stakeholder inputs in the development process of mobile apps. Several user-centered models have been proposed to guarantee that a thorough knowledge of the end user needs is considered in the development process of mobile apps.ObjectiveThe aim of this study is to design and validate an electronic patient-reported outcome mobile app that requires sustained active input by individuals during POWER, an observational study that aims at evaluating the relationship between physical activity levels and bleeding in patients with hemophilia A.MethodsWe adopted a user-centered design and engaged several stakeholders in the development and usability testing of this mobile app. During the concept generation and ideation phase, we organized a need-assessment focus group (FG) with patient representatives to elicit specific design requirements for the end users. We then conducted 2 exploratory FGs to seek additional inputs for the app's improvement and 2 confirmatory FGs to validate the app and test its usability in the field through the mobile health app usability questionnaire.ResultsThe findings from the thematic analysis of the need-assessment FG revealed that there was a demand for sense making, for simplification of app functionalities, for maximizing integration, and for minimizing the feeling of external control. Participants involved in the later stages of the design refinement contributed to improving the design further by upgrading the app's layout and making the experience with the app more efficient through functions such as chatbots and visual feedback on the number of hours a wearable device had been worn, to ensure that the observed data were actually registered. The end users rated the app highly during the quantitative assessment, with an average mobile health app usability questionnaire score of 5.32 (SD 0.66; range 4.44-6.23) and 6.20 (SD 0.43; range 5.72-6.88) out of 7 in the 2 iterative usability testing cycles.ConclusionsThe results of the usability test indicated a high, growing satisfaction with the electronic patient-reported outcome app. The adoption of a thorough user-centered design process using several types of FGs helped maximize the likelihood of sustained retention of the app's users and made it fit for data collection of relevant outcomes in the observational POWER study. The continuous use of the app and the actual level of engagement will be evaluated during the ongoing trial.Trial registrationClinicalTrials.gov NCT04165135; https://clinicaltrials.gov/ct2/show/NCT04165135.

Project description:BackgroundDelivery of major trauma care is complex and often fast paced. Clear and comprehensive documentation is paramount to support effective communication during complex clinical care episodes, and to allow collection of data for audit, research and continuous improvement. Clinical events are typically recorded on paper-based records that are developed for individual centres or systems. As one of the priorities laid out by the Scottish Trauma Network project was to develop an electronic data collection system, the TraumaApp was created as a data collection tool for major trauma that could be adopted worldwide.MethodsThe study was performed as a service evaluation based at the Queen Elizabeth University Hospital Emergency Department. Fifty staff members were recruited in pairs and listened to five paired major trauma standby and handover recordings. Participants were randomised to input data to the TraumaApp and one into the existing paper proforma. The time taken to input data add into was measured, along with time for clarifications and any errors made. Those using the app completed a System Usability Score.ResultsNo statistically significant difference was demonstrated between times taken for data entry for the digital and paper documentation, apart from the Case 5 Handover (p < 0.05). Case 1 showed a significantly higher time for clarifications and number of errors with digital data collection (p = 0.01 and p = 1.79E-05 respectively). There were no other differences between data for the app and the proforma. The mean System Usability score for this cohort was 75 out of 100, with a standard deviation of 17 (rounded to nearest integer).ConclusionDigital real-time recording of clinical events using a tool such as the TraumaApp is comparable to completion of paper proforma. The System Usability Score for the TraumaApp was above the internationally validated standard of acceptable usability. There was no evidence of improvement in use over time or familiarity, most likely due to the brevity of the assessments and the refined user interface. This would benefit from further research, exploring data completeness and a potential mixed methods approach to explore training requirements for use of the TraumaApp.

Project description:BackgroundThe Discovery Critical Care Research Network Program for Resilience and Emergency Preparedness (Discovery PREP) partnered with a third-party technology vendor to design and implement an electronic data capture tool that addressed multisite data collection challenges during public health emergencies (PHE) in the United States. The basis of the work was to design an electronic data capture tool and to prospectively gather data on usability from bedside clinicians during national health system stress queries and influenza observational studies.ObjectiveThe aim of this paper is to describe the lessons learned in the design and implementation of a novel electronic data capture tool with the goal of significantly increasing the nation's capability to manage real-time data collection and analysis during PHE.MethodsA multiyear and multiphase design approach was taken to create an electronic data capture tool, which was used to pilot rapid data capture during a simulated PHE. Following the pilot, the study team retrospectively assessed the feasibility of automating the data captured by the electronic data capture tool directly from the electronic health record. In addition to user feedback during semistructured interviews, the System Usability Scale (SUS) questionnaire was used as a basis to evaluate the usability and performance of the electronic data capture tool.ResultsParticipants included Discovery PREP physicians, their local administrators, and data collectors from tertiary-level academic medical centers at 5 different institutions. User feedback indicated that the designed system had an intuitive user interface and could be used to automate study communication tasks making for more efficient management of multisite studies. SUS questionnaire results classified the system as highly usable (SUS score 82.5/100). Automation of 17 (61%) of the 28 variables in the influenza observational study was deemed feasible during the exploration of automated versus manual data abstraction. The creation and use of the Project Meridian electronic data capture tool identified 6 key design requirements for multisite data collection, including the need for the following: (1) scalability irrespective of the type of participant; (2) a common data set across sites; (3) automated back end administrative capability (eg, reminders and a self-service status board); (4) multimedia communication pathways (eg, email and SMS text messaging); (5) interoperability and integration with local site information technology infrastructure; and (6) natural language processing to extract nondiscrete data elements.ConclusionsThe use of the electronic data capture tool in multiple multisite Discovery PREP clinical studies proved the feasibility of using the novel, cloud-based platform in practice. The lessons learned from this effort can be used to inform the improvement of ongoing global multisite data collection efforts during the COVID-19 pandemic and transform current manual data abstraction approaches into reliable, real time, and automated information exchange. Future research is needed to expand the ability to perform automated multisite data extraction during a PHE and beyond.

Project description:BackgroundThere is a need for valid approaches to measure sexual interactions to assess the impact of behavioural interventions and to predict the impact of behaviour changes. Different methods of asking about sexual behaviour often yield conflicting answers and men often report higher levels of heterosexual activity than women. To better understand self-reported sexual behaviour data and how best to collect it, we analyzed data collected as part of a larger project (ST IMPACTS) on the social and behavioural impact of introducing community-level HIV self-testing (HIVST) with counseling (semi-supervised with pre- and generic post-test counseling provided on delivery or collection of test kits) in an urban Malawian setting.MethodsInformation on sexual behaviour was collected from HIV self-testers over a three-month period. Three different methods were used: retrospective face-to-face interviews (FTFI); audio computer assisted self-interviews (ACASI) and a prospective coital diary. Both retrospective instruments were used before and after the three-month study period. Frequency and cross-tabulation, as well as scatterplots, were used for exploratory analyses. Chi-square tests were used to test for differences in proportions. Spearman's correlation coefficient was used to explore associations between both continuous and ordinal variables and Wilcoxon's paired sample and Mann-Whitney test was used to test for differences in such variables or between variables.ResultsThere was reasonable agreement between the two retrospective methods although both yielded inconsistent answers e.g. with lower reported numbers of life-time sexual partners at the end than at the beginning of the study period. The diary method elicited higher reported levels of sex with multiple partners than both retrospective instruments which may be due to inadequate recall. Over the study period 37.4% of men and 19.7% of women reported multiple sexual partners using the diary. There was no clear relationship between reported sexual behaviour and HIV status (prevalence 9.6%).ConclusionsDiaries may therefore have higher validity for sensitive behaviour reporting and thus be the preferred method in similar African contexts in measuring sexual behaviours.

Project description:BackgroundMaintaining a healthy lifestyle is important for wheelchair users' well-being, as it can have a major impact on their daily functioning. Mobile health (mHealth) apps can support a healthy lifestyle; however, these apps are not necessarily suitable for wheelchair users with spinal cord injury or lower limb amputation. Therefore, a new mHealth app (WHEELS) was developed to promote a healthy lifestyle for this population.ObjectiveThe objectives of this study were to develop the WHEELS mHealth app, and explore its usability, feasibility, and effectiveness.MethodsThe WHEELS app was developed using the intervention mapping framework. Intervention goals were determined based on a needs assessment, after which behavior change strategies were selected to achieve these goals. These were applied in an app that was pretested on ease of use and satisfaction, followed by minor adjustments. Subsequently, a 12-week pre-post pilot study was performed to explore usability, feasibility, and effectiveness of the app. Participants received either a remote-guided or stand-alone intervention. Responses to semistructured interviews were analyzed using content analysis, and questionnaires (System Usability Score [SUS], and Usefulness, Satisfaction, and Ease) were administered to investigate usability and feasibility. Effectiveness was determined by measuring outcomes on physical activity, nutrition, sleep quality (Pittsburgh Sleep Quality Index), body composition, and other secondary outcomes pre and post intervention, and by calculating effect sizes (Hedges g).ResultsSixteen behavior change strategies were built into an app to change the physical activity, dietary, sleep, and relaxation behaviors of wheelchair users. Of the 21 participants included in the pilot study, 14 participants completed the study. The interviews and questionnaires showed a varied user experience. Participants scored a mean of 58.6 (SD 25.2) on the SUS questionnaire, 5.4 (SD 3.1) on ease of use, 5.2 (SD 3.1) on satisfaction, and 5.9 (3.7) on ease of learning. Positive developments in body composition were found on waist circumference (P=.02, g=0.76), fat mass percentage (P=.004, g=0.97), and fat-free mass percentage (P=.004, g=0.97). Positive trends were found in body mass (P=.09, g=0.49), BMI (P=.07, g=0.53), daily grams of fat consumed (P=.07, g=0.56), and sleep quality score (P=.06, g=0.57).ConclusionsThe WHEELS mHealth app was successfully developed. The interview outcomes and usability scores are reasonable. Although there is room for improvement, the current app showed promising results and seems feasible to deploy on a larger scale.

Dataset Information

Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study.

Publications

Learnability of a Configurator Empowering End Users to Create Mobile Data Collection Instruments: Usability Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets