Dataset Information

A Tool for Rating the Value of Health Education Mobile Apps to Enhance Student Learning (MARuL): Development and Usability Study.

ABSTRACT:

Background

To realize the potential for mobile learning in clinical skills acquisition, medical students and their teachers should be able to evaluate the value of an app to support student learning of clinical skills. To our knowledge, there is currently no rubric for evaluation of quality or value that is specific for apps to support medical student learning. Such a rubric might assist students to be more confident in using apps to support their learning.

Objective

The objective of this study was to develop an instrument that can be used by health professional educators to rate the value of a mobile app to support health professional student learning.

Methods

Using the literature, we developed a list of potential criteria for the evaluation of educational app value, which were then refined with a student group using a modified nominal group technique. The refined list was organized into themes, and the initial rubric, Mobile App Rubric for Learning (MARuL, version 1), was developed. iOS and Android app stores were searched for clinical skills apps that met our inclusion criteria. After the 2 reviewers were trained and the item descriptions were refined (version 2), a random sample of 10 included apps, 5 for each mobile operating system, was reviewed. Interitem and interrater analyses and discussions with the reviewers resulted in refinement of MARuL to version 3. The reviewers completed a review of 41 clinical skills mobile apps, and a second round of interitem and interrater reliability testing was performed, leading to version 4 of the MARuL.

Results

Students identified 28 items (from an initial set of 144 possible items) during the nominal group phase, and these were then grouped into 4 themes: teaching and learning, user centered, professional, and usability. Testing and refinement with reviewers reduced the list to 26 items. Internal consistency for MARuL was excellent (α=.96), and the interrater reliability as measured by the intraclass correlation coefficient (ICC) was good (ICC=0.66).

Conclusions

MARuL offers a fast and user-friendly method for teachers to select valuable apps to enhance student learning.

SUBMITTER: Gladman T

PROVIDER: S-EPMC7428912 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMobile devices can provide extendable learning environments in higher education and motivate students to engage in adaptive and collaborative learning. Developers must design mobile apps that are practical, effective, and easy to use, and usability testing is essential for understanding how mobile apps meet users' needs. No previous reviews have investigated the usability of mobile apps developed for health care education.ObjectiveThe aim of this scoping review is to identify usability methods and attributes in usability studies of mobile apps for health care education.MethodsA comprehensive search was carried out in 10 databases, reference lists, and gray literature. Studies were included if they dealt with health care students and usability of mobile apps for learning. Frequencies and percentages were used to present the nominal data, together with tables and graphical illustrations. Examples include a figure of the study selection process, an illustration of the frequency of inquiry usability evaluation and data collection methods, and an overview of the distribution of the identified usability attributes. We followed the Arksey and O'Malley framework for scoping reviews.ResultsOur scoping review collated 88 articles involving 98 studies, mainly related to medical and nursing students. The studies were conducted from 22 countries and were published between 2008 and 2021. Field testing was the main usability experiment used, and the usability evaluation methods were either inquiry-based or based on user testing. Inquiry methods were predominantly used: 1-group design (46/98, 47%), control group design (12/98, 12%), randomized controlled trials (12/98, 12%), mixed methods (12/98, 12%), and qualitative methods (11/98, 11%). User testing methods applied were all think aloud (5/98, 5%). A total of 17 usability attributes were identified; of these, satisfaction, usefulness, ease of use, learning performance, and learnability were reported most frequently. The most frequently used data collection method was questionnaires (83/98, 85%), but only 19% (19/98) of studies used a psychometrically tested usability questionnaire. Other data collection methods included focus group interviews, knowledge and task performance testing, and user data collected from apps, interviews, written qualitative reflections, and observations. Most of the included studies used more than one data collection method.ConclusionsExperimental designs were the most commonly used methods for evaluating usability, and most studies used field testing. Questionnaires were frequently used for data collection, although few studies used psychometrically tested questionnaires. The usability attributes identified most often were satisfaction, usefulness, and ease of use. The results indicate that combining different usability evaluation methods, incorporating both subjective and objective usability measures, and specifying which usability attributes to test seem advantageous. The results can support the planning and conduct of future usability studies for the advancement of mobile learning apps in health care education.International registered report identifier (irrid)RR2-10.2196/19072.

Project description:BACKGROUND:E-learning technologies, including mobile apps, are used to a large extent in health care education. Mobile apps can provide extendable learning environments and motivate students for adaptive and collaborative learning outside the classroom context. Developers should design practical, effective, and easy-to-use mobile apps. Usability testing is an important part of app development in order to understand if apps meet the needs of users. OBJECTIVE:The aim of this study is to perform a scoping review of usability methods and attributes reported in usability studies of mobile apps for health care education. METHODS:The scoping review is guided by the methodological framework developed by Arksey & O'Malley and further developed by Levac et al and Kahlil et al. The stages we will follow are as follows: (1) identifying the research question; (2) identifying relevant studies; (3) selecting studies; (4) charting the data; and (5) summarizing and reporting the results. We have developed two research questions to meet the aim of the study, which are as follows: (1) What usability methods are used to evaluate the usability of mobile apps for health care education? and (2) What usability attributes are reported in the usability studies of mobile apps for health care education? We will apply a comprehensive search of the literature, including 10 databases, a reference search, and a search for grey literature. Two review authors will independently screen articles for eligibility. RESULTS:The initial electronic database searches were completed in March 2019. The literature search identified 14,297 unique references. Following title and abstract screening, the full texts of 369 records were obtained. The scoping review is expected to be completed in spring 2021. CONCLUSIONS:We expect the overview of usability methods and attributes reported in usability studies of mobile apps for health care education to contribute to the knowledge base for researchers and developers. It will give an overview of the research field and provide researchers and developers with relevant and important information on the usability research area, including highlighting possible research gaps. INTERNATIONAL REGISTERED REPORT IDENTIFIER (IRRID):DERR1-10.2196/19072.

Project description:BACKGROUND:Chronic rheumatic diseases need long-term treatment and professional supervision. Mobile apps promise to improve the lives of patients and physicians. In routine practice, however, rheumatology apps are largely unknown and little is known about their quality and safety. OBJECTIVE:The aim of this study was to provide an overview of mobile rheumatology apps currently available in German app stores, evaluate app quality using the Mobile Application Rating Scale (MARS), and compile brief, ready-to-use descriptions for patients and rheumatologists. METHODS:The German App Store and Google Play store were systematically searched to identify German rheumatology mobile apps for patient and physician use. MARS was used to independently assess app quality by 8 physicians, 4 using Android and 4 using iOS smartphones. Apps were randomly assigned so that 4 apps were rated by all raters and the remaining apps were rated by two Android and two iOS users. Furthermore, brief app descriptions including app developers, app categories, and features were compiled to inform potential users and developers. RESULTS:In total, 128 and 63 apps were identified in the German Google Play and App Store, respectively. After removing duplicates and only including apps that were available in both stores, 28 apps remained. Sixteen apps met the inclusion criteria, which were (1) German language, (2) availability in both app stores, (3) targeting patients or physicians as users, and (4) clearly including rheumatology or rheumatic diseases as subject matter. Exclusion criteria were (1) congress apps and (2) company apps with advertisements. Nine apps addressed patients and 7 apps addressed physicians. No clinical studies to support the effectiveness and safety of apps could be found. Pharmaceutical companies were the main developers of two apps. Rheuma Auszeit was the only app mainly developed by a patient organization. This app had the highest overall MARS score (4.19/5). Three out of 9 patient apps featured validated questionnaires. The median overall MARS score was 3.85/5, ranging from 2.81/5 to 4.19/5. One patient-targeted and one physician-targeted app had MARS scores >4/5. No significant rater gender or platform (iOS/Android) differences could be observed. The overall correlation between app store ratings and MARS scores was low and inconsistent between platforms. CONCLUSIONS:To our knowledge, this is the first study that systematically identified and evaluated mobile apps in rheumatology for patients and physicians available in German app stores. We found a lack of supporting clinical studies, use of validated questionnaires, and involvement of academic developers. Overall app quality was heterogeneous. To create high-quality apps, closer cooperation led by patients and physicians is vital.

Project description:BackgroundSelecting and integrating health-related apps into patient care is impeded by the absence of objective guidelines for identifying high-quality apps from the many thousands now available.ObjectiveThis study aimed to evaluate the App Rating Inventory, which was developed by the Defense Health Agency's Connected Health branch, to support clinical decisions regarding app selection and evaluate medical and behavioral apps.MethodsTo enhance the tool's performance, eliminate item redundancy, reduce scoring system subjectivity, and ensure a broad application of App Rating Inventory-derived results, inventory development included 3 rounds of validation testing and 2 trial periods conducted over a 6-month interval. The development focused on content validity testing, dimensionality (ie, whether the tool's criteria performed as operationalized), factor and commonality analysis, and interrater reliability (reliability scores improved from 0.62 to 0.95 over the course of development).ResultsThe development phase culminated in a review of 248 apps for a total of 6944 data points and a final 28-item, 3-category app rating system. The App Rating Inventory produces scores for the following three categories: evidence (6 items), content (11 items), and customizability (11 items). The final (fourth) metric is the total score, which constitutes the sum of the 3 categories. All 28 items are weighted equally; no item is considered more (or less) important than any other item. As the scoring system is binary (either the app contains the feature or it does not), the ratings' results are not dependent on a rater's nuanced assessments.ConclusionsUsing predetermined search criteria, app ratings begin with an environmental scan of the App Store and Google Play. This first step in market research funnels hundreds of apps in a given disease category down to a manageable top 10 apps that are, thereafter, rated using the App Rating Inventory. The category and final scores derived from the rating system inform the clinician about whether an app is evidence informed and easy to use. Although a rating allows a clinician to make focused decisions about app selection in a context where thousands of apps are available, clinicians must weigh the following factors before integrating apps into a treatment plan: clinical presentation, patient engagement and preferences, available resources, and technology expertise.

Project description:BackgroundMobile apps are becoming increasingly popular, with 5.70 million apps available in early 2021. Smartphones can provide portable and convenient access to health apps. Here, we consider apps for people with one of the estimated 7000 rare conditions, which are defined as having an incidence of <1 in 2000. The needs of people with rare conditions are known to be different from those of people with more common conditions. The former may be socially isolated (not knowing anyone else who has the condition) and may not be able to find reliable information about the disorder.ObjectiveThe aim of this review is to search for apps developed specifically for people diagnosed with a rare disease and to assess them for quality using the Mobile App Rating Scale (MARS). We examine features that address 6 identified needs of people with a rare disorder and make recommendations for future developers.MethodsGoogle Play Store (Android) and Apple App Store (iOS) were searched for relevant health-related apps specifically for rare diseases. The search included the names of 10 rare disease groups. App quality was determined using MARS, assessing app engagement, functionality, aesthetics, and information.ResultsWe found 29 relevant apps (from a total of 2272) addressing 14 rare diseases or disease groups. The most common rare conditions addressed were cystic fibrosis (n=6), hemophilia (n=5), and thalassemia (n=5). The most common app features were web-based information and symptom trackers. The mean MARS score was 3.44 (SD 0.84). Lowest scores were for engagement.ConclusionsMost apps provided factual and visual information, providing tools for self-monitoring and resources to help improve interactions during health consultations. App origin and quality varied greatly. Developers are recommended to consider ways to make appropriate apps more easily identifiable to consumers, to always include high-quality information, improve engagement, provide qualitative evaluations of the app, and include consumers and clinicians in the design.

Project description:BackgroundThe use of mobile apps in health care is growing. Current and future practitioners must be equipped with the skills to navigate and utilize apps in patient care, yet few strategies exist for training health care professional students on the usage of apps.ObjectiveTo characterize first-year pharmacy student use of medical apps, evaluate first-year pharmacy student's perception of skills in finding, evaluating, and using medical apps before and after a focused learning experience, and assess student satisfaction and areas for improvement regarding the learning experience.MethodsStudents listened to a recorded, Web-based lecture on finding, evaluating, and using mobile apps in patient care. A 2-hour, interactive workshop was conducted during which students were led by an instructor through a discussion on strategies for finding and using apps in health care. The students practiced evaluating 6 different health care-related apps. Surveys were conducted before and after the focused learning experience to assess students' perceptions of medical apps and current use and perspectives on satisfaction with the learning experience and role of technology in health care.ResultsThis educational intervention is the first described formal, interactive method to educate student pharmacists on medical apps. With a 99% response rate, surveys conducted before and after the learning experience displayed perceived improvement in student skills related to finding (52/119, 44% before vs 114/120, 95% after), evaluating (18/119, 15% before vs 112/120, 93% after), and using medical apps in patient care (31/119, 26% before vs 108/120, 90% after) and the health sciences classroom (38/119, 32% before vs 104/120, 87% after). Students described satisfaction with the educational experience and agreed that it should be repeated in subsequent years (89/120, 74% agreed or strongly agreed). Most students surveyed possessed portable electronic devices (107/119, 90% mobile phone) and agreed with the concept of medical apps being an important part of the health care profession in the future (112/119, 94% before and 115/120, 96% after).ConclusionsStudent pharmacists recognize the key role technology plays in the future of health care. A medical apps workshop was successful in improving student pharmacists' perceptions of ability to find, evaluate, and use medical apps.