Dataset Information

A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing.

ABSTRACT:

Background

The usability of electronic health (eHealth) and mobile health apps is of paramount importance as it impacts the quality of care. Methodological quality assessment is a common practice in the field of health for different designs and types of studies. However, we were unable to find a scale to assess the methodological quality of studies on the usability of eHealth products or services.

Objective

This study aimed to develop a scale to assess the methodological quality of studies assessing usability of mobile apps and to perform a preliminary analysis of of the scale's feasibility, reliability, and construct validity on studies assessing usability of mobile apps, measuring aspects of physical activity.

Methods

A 3-round Delphi panel was used to generate a pool of items considered important when assessing the quality of studies on the usability of mobile apps. These items were used to write the scale and the guide to assist its use. The scale was then used to assess the quality of studies on usability of mobile apps for physical activity, and it assessed in terms of feasibility, interrater reliability, and construct validity.

Results

A total of 25 experts participated in the Delphi panel, and a 15-item scale was developed. This scale was shown to be feasible (time of application mean 13.10 [SD 2.59] min), reliable (intraclass correlation coefficient=0.81; 95% CI 0.55-0.93), and able to discriminate between low- and high-quality studies (high quality: mean 9.22 [SD 0.36]; low quality: mean 6.86 [SD 0.80]; P=.01).

Conclusions

The scale that was developed can be used both to assess the methodological quality of usability studies and to inform its planning.

SUBMITTER: Silva AG

PROVIDER: S-EPMC6884719 | biostudies-literature | 2019 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing.

Silva Anabela G AG Simões Patrícia P Santos Rita R Queirós Alexandra A Rocha Nelson P NP Rodrigues Mário M

Journal of medical Internet research 20191115 11

<h4>Background</h4>The usability of electronic health (eHealth) and mobile health apps is of paramount importance as it impacts the quality of care. Methodological quality assessment is a common practice in the field of health for different designs and types of studies. However, we were unable to find a scale to assess the methodological quality of studies on the usability of eHealth products or services.<h4>Objective</h4>This study aimed to develop a scale to assess the methodological quality o ...[more]

PMID: 31730036

Similar Datasets

Project description:BackgroundThe clinimetric properties of knee goniometry are essential to appreciate in light of its extensive use in the orthopaedic and rehabilitative communities. Intra-observer reliability is thought to be satisfactory, but the validity and inter-rater reliability of knee goniometry often demonstrate unacceptable levels of variation. This study tests the validity and reliability of measuring knee range of motion using goniometry and photographic records.MethodsDesignMethodology study assessing the validity and reliability of one method ('Marker Method') which uses a skin marker over the greater trochanter and another method ('Line of Femur Method') which requires estimation of the line of femur.SettingRadiology and orthopaedic departments of two teaching hospitals.Participants31 volunteers (13 arthritic and 18 healthy subjects). Knee range of motion was measured radiographically and photographically using a goniometer. Three assessors were assessed for reliability and validity.Main outcomesAgreement between methods and within raters was assessed using concordance correlation coefficient (CCCs). Agreement between raters was assessed using intra-class correlation coefficients (ICCs). 95% limits of agreement for the mean difference for all paired comparisons were computed.ResultsValidity (referenced to radiographs): Each method for all 3 raters yielded very high CCCs for flexion (0.975 to 0.988), and moderate to substantial CCCs for extension angles (0.478 to 0.678). The mean differences and 95% limits of agreement were narrower for flexion than they were for extension. Intra-rater reliability: For flexion and extension, very high CCCs were attained for all 3 raters for both methods with slightly greater CCCs seen for flexion (CCCs varied from 0.981 to 0.998). Inter-rater reliability: For both methods, very high ICCs (min to max: 0.891 to 0.995) were obtained for flexion and extension. Slightly higher coefficients were obtained for flexion compared to extension, and with the Marker compared to the Line of Femur Method. For intra- and inter-rater reliability, the mean differences (within 2 degrees) and 95% limits of agreement (within 5 degrees) were generally clinically acceptable for both methods.ConclusionPhotography potentially offers a superior method of measurement over standard goniometry as visualising the centre of the knee is simplified in a two-dimensional plane and the permanent record provides greater assessor transparency as well as opportunity to confer. The Marker and Line of Femur Methods have moderate to substantial validity, but the inter- and intra-rater repeatability for trained observers are excellent with both methods yielding small mean differences with narrow limits of agreement. The Line of Femur Method offers the added advantage that it does not rely on inter-clinician consistency in identifying the greater trochanter.

Project description:Objective(i) To develop the Pharmacy Value-Added Services Questionnaire (PVASQ) using emerging themes generated from interviews. (ii) To establish reliability and validity of questionnaire instrument.MethodsUsing an extended Theory of Planned Behavior as the theoretical model, face-to-face interviews generated salient beliefs of pharmacy value-added services. The PVASQ was constructed initially in English incorporating important themes and later translated into the Malay language with forward and backward translation. Intention (INT) to adopt pharmacy value-added services is predicted by attitudes (ATT), subjective norms (SN), perceived behavioral control (PBC), knowledge and expectations. Using a 7-point Likert-type scale and a dichotomous scale, test-retest reliability (N=25) was assessed by administrating the questionnaire instrument twice at an interval of one week apart. Internal consistency was measured by Cronbach's alpha and construct validity between two administrations was assessed using the kappa statistic and the intraclass correlation coefficient (ICC). Confirmatory Factor Analysis, CFA (N=410) was conducted to assess construct validity of the PVASQ.ResultsThe kappa coefficients indicate a moderate to almost perfect strength of agreement between test and retest. The ICC for all scales tested for intra-rater (test-retest) reliability was good. The overall Cronbach' s alpha (N=25) is 0.912 and 0.908 for the two time points. The result of CFA (N=410) showed most items loaded strongly and correctly into corresponding factors. Only one item was eliminated.ConclusionsThis study is the first to develop and establish the reliability and validity of the Pharmacy Value-Added Services Questionnaire instrument using the Theory of Planned Behavior as the theoretical model. The translated Malay language version of PVASQ is reliable and valid to predict Malaysian patients' intention to adopt pharmacy value-added services to collect partial medicine supply.

Project description:OBJECTIVES:The aim of this study was to assess the reliability and validity of the Turkish version of the Individual-Level Abortion Stigma Scale. SETTING:Three hospitals affiliated to Public Hospitals Union General Secretariat in Manisa, a city located in Western Turkey. DESIGN:This was a methodological study. PARTICIPANTS:A total of 230 women aged 22-49 years who previously underwent abortion and volunteered to participate in the study. PRIMARY AND SECONDARY OUTCOME MEASURES:Data analysis was performed using the SPSS and LISREL software. The Content Validity Index was calculated based on the values obtained from expert opinions. In the reliability analysis, Cronbach's alpha values were calculated for the overall scale and its subscales. To test the stability of the scale over time, the scale was readministered to 30 women in the study 15 days later and the test-retest values (Pearson's correlation coefficients) were determined. In the validity analysis, the exploratory and confirmatory factor analyses were performed. In the exploratory factor analysis, the principal component analysis was performed using the varimax rotation and the factor loading of the scale was examined. Confirmatory factor analysis was performed to investigate the item-factor construct of the scale. RESULTS:The Content Validity Index was calculated as 0.86. The Cronbach's alpha values ranged between 0.83 and 0.94. The item-total correlation coefficients ranged between 0.36 and 0.72, and the test-retest reliability was 0.96 for the overall scale and its subscales (p<0.05). The total variance revealed by five factors was 75%. Confirmatory factor analysis of the scale revealed an acceptable compliance level (root mean square error of approximation=0.092, χ2/df=2.91, Goodness of Fit Index=0.831, Comparative Fit Index=0.928, p<0.001). CONCLUSIONS:The results suggest that this scale has the potential for use in research and clinical settings to measure abortion stigma among women.

Project description:BackgroundThe Exercise Adherence Rating Scale (EARS) is a commonly used outcome tool, which helps to identify the adherence rate of exercises and reasons for adherence and non-adherence. There is no evidence of the availability of any measurement tools to assess exercise adherence in the Nepalese context and cultural background. Therefore, we conducted a cross-cultural adaptation of the EARS into the Nepali language and investigated its reliability and validity.MethodsCross-cultural adaptation of the EARS was done based on Beaton guidelines. Psychometric properties were evaluated among 18 participants aged 18 years or older with pre-diabetes or confirmed diagnosis of any disease who were prescribed with home exercises by physiotherapists. Any disease that limited participants from doing exercise and individuals unwilling to participate were excluded. Reliability was evaluated through internal consistency, using Cronbach's alpha. Exploratory Factor Analysis (EFA) was performed to explore construct validity and confirm its unidimensionality. Receiver Operating Characteristic (ROC) curve was analyzed to identify cut-off score, sensitivity and specificity of the tool.ResultsThe Cronbach's alpha was 0.94 for EARS-adherence behavior. The EFA of 6-items adherence behavior revealed the presence of one factor with an eigenvalue exceeding one. The scree-plot suggested for extraction of only one factor with strong loading (75.84%). The Area Under the Curve was 0.91 with 95% confidence interval 0.77-1.00 at p = 0.004. The cutoff score was found 17.5 with 89% sensitivity and 78% specificity.ConclusionsThe EARS was cross-culturally adapted to the Nepali language. The reliability and construct validity of the Nepali version of the EARS were acceptable to assess exercise adherence in Nepali-speaking individuals. This validated tool might facilitate the evaluation of exercise-related interventions. Future studies could investigate other psychometric properties of the Nepali EARS.

Project description:BackgroundIn Mali and Senegal, over 1% of women die giving birth in hospital. At some hospitals, over a third of infants are stillborn. Many deaths are due to substandard medical practices. Criterion-based clinical audits (CBCA) are increasingly used to measure and improve obstetrical care in resource-limited settings, but their measurement properties have not been formally evaluated. In 2011, we published a systematic review of obstetrical CBCA highlighting insufficient considerations of validity and reliability. The objective of this study is to develop an obstetrical CBCA adapted to the West African context and assess its reliability and validity. This work was conducted as a sub-study within a cluster randomized trial known as QUARITE.MethodsCriteria were selected based on extensive literature review and expert opinion. Early 2010, two auditors applied the CBCA to identical samples at 8 sites in Mali and Senegal (n = 185) to evaluate inter-rater reliability. In 2010-11, we conducted CBCA at 32 hospitals to assess construct validity (n = 633 patients). We correlated hospital characteristics (resource availability, facility perinatal and maternal mortality) with mean hospital CBCA scores. We used generalized estimating equations to assess whether patient CBCA scores were associated with perinatal mortality.ResultsResults demonstrate substantial (ICC = 0.67, 95% CI 0.54; 0.76) to elevated inter-rater reliability (ICC = 0.84, 95% CI 0.77; 0.89) in Senegal and Mali, respectively. Resource availability positively correlated with mean hospital CBCA scores and maternal and perinatal mortality were inversely correlated with hospital CBCA scores. Poor CBCA scores, adjusted for hospital and patient characteristics, were significantly associated with perinatal mortality (OR 1.84, 95% CI 1.01-3.34).ConclusionOur CBCA has substantial inter-rater reliability and there is compelling evidence of its validity as the tool performs according to theory.Trial registrationCurrent Controlled Trials ISRCTN46950658.

Dataset Information

A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing.

Background

Objective

Methods

Results

Conclusions

Publications

A Scale to Assess the Methodological Quality of Studies Assessing Usability of Electronic Health Products and Services: Delphi Study Followed by Validity and Reliability Testing.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure