Dataset Information

Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis.

ABSTRACT:

Background

Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies.

Objective

The objective of this project was to conduct a systematic review to (1) identify existing instruments for eHealth research and implementation evaluation from the patient's point of view, (2) characterize measurement components, and (3) assess psychometrics.

Methods

Concepts from existing models and published studies of technology use and adoption were identified and used to inform a search strategy. Search terms were broadly categorized as platforms (eg, email), measurement (eg, survey), function/information use (eg, self-management), health care occupations (eg, nurse), and eHealth/telemedicine (eg, mHealth). A computerized database search was conducted through June 2014. Included articles (1) described development of an instrument, or (2) used an instrument that could be traced back to its original publication, or (3) modified an instrument, and (4) with full text in English language, and (5) focused on the patient perspective on technology, including patient preferences and satisfaction, engagement with technology, usability, competency and fluency with technology, computer literacy, and trust in and acceptance of technology. The review was limited to instruments that reported at least one psychometric property. Excluded were investigator-developed measures, disease-specific assessments delivered via technology or telephone (eg, a cancer-coping measure delivered via computer survey), and measures focused primarily on clinician use (eg, the electronic health record).

Results

The search strategy yielded 47,320 articles. Following elimination of duplicates and non-English language publications (n=14,550) and books (n=27), another 31,647 articles were excluded through review of titles. Following a review of the abstracts of the remaining 1096 articles, 68 were retained for full-text review. Of these, 16 described an instrument and six used an instrument; one instrument was drawn from the GEM database, resulting in 23 articles for inclusion. None included a complete psychometric evaluation. The most frequently assessed property was internal consistency (21/23, 91%). Testing for aspects of validity ranged from 48% (11/23) to 78% (18/23). Approximately half (13/23, 57%) reported how to score the instrument. Only six (26%) assessed the readability of the instrument for end users, although all the measures rely on self-report.

Conclusions

Although most measures identified in this review were published after the year 2000, rapidly changing technology makes instrument development challenging. Platform-agnostic measures need to be developed that focus on concepts important for use of any type of eHealth innovation. At present, there are important gaps in the availability of psychometrically sound measures to evaluate eHealth technologies.

SUBMITTER: Wakefield BJ

PROVIDER: S-EPMC5656774 | biostudies-literature | 2017 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis.

Wakefield Bonnie J BJ Turvey Carolyn L CL Nazi Kim M KM Holman John E JE Hogan Timothy P TP Shimada Stephanie L SL Kennedy Diana R DR

Journal of medical Internet research 20171011 10

<h4>Background</h4>Significant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies.<h4>Objective</h4>The objective of this project was to conduct a sy ...[more]

PMID: 29021128

Similar Datasets

Project description:AimsOccupational Burnout (OB) is currently measured through several Patient-Reported Outcome Measures (PROMs) and some of them have become widely used in occupational health research and practice. We, therefore, aimed to review and grade the psychometric validity of the five OB PROMs considered as valid for OB measure in mental health professionals (the Maslach Burnout Inventory (MBI), the Pines' Burnout Measure (BM), the Psychologist Burnout Inventory (PBI), the OLdenburg Burnout Inventory (OLBI) and the Copenhagen Burnout Inventory (CBI)).MethodsWe conducted systematic literature searches in MEDLINE, PsycINFO and EMBASE databases. We reviewed studies published between January 1980 and September 2018 following a methodological framework, in which each step of PROM validation, the reference method, analytical technics and result interpretation criteria were assessed. Using the COnsensus-based Standards for the selection of health Measurement Instruments we evaluated the risk of bias in studies assessing content and criterion validity, structural validity, internal consistency, reliability, measurement error, hypotheses testing and responsiveness of each PROM. Finally, we assessed the level of evidence for the validity of each PROM using the GRADE approach.ResultsWe identified 6541 studies, 19 of which were included for review. Fifteen studies dealt with MBI whereas BM, PBI, OLBI and CBI were each examined in only one study. OLBI had the most complete validation, followed by CBI, MBI, BM and PBI, respectively. When examining the result interpretation correctness, the strongest disagreement was observed for MBI (27% of results), BM (25%) and CBI (17%). There was no disagreement regarding PBI and OLBI. For OLBI and CBI, the quality of evidence for sufficient content validity, the crucial psychometric property, was moderate; for MBI, BM and PBI, it was very low.ConclusionTo be validly and reliably used in medical research and practice, PROM should exhibit robust psychometric properties. Among the five PROMs reviewed, CBI and, to a lesser extent, OLBI meet this prerequisite. The cross-cultural validity of these PROMs was beyond the scope of our work and should be addressed in the future. Moreover, the development of a diagnostic standard for OB would be helpful to assess the sensitivity and specificity of the PROMs and further reexamine their validity.The study protocol was registered in PROSPERO (CRD 42019124621).

Project description:BackgroundThe management of people living with HIV and AIDS is multidimensional and complex. Using patient-reported outcome measures (PROMs) has been increasingly recognized to be the key factor for providing patient-centered health care to meet the lifelong needs of people living with HIV and AIDS from diagnosis to death. However, there is currently no consensus on a PROM recommended for health care providers and researchers to assess health outcomes in people living with HIV and AIDS.ObjectiveThe purpose of this systematic review was to summarize and categorize the available validated HIV-specific PROMs in adults living with HIV and AIDS and to assess these PROMs using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) methodology.MethodsThis systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search of 3 recommended databases (PubMed, Embase, and PsychINFO) was conducted on January 15, 2021. Studies were included if they assessed any psychometric property of HIV-specific PROMs in adults living with HIV and AIDS and met the eligibility criteria. The PROMs were assessed for 9 psychometric properties, evaluated in each included study following the COSMIN methodology by assessing the following: the methodological quality assessed using the COSMIN risk of bias checklist; overall rating of results; level of evidence assessed using the modified Grading of Recommendations, Assessment, Development, and Evaluation approach; and level of recommendation.ResultsA total of 88 PROMs classified into 8 categories, assessing the psychometric properties of PROMs for adults living with HIV and AIDS, were identified in 152 studies including 79,213 people living with HIV and AIDS. The psychometric properties of most included PROMs were rated with insufficient evidence. The PROMs that received class A recommendation were the Poz Quality of Life, HIV Symptom Index or Symptoms Distress Module of the Adult AIDS Clinical Trial Group, and People Living with HIV Resilience Scale. In addition, because of a lack of evidence, recommendations regarding use could not be made for most of the remaining assessed PROMs (received class B recommendation).ConclusionsThis systematic review recommends 3 PROMs to assess health outcomes in adults living with HIV and AIDS. However, all these PROMs have some shortcomings. In addition, most of the included PROMs do not have sufficient evidence for assessing their psychometric properties and require a more comprehensive validation of the psychometric properties in the future to provide more scientific evidence. Thus, our findings may provide a reference for the selection of high-quality HIV-specific PROMs by health care providers and researchers for clinical practice and research.

Project description:PurposeTo review studies about the reliability and validity of self-reported alcohol consumption measures among adults, an area which needs updating to reflect current research.MethodsDatabases (PUBMED (1966-present), MEDLINE (1946-present), EMBASE (1947-present), Cumulative Index of Nursing and Allied Health Literature (CINAHL) (1937-present), PsycINFO (1887-present) and Social Science Citation Index (1976-present)) were searched systematically for studies from inception to 11th August 2017. Pairs of independent reviewers screened study titles, abstracts and full texts with high agreement and a third author resolved disagreements. A comprehensive quality assessment was conducted of the reported psychometric properties of measures of alcohol consumption using the COnsensus-based Standards for the selection of health Measurement Instruments (COSMIN) to derive ratings of poor, fair, good or excellent for each checklist item relating to each psychometric property.ResultsTwenty-eight studies met inclusion criteria and, collectively, they investigated twenty-one short-term recall measures, fourteen quantity-frequency measures and eleven graduated-frequency measures. All measures demonstrated adequate/good test-retest reliability and convergent validity. Quantity-frequency measures demonstrated adequate/good criterion validity; graduated-frequency and short-term recall measures demonstrated adequate/good divergent validity. Quantity-frequency measures and short-term recall measures demonstrated adequate/good hypothesis validity; short-term recall measures demonstrated adequate construct validity. Methodological quality varied within and between studies.ConclusionsIt was difficult to discern conclusively which measure was the most reliable and valid given that no study assessed all psychometric properties and the included studies varied in the psychometric properties that they selected to assess. However, when the results from the range of studies were considered and summed, they tended to indicate that the quantity-frequency measure compared to the other two measures performed best in psychometric terms and, therefore, it is likely to produce the most reliable and valid assessment of alcohol consumption in population surveys.

Project description:IntroductionImproving social inclusion opportunities for population health has been identified as a priority area for international policy. There is a need to comprehensively examine and evaluate the quality of psychometric properties of measures of social inclusion that are used to guide social policy and outcomes.ObjectiveTo conduct a systematic review of the literature on all current measures of social inclusion for any population group, to evaluate the quality of the psychometric properties of identified measures, and to evaluate if they capture the construct of social inclusion.MethodsA systematic search was performed using five electronic databases: CINAHL, PsycINFO, Embase, ERIC and Pubmed and grey literature were sourced to identify measures of social inclusion. The psychometric properties of the social inclusion measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.ResultsOf the 109 measures identified, twenty-five measures, involving twenty-five studies and one manual met the inclusion criteria. The overall quality of the reviewed measures was variable, with the Social and Community Opportunities Profile-Short, Social Connectedness Scale and the Social Inclusion Scale demonstrating the strongest evidence for sound psychometric quality. The most common domain included in the measures was connectedness (21), followed by participation (19); the domain of citizenship was covered by the least number of measures (10). No single instrument measured all aspects within the three domains of social inclusion. Of the measures with sound psychometric evidence, the Social and Community Opportunities Profile-Short captured the construct of social inclusion best.ConclusionsThe overall quality of the psychometric properties demonstrate that the current suite of available instruments for the measurement of social inclusion are promising but need further refinement. There is a need for a universal working definition of social inclusion as an overarching construct for ongoing research in the area of the psychometric properties of social inclusion instruments.

Project description:BackgroundMobile health interventions delivered through mobile apps are increasingly used in physiotherapy care. This may be because of the potential of apps to facilitate changes in behavior, which is central to the aims of care delivered by physiotherapists. A benefit of using apps is their ability to incorporate behavior change techniques (BCTs) that can optimize the effectiveness of physiotherapeutic interventions. Research continues to suggest that despite their importance, behavior change strategies are often missing in patient management. Evaluating mobile apps that physiotherapists can use to drive behavior change may inform clinical practice and potentially improve patient outcomes. Examining the quality of apps and exploring their key features that can support behavior change and physiotherapy care are important aspects of such an evaluation.ObjectiveThe primary aim of this study was to describe the range of mobile apps in app stores that are intended for use by patients to support physiotherapy care. The secondary aims were to assess app quality, BCTs, and their behavior change potential.MethodsA systematic review of mobile apps in app stores was undertaken. The Apple App Store and Google Play were searched using a 2-step search strategy, using terms relevant to the physiotherapy discipline. Strict inclusion and exclusion criteria were applied: apps had to be intended for use by patients and be self-contained (or stand-alone) without the requirement to be used in conjunction with a partner wearable device or another plugin. Included apps were coded for BCTs using the Behavior Change Technique Taxonomy version 1. App quality was assessed using the Mobile App Rating Scale, and the App Behavior Change Scale was used to assess the app's potential to change behavior.ResultsIn total, 1240 apps were screened, and 35 were included. Of these 35 apps, 22 (63%) were available on both the Apple App Store and Google Play platforms. In total, 24 (69%) were general in their focus (eg, not condition-specific), with the remaining 11 (31%) being more specific (eg, knee rehabilitation and pelvic floor training). The mean app quality score (Mobile App Rating Scale) was 3.7 (SD 0.4) of 5 (range 2.8-4.5). The mean number of BCTs identified per app was 8.5 (SD 3.6). BCTs most frequently included in the apps were instruction on how to perform a behavior (n=32), action planning (n=30), and self-monitoring of behavior (n=28). The mean behavior change potential score (App Behavior Change Scale) was 8.5 (SD 3.1) of 21 (range 3-15).ConclusionsMobile apps available to support patient care received from a physiotherapist are of variable quality. Although they contain some BCTs, the potential for behavior change varied widely across apps.International registered report identifier (irrid)RR2-10.2196/29047.

Project description:BACKGROUND:Developing successful interventions for chronic musculoskeletal pain requires valid, responsive, and reliable outcome measures. The Minneapolis VA Evidence-based Synthesis Program completed a focused evidence review on key psychometric properties of 17 self-report measures of pain severity and pain-related functional impairment suitable for clinical research on chronic musculoskeletal pain. METHODS:Pain experts of the VA Pain Measurement Outcomes Workgroup identified 17 pain measures to undergo systematic review. In addition to a MEDLINE search on these 17 measures (1/2000-1/2017), we hand-searched (without publication date limits) the reference lists of all included studies, prior systematic reviews, and-when available-Web sites dedicated to each measure (PROSPERO registration CRD42017056610). Our primary outcome was the measure's minimal important difference (MID). Secondary outcomes included responsiveness, validity, and test-retest reliability. Outcomes were synthesized through evidence mapping and qualitative comparison. RESULTS:Of 1635 abstracts identified, 331 articles underwent full-text review, and 43 met inclusion criteria. Five measures (Oswestry Disability Index (ODI), Roland-Morris Disability Questionnaire (RMDQ), SF-36 Bodily Pain Scale (SF-36 BPS), Numeric Rating Scale (NRS), and Visual Analog Scale (VAS)) had data reported on MID, responsiveness, validity, and test-retest reliability. Seven measures had data reported on three of the four psychometric outcomes. Eight measures had reported MIDs, though estimation methods differed substantially and often were not clinically anchored. CONCLUSIONS:In this focused evidence review, the most evidence on key psychometric properties in chronic musculoskeletal pain populations was found for the ODI, RMDQ, SF-36 BPS, NRS, and VAS. Key limitations in the field include substantial variation in methods of estimating psychometric properties, defining chronic musculoskeletal pain, and reporting patient demographics. TRIAL REGISTRATION:Registered in the PROSPERO database: CRD42017056610.

Dataset Information

Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis.

Background

Objective

Methods

Results

Conclusions

Publications

Psychometric Properties of Patient-Facing eHealth Evaluation Measures: Systematic Review and Analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets