Dataset Information

Psychometric assessment of three newly developed implementation outcome measures.

ABSTRACT: Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).Thirty-six implementation scientists and 27 mental health professionals assigned 31 items to the constructs and rated their confidence in their assignments. The Wilcoxon one-sample signed rank test was used to assess substantive and discriminant content validity. Exploratory and confirmatory factor analysis (EFA and CFA) and Cronbach alphas were used to assess the validity of the conceptual model. Three hundred twenty-six mental health counselors read one of six randomly assigned vignettes depicting a therapist contemplating adopting an evidence-based practice (EBP). Participants used 15 items to rate the therapist's perceptions of the acceptability, appropriateness, and feasibility of adopting the EBP. CFA and Cronbach alphas were used to refine the scales, assess structural validity, and assess reliability. Analysis of variance (ANOVA) was used to assess known-groups validity. Finally, half of the counselors were randomly assigned to receive the same vignette and the other half the opposite vignette; and all were asked to re-rate acceptability, appropriateness, and feasibility. Pearson correlation coefficients were used to assess test-retest reliability and linear regression to assess sensitivity to change.All but five items exhibited substantive and discriminant content validity. A trimmed CFA with five items per construct exhibited acceptable model fit (CFI = 0.98, RMSEA = 0.08) and high factor loadings (0.79 to 0.94). The alphas for 5-item scales were between 0.87 and 0.89. Scale refinement based on measure-specific CFAs and Cronbach alphas using vignette data produced 4-item scales (?'s from 0.85 to 0.91). A three-factor CFA exhibited acceptable fit (CFI = 0.96, RMSEA = 0.08) and high factor loadings (0.75 to 0.89), indicating structural validity. ANOVA showed significant main effects, indicating known-groups validity. Test-retest reliability coefficients ranged from 0.73 to 0.88. Regression analysis indicated each measure was sensitive to change in both directions.The AIM, IAM, and FIM demonstrate promising psychometric properties. Predictive validity assessment is planned.

SUBMITTER: Weiner BJ

PROVIDER: S-EPMC5576104 | biostudies-literature | 2017 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Psychometric assessment of three newly developed implementation outcome measures.

Weiner Bryan J BJ Lewis Cara C CC Stanick Cameo C Powell Byron J BJ Dorsey Caitlin N CN Clary Alecia S AS Boynton Marcella H MH Halko Heather H

Implementation science : IS 20170829 1

<h4>Background</h4>Implementation outcome measures are essential for monitoring and evaluating the success of implementation efforts. Yet, currently available measures lack conceptual clarity and have largely unknown reliability and validity. This study developed and psychometrically assessed three new measures: the Acceptability of Intervention Measure (AIM), Intervention Appropriateness Measure (IAM), and Feasibility of Intervention Measure (FIM).<h4>Methods</h4>Thirty-six implementation scien ...[more]

PMID: 28851459

Similar Datasets

Project description:BackgroundBeta-propeller protein-associated neurodegeneration (BPAN) is a rare neurodegenerative disorder characterized by iron accumulation in the brain with spectrum of neurodevelopmental and movement phenotypes. In anticipation of future clinical trials and to inform clinical care, there is an unmet need to capture the phenotypic diversity of this rare disorder and better define disease subtypes.MethodsA total of 27 individuals with BPAN were included in our natural history study, from which traditional outcome measures were obtained in 18 subjects. Demographic and diagnostic information, along with acquisition of basic developmental skills and overall neurologic severity were extracted from the medical records. Functional outcome measures were administered at the time of the evaluation or applied retrospectively at the last clinical encounter for patients who were not able to travel for in person. Based on age and functional level, the following assessments were administered: Leiter-3, Gross Motor Function Measure (GMFM)-66 Item Sets, Vineland-3, and Peabody-2.ResultsOverall, cognitive function was more impaired compared to gross motor function. Onset of symptoms of BPAN within the first 6 months of life was associated with decreased gain of ambulation and gain of spoken language (ambulation: log-rank test p = 0.0015; gain of first word: p = 0.0015). There was no difference in age at seizure onset by age at initial symptom onset (p = 0.8823). Collection of prospective outcome measures was limited by attention and behavior in our patient population, reinforcing the complexity of phenotype assessment and inadequacy of available standardized tests. Overall, gross motor and adaptive behavior assessments were better able to capture the dynamic range of function across the BPAN population than the fine motor and non-verbal cognitive tests. Floor effects were noted across outcome measures in a subset of individuals for cognitive and adaptive behavior tests.ConclusionOur data suggest the distinct phenotypes of BPAN: a severe, early onset form and an attenuated form with higher cognitive capabilities. Early age at onset was a key factor in predicting future neurologic impairment.

Project description:BackgroundThe management of people living with HIV and AIDS is multidimensional and complex. Using patient-reported outcome measures (PROMs) has been increasingly recognized to be the key factor for providing patient-centered health care to meet the lifelong needs of people living with HIV and AIDS from diagnosis to death. However, there is currently no consensus on a PROM recommended for health care providers and researchers to assess health outcomes in people living with HIV and AIDS.ObjectiveThe purpose of this systematic review was to summarize and categorize the available validated HIV-specific PROMs in adults living with HIV and AIDS and to assess these PROMs using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) methodology.MethodsThis systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search of 3 recommended databases (PubMed, Embase, and PsychINFO) was conducted on January 15, 2021. Studies were included if they assessed any psychometric property of HIV-specific PROMs in adults living with HIV and AIDS and met the eligibility criteria. The PROMs were assessed for 9 psychometric properties, evaluated in each included study following the COSMIN methodology by assessing the following: the methodological quality assessed using the COSMIN risk of bias checklist; overall rating of results; level of evidence assessed using the modified Grading of Recommendations, Assessment, Development, and Evaluation approach; and level of recommendation.ResultsA total of 88 PROMs classified into 8 categories, assessing the psychometric properties of PROMs for adults living with HIV and AIDS, were identified in 152 studies including 79,213 people living with HIV and AIDS. The psychometric properties of most included PROMs were rated with insufficient evidence. The PROMs that received class A recommendation were the Poz Quality of Life, HIV Symptom Index or Symptoms Distress Module of the Adult AIDS Clinical Trial Group, and People Living with HIV Resilience Scale. In addition, because of a lack of evidence, recommendations regarding use could not be made for most of the remaining assessed PROMs (received class B recommendation).ConclusionsThis systematic review recommends 3 PROMs to assess health outcomes in adults living with HIV and AIDS. However, all these PROMs have some shortcomings. In addition, most of the included PROMs do not have sufficient evidence for assessing their psychometric properties and require a more comprehensive validation of the psychometric properties in the future to provide more scientific evidence. Thus, our findings may provide a reference for the selection of high-quality HIV-specific PROMs by health care providers and researchers for clinical practice and research.

Dataset Information

Psychometric assessment of three newly developed implementation outcome measures.

Publications

Psychometric assessment of three newly developed implementation outcome measures.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets