Evaluating the Psychometric Quality of Social Skills Measures: A Systematic Review.
Ontology highlight
ABSTRACT: Introduction
Impairments in social functioning are associated with an array of adverse outcomes. Social skills measures are commonly used by health professionals to assess and plan the treatment of social skills difficulties. There is a need to comprehensively evaluate the quality of psychometric properties reported across these measures to guide assessment and treatment planning.Objective
To conduct a systematic review of the literature on the psychometric properties of social skills and behaviours measures for both children and adults.Methods
A systematic search was performed using four electronic databases: CINAHL, PsycINFO, Embase and Pubmed; the Health and Psychosocial Instruments database; and grey literature using PsycExtra and Google Scholar. The psychometric properties of the social skills measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.Results
Thirty-Six studies and nine manuals were included to assess the psychometric properties of thirteen social skills measures that met the inclusion criteria. Most measures obtained excellent overall methodological quality scores for internal consistency and reliability. However, eight measures did not report measurement error, nine measures did not report cross-cultural validity and eleven measures did not report criterion validity.Conclusions
The overall quality of the psychometric properties of most measures was satisfactory. The SSBS-2, HCSBS and PKBS-2 were the three measures with the most robust evidence of sound psychometric quality in at least seven of the eight psychometric properties that were appraised. A universal working definition of social functioning as an overarching construct is recommended. There is a need for ongoing research in the area of the psychometric properties of social skills and behaviours instruments.
SUBMITTER: Cordier R
PROVIDER: S-EPMC4494713 | biostudies-literature | 2015
REPOSITORIES: biostudies-literature
Publications
Evaluating the Psychometric Quality of Social Skills Measures: A Systematic Review.
Cordier Reinie R Speyer Renée R Chen Yu-Wei YW Wilkes-Gillan Sarah S Brown Ted T Bourke-Taylor Helen H Doma Kenji K Leicht Anthony A
PloS one 20150707 7
<h4>Introduction</h4>Impairments in social functioning are associated with an array of adverse outcomes. Social skills measures are commonly used by health professionals to assess and plan the treatment of social skills difficulties. There is a need to comprehensively evaluate the quality of psychometric properties reported across these measures to guide assessment and treatment planning.<h4>Objective</h4>To conduct a systematic review of the literature on the psychometric properties of social s ...[more]
Similar Datasets
Project description:IntroductionThere is a need to comprehensively examine and evaluate the quality of the psychometric properties of school connectedness measures to inform school based assessment and intervention planning.ObjectiveTo systematically review the literature on the psychometric properties of self-report measures of school connectedness for students aged six to 14 years.MethodsA systematic search of five electronic databases and gray literature was conducted. The COnsensus-based Standards for the selection of heath Measurement INstruments (COSMIN) taxonomy of measurement properties was used to evaluate the quality of studies and a pre-set psychometric criterion was used to evaluate the overall quality of psychometric properties.ResultsThe measures with the strongest psychometric properties was the School Climate Measure and the 35-item version Student Engagement Instrument exploring eight and 12 (of 15) school connectedness components respectively.ConclusionsThe overall quality of psychometric properties was limited suggesting school connectedness measures available require further development and evaluation.
| S-EPMC6133283 | biostudies-literature
Project description:IntroductionImproving social inclusion opportunities for population health has been identified as a priority area for international policy. There is a need to comprehensively examine and evaluate the quality of psychometric properties of measures of social inclusion that are used to guide social policy and outcomes.ObjectiveTo conduct a systematic review of the literature on all current measures of social inclusion for any population group, to evaluate the quality of the psychometric properties of identified measures, and to evaluate if they capture the construct of social inclusion.MethodsA systematic search was performed using five electronic databases: CINAHL, PsycINFO, Embase, ERIC and Pubmed and grey literature were sourced to identify measures of social inclusion. The psychometric properties of the social inclusion measures were evaluated against the COSMIN taxonomy of measurement properties using pre-set psychometric criteria.ResultsOf the 109 measures identified, twenty-five measures, involving twenty-five studies and one manual met the inclusion criteria. The overall quality of the reviewed measures was variable, with the Social and Community Opportunities Profile-Short, Social Connectedness Scale and the Social Inclusion Scale demonstrating the strongest evidence for sound psychometric quality. The most common domain included in the measures was connectedness (21), followed by participation (19); the domain of citizenship was covered by the least number of measures (10). No single instrument measured all aspects within the three domains of social inclusion. Of the measures with sound psychometric evidence, the Social and Community Opportunities Profile-Short captured the construct of social inclusion best.ConclusionsThe overall quality of the psychometric properties demonstrate that the current suite of available instruments for the measurement of social inclusion are promising but need further refinement. There is a need for a universal working definition of social inclusion as an overarching construct for ongoing research in the area of the psychometric properties of social inclusion instruments.
| S-EPMC5466312 | biostudies-literature
Project description:AimsOccupational Burnout (OB) is currently measured through several Patient-Reported Outcome Measures (PROMs) and some of them have become widely used in occupational health research and practice. We, therefore, aimed to review and grade the psychometric validity of the five OB PROMs considered as valid for OB measure in mental health professionals (the Maslach Burnout Inventory (MBI), the Pines' Burnout Measure (BM), the Psychologist Burnout Inventory (PBI), the OLdenburg Burnout Inventory (OLBI) and the Copenhagen Burnout Inventory (CBI)).MethodsWe conducted systematic literature searches in MEDLINE, PsycINFO and EMBASE databases. We reviewed studies published between January 1980 and September 2018 following a methodological framework, in which each step of PROM validation, the reference method, analytical technics and result interpretation criteria were assessed. Using the COnsensus-based Standards for the selection of health Measurement Instruments we evaluated the risk of bias in studies assessing content and criterion validity, structural validity, internal consistency, reliability, measurement error, hypotheses testing and responsiveness of each PROM. Finally, we assessed the level of evidence for the validity of each PROM using the GRADE approach.ResultsWe identified 6541 studies, 19 of which were included for review. Fifteen studies dealt with MBI whereas BM, PBI, OLBI and CBI were each examined in only one study. OLBI had the most complete validation, followed by CBI, MBI, BM and PBI, respectively. When examining the result interpretation correctness, the strongest disagreement was observed for MBI (27% of results), BM (25%) and CBI (17%). There was no disagreement regarding PBI and OLBI. For OLBI and CBI, the quality of evidence for sufficient content validity, the crucial psychometric property, was moderate; for MBI, BM and PBI, it was very low.ConclusionTo be validly and reliably used in medical research and practice, PROM should exhibit robust psychometric properties. Among the five PROMs reviewed, CBI and, to a lesser extent, OLBI meet this prerequisite. The cross-cultural validity of these PROMs was beyond the scope of our work and should be addressed in the future. Moreover, the development of a diagnostic standard for OB would be helpful to assess the sensitivity and specificity of the PROMs and further reexamine their validity.The study protocol was registered in PROSPERO (CRD 42019124621).
| S-EPMC8057391 | biostudies-literature
Project description:BackgroundCulinary skills are important objects of study in the field of Public Health. Studies that propose to develop instruments for assessing such construct show lack of methodological uniformity to report validity and reliability of their instruments.ObjectiveTo identify studies that have developed instruments to measure culinary skills in adult population, and critically assess their psychometric properties.DesignWe conducted a systematic review according to the PRISMA statement. We searched literature PubMed/Medline, Scopus, LILACS, and Web of Science databases until January 2021, and consulted Google Scholar for relevant grey literature. Two reviewers independently selected the studies, conducted data extraction, and assessed the psychometric quality of the instruments. A third reviewer resolved any doubts or disagreements in all steps of the systematic review.ResultsThe search identified 1148 potentially relevant studies, out of which 9 met the inclusion criteria. In addition, we included 3 studies by searching the related articles and the reference lists of these studies, totaling 12 included studies in this review. Ten studies reported the development of tools measuring culinary skills in adults and 2 studies performed cross-cultural adaptations of original instruments. We considered adequate quality of internal consistency reliability in four studies. One study received adequate rating for test-retest reliability. No studies presented adequate rating for content validity and four studies showed satisfactory results for at least one type of construct validity. One study reported criterion validity and the quality of this psychometric property was inadequate.ConclusionsWe identified many studies that surveyed culinary skills. Although the isolated measures appraised in this review show good promise in terms of quality of psychometric properties, no studies presented adequate measures for each aspect of reliability and validity. A more consistent and consensual definition of culinary skills is recommended. The flaws observed in these studies show that there is a need for ongoing research in the area of the psychometric properties of instruments assessing culinary skills.
| S-EPMC8351978 | biostudies-literature
Project description:BackgroundThe management of people living with HIV and AIDS is multidimensional and complex. Using patient-reported outcome measures (PROMs) has been increasingly recognized to be the key factor for providing patient-centered health care to meet the lifelong needs of people living with HIV and AIDS from diagnosis to death. However, there is currently no consensus on a PROM recommended for health care providers and researchers to assess health outcomes in people living with HIV and AIDS.ObjectiveThe purpose of this systematic review was to summarize and categorize the available validated HIV-specific PROMs in adults living with HIV and AIDS and to assess these PROMs using the Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) methodology.MethodsThis systematic review followed the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) guidelines. A literature search of 3 recommended databases (PubMed, Embase, and PsychINFO) was conducted on January 15, 2021. Studies were included if they assessed any psychometric property of HIV-specific PROMs in adults living with HIV and AIDS and met the eligibility criteria. The PROMs were assessed for 9 psychometric properties, evaluated in each included study following the COSMIN methodology by assessing the following: the methodological quality assessed using the COSMIN risk of bias checklist; overall rating of results; level of evidence assessed using the modified Grading of Recommendations, Assessment, Development, and Evaluation approach; and level of recommendation.ResultsA total of 88 PROMs classified into 8 categories, assessing the psychometric properties of PROMs for adults living with HIV and AIDS, were identified in 152 studies including 79,213 people living with HIV and AIDS. The psychometric properties of most included PROMs were rated with insufficient evidence. The PROMs that received class A recommendation were the Poz Quality of Life, HIV Symptom Index or Symptoms Distress Module of the Adult AIDS Clinical Trial Group, and People Living with HIV Resilience Scale. In addition, because of a lack of evidence, recommendations regarding use could not be made for most of the remaining assessed PROMs (received class B recommendation).ConclusionsThis systematic review recommends 3 PROMs to assess health outcomes in adults living with HIV and AIDS. However, all these PROMs have some shortcomings. In addition, most of the included PROMs do not have sufficient evidence for assessing their psychometric properties and require a more comprehensive validation of the psychometric properties in the future to provide more scientific evidence. Thus, our findings may provide a reference for the selection of high-quality HIV-specific PROMs by health care providers and researchers for clinical practice and research.
| S-EPMC9782451 | biostudies-literature
Project description:BackgroundThe current state of evidence regarding measures that assess evidence-informed decision-making (EIDM) competence attributes (i.e., knowledge, skills, attitudes/beliefs, behaviours) among nurses is unknown. This systematic review provides a narrative synthesis of the psychometric properties and general characteristics of EIDM competence attribute measures in nursing.MethodsThe search strategy included online databases, hand searches, grey literature, and content experts. To align with the Cochrane Handbook of Systematic Reviews, psychometric outcome data (i.e., acceptability, reliability, validity) were extracted in duplicate, while all remaining data (i.e., study and measure characteristics) were extracted by one team member and checked by a second member for accuracy. Acceptability data was defined as measure completion time and overall rate of missing data. The Standards for Educational and Psychological Testing was used as the guiding framework to define reliability, and validity evidence, identified as a unified concept comprised of four validity sources: content, response process, internal structure and relationships to other variables. A narrative synthesis of measure and study characteristics, and psychometric outcomes is presented across measures and settings.ResultsA total of 5883 citations were screened with 103 studies and 35 unique measures included in the review. Measures were used or tested in acute care (n?=?31 measures), public health (n?=?4 measures), home health (n?=?4 measures), and long-term care (n?=?1 measure). Half of the measures assessed a single competence attribute (n?=?19; 54.3%). Three measures (9%) assessed four competence attributes of knowledge, skills, attitudes/beliefs and behaviours. Regarding acceptability, overall missing data ranged from 1.6-25.6% across 11 measures and completion times ranged from 5 to 25?min (n?=?4 measures). Internal consistency reliability was commonly reported (21 measures), with Cronbach's alphas ranging from 0.45-0.98. Two measures reported four sources of validity evidence, and over half (n?=?19; 54%) reported one source of validity evidence.ConclusionsThis review highlights a gap in the testing and use of competence attribute measures related to evidence-informed decision making in community-based and long-term care settings. Further development of measures is needed conceptually and psychometrically, as most measures assess only a single competence attribute, and lack assessment and evidence of reliability and sources of established validity evidence.RegistrationPROSPERO #CRD42018088754.
| S-EPMC7254762 | biostudies-literature
Project description:BackgroundSignificant resources are being invested into eHealth technology to improve health care. Few resources have focused on evaluating the impact of use on patient outcomes A standardized set of metrics used across health systems and research will enable aggregation of data to inform improved implementation, clinical practice, and ultimately health outcomes associated with use of patient-facing eHealth technologies.ObjectiveThe objective of this project was to conduct a systematic review to (1) identify existing instruments for eHealth research and implementation evaluation from the patient's point of view, (2) characterize measurement components, and (3) assess psychometrics.MethodsConcepts from existing models and published studies of technology use and adoption were identified and used to inform a search strategy. Search terms were broadly categorized as platforms (eg, email), measurement (eg, survey), function/information use (eg, self-management), health care occupations (eg, nurse), and eHealth/telemedicine (eg, mHealth). A computerized database search was conducted through June 2014. Included articles (1) described development of an instrument, or (2) used an instrument that could be traced back to its original publication, or (3) modified an instrument, and (4) with full text in English language, and (5) focused on the patient perspective on technology, including patient preferences and satisfaction, engagement with technology, usability, competency and fluency with technology, computer literacy, and trust in and acceptance of technology. The review was limited to instruments that reported at least one psychometric property. Excluded were investigator-developed measures, disease-specific assessments delivered via technology or telephone (eg, a cancer-coping measure delivered via computer survey), and measures focused primarily on clinician use (eg, the electronic health record).ResultsThe search strategy yielded 47,320 articles. Following elimination of duplicates and non-English language publications (n=14,550) and books (n=27), another 31,647 articles were excluded through review of titles. Following a review of the abstracts of the remaining 1096 articles, 68 were retained for full-text review. Of these, 16 described an instrument and six used an instrument; one instrument was drawn from the GEM database, resulting in 23 articles for inclusion. None included a complete psychometric evaluation. The most frequently assessed property was internal consistency (21/23, 91%). Testing for aspects of validity ranged from 48% (11/23) to 78% (18/23). Approximately half (13/23, 57%) reported how to score the instrument. Only six (26%) assessed the readability of the instrument for end users, although all the measures rely on self-report.ConclusionsAlthough most measures identified in this review were published after the year 2000, rapidly changing technology makes instrument development challenging. Platform-agnostic measures need to be developed that focus on concepts important for use of any type of eHealth innovation. At present, there are important gaps in the availability of psychometrically sound measures to evaluate eHealth technologies.
| S-EPMC5656774 | biostudies-literature
Project description:ObjectiveTo identify and assess the psychometric properties of patient-reported outcome measures (PROMs) of health-related quality of life (HRQoL) in people living with HIV (PLWH).MethodsNine databases were searched from January 1996 to October 2020. Methodological quality was assessed by using the Consensus-based Standards for the Selection of Health Measurement Instruments (COSMIN) Risk of Bias Checklist. We used the COSMIN criteria to summarize and rate the psychometric properties of each PROM. A modified Grading, Recommendations, Assessment, Development, and Evaluation (GRADE) system was used to assess the certainty of evidence.ResultsSixty-nine studies reported on the psychometric properties of 30 identified instruments. All studies were considered to have adequate methodological quality in terms of content validity, construct validity, and internal consistency. Limited information was retrieved on cross-cultural validity, criterion validity, reliability, hypothesis testing, and responsiveness. High-quality evidence on psychometric properties was provided for the Medical Outcomes Study HIV Health Survey (MOS-HIV), the brief version of the World Health Organization's Quality of Life Instrument in HIV Infection (WHOQoL-HIV-BREF), 36-Item Short Form Survey (SF-36), Multidimensional Quality of Life Questionnaire for Persons with HIV/AIDS (MQoL-HIV), and WHOQoL-HIV.ConclusionsThe findings from the included studies highlighted that among HIV-specific and generic HRQoL PROMs, MOS-HIV, WHOQoL-HIV-BREF, SF-36, MQoL-HIV, and WHOQoL-HIV are strongly recommended to evaluate HRQoL in PLWH in research and clinics based on the specific aims of assessments and the response burden for participants.
| S-EPMC8744327 | biostudies-literature
Project description:OBJECTIVES:This systematic review aimed to (a) provide an overview of existing quality measures in the field of oral health care, and to (b) evaluate the scientific soundness and applicability of these quality measures. METHODS:A systematic search was conducted in three electronic databases MEDLINE (via PubMed), EMBASE (via OVID) and LILACS (via BIREME). The search was restricted to articles published between 2002 and 2018. Publications reporting on the development process or clinimetric properties of oral health care quality measures for outpatient oral health care in dental practices were included. The identified publications reporting on oral health care quality measures were critically appraised with the Appraisal of Indicators through Research and Evaluation 2.0 (AIRE 2.0) instrument to evaluate the soundness and applicability of the measures. RESULTS:The search strategy resulted in 2541 unique and potentially relevant articles. In total, 24 publications were included yielding 215 quality measures. The critical appraisal showed a large variation in the quality of the included publications (AIRE scores ranging from 38 to 78 out of 80 possible points). The majority of measures (n = 71) referred to treatment and preventive services. Comparably, few measures referred to the domain patient safety (n = 3). The development process of measures often exhibited a lack of involvement of patients and dental professionals. Few projects reported on the validity (n = 2) and reliability (n = 3) of the measures. Four projects piloted the measures for implementation in practice. CONCLUSIONS:This systematic review provides an overview of the status quo with respect to existing quality measures in oral health care. Potential opportunities include the piloting and testing of quality measures and the establishment of suitable information systems that allow the provision of transparent routine feedback on the quality of oral health care.
| S-EPMC7379624 | biostudies-literature
Project description:BackgroundAdult scoliosis represents a distinct subgroup of scoliosis patients for whom the diagnosis can have a large impact on their health-related quality of life (HR-QOL). Therefore, HR-QOL patient-reported outcome measures (PROMs) are essential to assess disease progression and the impact of interventions. The objective of this systematic review is to evaluate the measurement properties of HR-QOL PROMs in adult scoliosis patients.MethodsWe will conduct a literature search, from their inception onwards, of multiple electronic databases including AMED, CINAHL, EMBASE, Medline, PsychINFO and PubMed. The searches will be performed in two stages. For both stages of the search, participants will be aged 18 and over with a diagnosis of scoliosis. The primary outcome of interest in the stage one searches will be studies which use PROMs to investigate HR-QOL as defined by the Core Outcome Measures in Effectiveness Trials (COMET) taxonomy, the secondary outcome will be to assess the frequency of use of the various PROMs. In stage two, the primary outcome of interest will be studies which assess the measurement properties of the HR-QOL PROMs identified in stage one. No specific measurement property will be given priority. No planned secondary outcomes have been identified but will be reported if discovered. In stage one, the only restriction on study design will be the exclusion of systematic reviews. In Stage two the only restriction on study design will be the exclusion of full-text articles not available in the English language. Two reviewers will independently screen all citations and abstract data. Potential conflicts will be resolved through discussion. The study methodological quality (or risk of bias) will be appraised using the Consensus-based Standards for the selection of Health Measurement Instruments (COSMIN) checklist. The overall strength of the body of evidence will then be assessed using the Grading of Recommendations, Assessment, Development and Evaluation (GRADE) approach. A narrative synthesis will be provided with information presented in the main text and tables to summarise and explain the characteristics and findings of the included studies. The narrative synthesis will explore the evidence for currently used PROMs in adult scoliosis patients and any areas that require further study.DiscussionThe review will help clinicians and researchers identify a HR-QOL PROM for use in patients with adult scoliosis. Findings from the review will be published and disseminated through a peer-reviewed journal and conference presentations.Systematic review registrationThis systematic review has been registered with the International Prospective Register of Systematic Reviews (PROSPERO), reference number: CRD42020219437.
| S-EPMC8474779 | biostudies-literature