Assessing medical professionalism: A systematic review of instruments and their measurement properties.
Ontology highlight
ABSTRACT: Background
Over the last three decades, various instruments were developed and employed to assess medical professionalism, but their measurement properties have yet to be fully evaluated. This study aimed to systematically evaluate these instruments' measurement properties and the methodological quality of their related studies within a universally acceptable standardized framework and then provide corresponding recommendations.Methods
A systematic search of the electronic databases PubMed, Web of Science, and PsycINFO was conducted to collect studies published from 1990-2015. After screening titles, abstracts, and full texts for eligibility, the articles included in this study were classified according to their respective instrument's usage. A two-phase assessment was conducted: 1) methodological quality was assessed by following the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) checklist; and 2) the quality of measurement properties was assessed according to Terwee's criteria. Results were integrated using best-evidence synthesis to look for recommendable instruments.Results
After screening 2,959 records, 74 instruments from 80 existing studies were included. The overall methodological quality of these studies was unsatisfactory, with reasons including but not limited to unknown missing data, inadequate sample sizes, and vague hypotheses. Content validity, cross-cultural validity, and criterion validity were either unreported or negative ratings in most studies. Based on best-evidence synthesis, three instruments were recommended: Hisar's instrument for nursing students, Nurse Practitioners' Roles and Competencies Scale, and Perceived Faculty Competency Inventory.Conclusion
Although instruments measuring medical professionalism are diverse, only a limited number of studies were methodologically sound. Future studies should give priority to systematically improving the performance of existing instruments and to longitudinal studies.
SUBMITTER: Li H
PROVIDER: S-EPMC5428933 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Background:Professionalism, which encompasses behavioral, ethical, and related domains, is a core competency of medical practice. While observer-based instruments to assess medical professionalism are available, information on their psychometric properties and utility is limited. Objective:We systematically reviewed the psychometric properties and utility of existing observer-based instruments for assessing professionalism in medical trainees. Methods:After selecting eligible studies, we employed the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) criteria to score study methodological quality. We identified eligible instruments and performed quality assessment of psychometric properties for each selected instrument. We scored the utility of each instrument based on the ability to distinguish performance levels over time, availability of objective scoring criteria, validity evidence in medical students and residents, and instrument length. Results:Ten instruments from 16 studies met criteria for consideration, with studies having acceptable methodological quality. Psychometric properties were variably assessed. Among 10 instruments, the Education Outcomes Service (EOS) group questionnaire and Professionalism Mini-Evaluation Exercise (P-MEX) possessed the best psychometric properties, with the P-MEX scoring higher on utility than the EOS group questionnaire. Conclusions:We identified 2 instruments with best psychometric properties, with 1 also showing acceptable utility for assessing professionalism in trainees. The P-MEX may be an option for program directors to adopt as an observer-based instrument for formative assessment of medical professionalism. Further studies of the 2 instruments to aggregate additional validity evidence is recommended, particularly in the domain of content validity before they are used in specific cultural settings and in summative assessments.
| S-EPMC6314360 | biostudies-literature
Project description:BackgroundThe internet is now a major source of health information. With the growth of internet users, eHealth literacy has emerged as a new concept for digital health care. Therefore, health professionals need to consider the eHealth literacy of consumers when providing care utilizing digital health technologies.ObjectiveThis study aimed to identify currently available eHealth literacy instruments and evaluate their measurement properties to provide robust evidence to researchers and clinicians who are selecting an eHealth literacy instrument.MethodsWe conducted a systematic review and meta-analysis of self-reported eHealth literacy instruments by applying the updated COSMIN (COnsensus-based Standards for the selection of health Measurement INstruments) methodology.ResultsThis study included 7 instruments from 41 articles describing 57 psychometric studies, as identified in 4 databases (PubMed, CINAHL, Embase, and PsycInfo). No eHealth literacy instrument provided evidence for all measurement properties. The eHealth literacy scale (eHEALS) was originally developed with a single-factor structure under the definition of eHealth literacy before the rise of social media and the mobile web. That instrument was evaluated in 18 different languages and 26 countries, involving diverse populations. However, various other factor structures were exhibited: 7 types of two-factor structures, 3 types of three-factor structures, and 1 bifactor structure. The transactional eHealth literacy instrument (TeHLI) was developed to reflect the broader concept of eHealth literacy and was demonstrated to have a sufficient low-quality and very low-quality evidence for content validity (relevance, comprehensiveness, and comprehensibility) and sufficient high-quality evidence for structural validity and internal consistency; however, that instrument has rarely been evaluated.ConclusionsThe eHealth literacy scale was the most frequently investigated instrument. However, it is strongly recommended that the instrument's content be updated to reflect recent advancements in digital health technologies. In addition, the transactional eHealth literacy instrument needs improvements in content validity and further psychometric studies to increase the credibility of its synthesized evidence.
| S-EPMC8663713 | biostudies-literature
Project description:BackgroundIndependent mobility is a key factor in determining readiness for discharge for older patients following acute hospitalisation and has also been identified as a predictor of many important outcomes for this patient group. This review aimed to identify a physical performance instrument that is not disease specific that has the properties required to accurately measure and monitor the mobility of older medical patients in the acute hospital setting.MethodsDatabases initially searched were Medline, Cinahl, Embase, Cochrane Database of Systematic Reviews and the Cochrane Central Register of Controlled Trials without language restriction or limits on year of publication until July 2005. After analysis of this yield, a second step was the systematic search of Medline, Cinahl and Embase until August 2005 for evidence of the clinical utility of each potentially suitable instrument. Reports were included in this review if instruments described had face validity for measuring from bed bound to independent levels of ambulation, the items were suitable for application in an acute hospital setting and the instrument required observation (rather than self-report) of physical performance. Evidence of the clinical utility of each potentially suitable instrument was considered if data on measurement properties were reported.ResultsThree instruments, the Elderly Mobility Scale (EMS), Hierarchical Assessment of Balance and Mobility (HABAM) and the Physical Performance Mobility Examination (PPME) were identified as potentially relevant. Clinimetric evaluation indicated that the HABAM has the most desirable properties of these three instruments. However, the HABAM has the limitation of a ceiling effect in an older acute medical patient population and reliability and minimally clinically important difference (MCID) estimates have not been reported for the Rasch refined HABAM. These limitations support the proposal that a new mobility instrument is required for older acute medical patients.ConclusionNo existing instrument has the properties required to accurately measure and monitor mobility of older acute medical patients.
| S-EPMC2430553 | biostudies-literature
Project description:Permanent functional impairment (PFI) of the spine is a rating system used by compensation authorities, such as workers compensation boards, to establish an appropriate level of financial compensation for persistent loss of function. Determination of PFI of the spine is commonly based on the assessment of spinal movement combined with other measures of physical and functional impairments; however, the reliability and validity of the measurement instruments used for these evaluations have yet to be established. The aim of this study is to systematically review and synthesise the literature concerning measurement properties of the various and different instruments used for assessing PFI of the spine.Three conceptual groups of terms (1) PFI, (2) spinal disorder and (3) measurement properties will be combined to search Medline, EMBASE, CINAHL, Web of Science, Scopus, PEDro, OTSeeker and Health and Safety Science Abstracts. We will examine peer-reviewed, full-text articles over the full available date range. Two reviewers will independently screen citations (title, abstract and full text) and perform data extraction. Included studies will be appraised as to their methodological quality using the COnsensus-based Standards for the selection of health Measurement INstruments criteria. Findings will be summarised and presented descriptively, with meta-analysis pursued as appropriate.This review will summarise the current level of evidence of measurement properties of instruments used for assessing PFI of the spine. Findings of this review may be applicable to clinicians, policy-makers, workers' compensation boards, other insurers and health and safety organisations. The findings will likely provide a foundation and direction for future research priorities for assessing spinal PFI.CRD42017060390.
| S-EPMC5829857 | biostudies-other
Project description:AimThe aim of this study was to summarize and evaluate the empirical evidence on the measurement properties of diaper dermatitis (DD) measurement instruments in children.DesignSystematic review.MethodsMEDLINE, CINAHL and EMBASE were systematically searched until 14 June 2021. Citation searching was conducted in Scopus. The risk of bias, the reported measurement properties and the quality of evidence were evaluated using the COSMIN framework. The reporting follows the PRISMA 2020 statement.ResultsWe identified 1200 records in the databases and 108 records during citation searching and included four studies describing three measurement instruments for DD in children and their measurement properties. We considered the content validity inconsistent for all three instruments. The study authors reported internal consistency, reliability and construct validity for one instrument. We rated the quality of evidence from very low to moderate.
| S-EPMC10416076 | biostudies-literature
Project description:Policy Points: Investigations on systematic methodologies for measuring integrated care should coincide with the growing interest in this field of research. A systematic review of instruments provides insights into integrated care measurement, including setting the research agenda for validating available instruments and informing the decision to develop new ones. This study is the first systematic review of instruments measuring integrated care with an evidence synthesis of the measurement properties. We found 209 index instruments measuring different constructs related to integrated care; the strength of evidence on the adequacy of the majority of their measurement properties remained largely unassessed.Integrated care is an important strategy for increasing health system performance. Despite its growing significance, detailed evidence on the measurement properties of integrated care instruments remains vague and limited. Our systematic review aims to provide evidence on the state of the art in measuring integrated care.Our comprehensive systematic review framework builds on the Rainbow Model for Integrated Care (RMIC). We searched MEDLINE/PubMed for published articles on the measurement properties of instruments measuring integrated care and identified eligible articles using a standard set of selection criteria. We assessed the methodological quality of every validation study reported using the COSMIN checklist and extracted data on study and instrument characteristics. We also evaluated the measurement properties of each examined instrument per validation study and provided a best evidence synthesis on the adequacy of measurement properties of the index instruments.From the 300 eligible articles, we assessed the methodological quality of 379 validation studies from which we identified 209 index instruments measuring integrated care constructs. The majority of studies reported on instruments measuring constructs related to care integration (33%) and patient-centered care (49%); fewer studies measured care continuity/comprehensive care (15%) and care coordination/case management (3%). We mapped 84% of the measured constructs to the clinical integration domain of the RMIC, with fewer constructs related to the domains of professional (3.7%), organizational (3.4%), and functional (0.5%) integration. Only 8% of the instruments were mapped to a combination of domains; none were mapped exclusively to the system or normative integration domains. The majority of instruments were administered to either patients (60%) or health care providers (20%). Of the measurement properties, responsiveness (4%), measurement error (7%), and criterion (12%) and cross-cultural validity (14%) were less commonly reported. We found <50% of the validation studies to be of good or excellent quality for any of the measurement properties. Only a minority of index instruments showed strong evidence of positive findings for internal consistency (15%), content validity (19%), and structural validity (7%); with moderate evidence of positive findings for internal consistency (14%) and construct validity (14%).Our results suggest that the quality of measurement properties of instruments measuring integrated care is in need of improvement with the less-studied constructs and domains to become part of newly developed instruments.
| S-EPMC5192798 | biostudies-other
Project description:IntroductionHealth professionals are often involved in the process of breaking bad news (BBN), which remains a difficult challenge, as it requires not only theoretical knowledge, but also the development of humanistic, emotional and communication skills. Therefore, optimal BBN assessment is essential. In this regard, sound measurement instruments are needed to evaluate BBN properly in research, teaching and clinical settings. Several instruments have been designed and validated to assess BBN. In this context, choosing the most appropriate instrument for assessing health professionals' skills in BBN is essential. The aims of this systematic review are to: (1) identify all the instruments used for assessing health professionals' skills in BBN; and (2) critically appraise their measurement properties.MethodsA systematic review will be undertaken according to the most up-to-date COnsensus-based Standards for the selection of health status Measurement INstruments' (COSMIN) methodology. The protocol of this systematic review was developed using the Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidelines. The search strategy will be performed following the Peer Review of Electronic Search Strategies. The search strategy will be conducted in CINAHL, MEDLINE, Embase, PsycINFO, SciELO and Open Grey. Two review authors will independently appraise the full-text articles according to the COSMIN Risk of Bias checklist. Quality ratings and evidence synthesis will be performed using a modified Grading of Recommendations Assessment, Development and Evaluation approach.Ethics and disseminationEthical approval is not necessary for systematic review protocols. The results will be disseminated by publication in a peer-reviewed journal and presented at a relevant conference.Prospero registration numberCRD42020207586.
| S-EPMC8351499 | biostudies-literature
Project description:Although many instruments are used to assess the families of people with diabetes, their measurement properties have not been systematically reviewed. We aimed to identify and evaluate the psychometric properties of the instruments used to assess family functioning in adults with diabetes.MethodsA systematic literature review, according to the JBI systematic reviews of measurement properties, was conducted using different databases, including gray literature. PROSPERO registration number: CRD42021239733. Two independent reviewers searched, screened, and assessed the risk of bias among the articles according to the COSMIN methodology. The quality of each included instrument was assessed using the updated criteria for good measurement properties.ResultsEighty-one studies were included, and thirty-one eligible instruments were identified. The psychometric properties frequently assessed were structural validity, internal consistency, and construct validity.ConclusionsAlthough 31 instruments were included, none of their psychometric properties were scored as "very good". From the instruments scored as adequate on development and content validity, five stood out for their quality appraisal.. The development of new instruments is not recommended. More studies should be conducted on the existing instruments to assess the less commonly evaluated psychometric properties. Using valid instruments to develop and evaluate interventions is essential to promote health literacy and the effectiveness of diabetes management.
| S-EPMC9859216 | biostudies-literature
Project description:BackgroundWhiplash is a common traumatic cervical injury which is most often a consequence of rear-end motor vehicle accidents. It has been estimated that up to 50% of the whiplash patients suffer from chronic symptoms, resulting in extensive individual and societal burden. Several measurement instruments are used for initial assessment of whiplash and evaluation of response to treatment. However, a comprehensive assessment of the performance of these measures is lacking. Furthermore, there is no consensus on the most relevant outcome domains and their corresponding measurement instruments of choice. This systematic review aims to identify, describe, and critically appraise the performance properties of health-related measurement instruments in whiplash population.MethodsThe following literature databases will be searched from their date of establishment: PubMed, Embase®, MEDLINE, CINAHL Complete, PsycINFO, and HAPI. All original articles evaluating the reliability, validity, responsiveness, and feasibility of health-related measurement instruments in whiplash will be included, without additional restriction on their intended use, source of data, and structure. Risk of bias will be assessed using the COSMIN Risk of Bias checklist. Findings of the studies will be judged against the criteria for good measurement properties, and results from all studies will be qualitatively summarized to generate an overall quality of findings. Overall quality of evidence will be determined using a modified GRADE approach, which will be used in conjunction with the overall quality of results for generation of recommendations. Two reviewers will perform all steps of the review independently. Discrepancies will be discussed between the reviewers, and in case of remaining disagreement, the senior reviewer will make the final decision.DiscussionThis systematic review will summarize the body of literature on health-related measurement instruments in whiplash, aiming to facilitate the selection of high-quality measurement instrument for researchers and physicians. Findings of this study will guide the ongoing efforts for development of a core outcome set.Systematic review registrationPROSPERO reference number CRD42018070901.
| S-EPMC6688369 | biostudies-literature
Project description:BackgroundA plethora of outcome measurement instruments (OMIs) are being used in port wine stain (PWS) studies. It is currently unclear how valid, responsive, and reliable these are.ObjectivesThe aim of this systematic review was to appraise the content validity and other measurement properties of OMIs for PWS treatment to identify the most appropriate instruments and future research priorities.MethodsThis study was performed using the updated Consensus-Based Standards for the Selection of Health Measurement Instruments (COSMIN) methodology and adhered to PRISMA guidelines. Comprehensive searches in Medline and Embase were performed. Studies in which an OMI for PWS patients was developed or its measurement properties were evaluated were included. Two investigators independently extracted data and assessed the quality of included studies and instruments to perform qualitative synthesis of the evidence.ResultsIn total, 1,034 articles were screened, and 77 full-text articles were reviewed. A total of 8 studies were included that reported on 6 physician-reported OMIs of clinical improvement and 6 parent- or patient-reported OMIs of life impact, of which 3 for health-related quality of life and 1 for perceived stigmatization. Overall, the quality of OMI development was inadequate (63%) or doubtful (37%). Each instrument has undergone a very limited evaluation in PWS patients. No content validity studies were performed. The quality of evidence for content validity was very low (78%), low (15%), or moderate (7%), with sufficient comprehensibility, mostly sufficient comprehensiveness, and mixed relevance. No studies on responsiveness, minimal important change, and cross-cultural validity were retrieved. There was moderate- to very low-quality evidence for sufficient inter-rater reliability for some clinical PWS OMIs. Internal consistency and measurement error were indeterminate in all studies.ConclusionsThere was insufficient evidence to properly guide outcome selection. Additional assessment of the measurement properties of OMIs is needed, preferentially guided by a core domain set tailored to PWS.
| S-EPMC8117375 | biostudies-literature