Review of instruments for peer assessment of physicians.
Ontology highlight
ABSTRACT: Objectives
To identify existing instruments for rating peers (professional colleagues) in medical practice and to evaluate them in terms of how they have been developed, their validity and reliability, and their appropriateness for use in clinical settings, including primary care.Design
Systematic literature review.Data sources
Electronic search techniques, snowball sampling, and correspondence with specialists.Study selection
The peer assessment instruments identified were evaluated in terms of how they were developed and to what extent, if relevant, their psychometric properties had been determined.Results
A search of six electronic databases identified 4566 possible articles. After appraisal of the abstracts and in depth assessment of 42 articles, three rating scales fulfilled the inclusion criteria and were fully appraised. The three instruments did not meet established standards of instrument development, as no reference was made to a theoretical framework and the published psychometric data omitted essential work on construct and criterion validity. Rater training was absent, and guidance consisted of short written instructions. Two instruments were developed for a hospital setting in the United States and one for a primary care setting in Canada.Conclusions
The instruments developed to date for physicians to evaluate characteristics of colleagues need further assessment of validity before their widespread use is merited.
SUBMITTER: Evans R
PROVIDER: S-EPMC416602 | biostudies-literature | 2004 May
REPOSITORIES: biostudies-literature
Publications
Review of instruments for peer assessment of physicians.
Evans Richard R Elwyn Glyn G Edwards Adrian A
BMJ (Clinical research ed.) 20040501 7450
<h4>Objectives</h4>To identify existing instruments for rating peers (professional colleagues) in medical practice and to evaluate them in terms of how they have been developed, their validity and reliability, and their appropriateness for use in clinical settings, including primary care.<h4>Design</h4>Systematic literature review.<h4>Data sources</h4>Electronic search techniques, snowball sampling, and correspondence with specialists.<h4>Study selection</h4>The peer assessment instruments ident ...[more]
Similar Datasets
Project description:BackgroundVariation in assessments is a universal given, and work disability assessments by insurance physicians are no exception. Little is known about the considerations and views of insurance physicians that may partly explain such variation. On the basis of the Attitude--Social norm--self Efficacy (ASE) model, we have developed measurement instruments for assessment behaviour and its determinants.MethodsBased on theory and interviews with insurance physicians the questionnaire included blocks of items concerning background variables, intentions, attitudes, social norms, self-efficacy, knowledge, barriers and behaviour of the insurance physicians in relation to work disability assessment issues. The responses of 231 insurance physicians were suitable for further analysis. Factor analysis and reliability analysis were used to form scale variables and homogeneity analysis was used to form dimension variables. Thus, we included 169 of the 177 original items.ResultsFactor analysis and reliability analysis yielded 29 scales with sufficient reliability. Homogeneity analysis yielded 19 dimensions. Scales and dimensions fitted with the concepts of the ASE model. We slightly modified the ASE model by dividing behaviour into two blocks: behaviour that reflects the assessment process and behaviour that reflects assessment behaviour.The picture that emerged from the descriptive results was of a group of physicians who were motivated in their job and positive about the Dutch social security system in general. However, only half of them had a positive opinion about the Dutch Work and Income (Capacity for Work) Act (WIA). They also reported serious barriers, the most common of which was work pressure. Finally, 73% of the insurance physicians described the majority of their cases as 'difficult'.ConclusionsThe scales and dimensions developed appear to be valid and offer a promising basis for future research. The results suggest that the underlying ASE model, in modified form, is suitable for describing the assessment behaviour of insurance physicians and the determinants of this behaviour. The next step in this line of research should be to validate the model using structural equation modelling. Finally, the predictive value should be tested in relation to outcome measurements of work disability assessments.
| S-EPMC3086528 | biostudies-literature
Project description:ObjectiveTelephone-based cognitive assessment (TBCA) has long been studied but less widely adopted in routine neuropsychological practice. Increased interest in remote neuropsychological assessment techniques in the face of the coronavirus 2019 (COVID-19) pandemic warrants an updated review of relevant remote assessment literature. While recent reviews of videoconference-based neuropsychological applications have been published, no updated compilation of empirical TBCA research has been completed. Therefore, this scoping review offers relevant empirical research to inform clinical decision-making specific to teleneuropsychology.MethodPeer-reviewed studies addressing TBCA were included. Broad search terms were related to telephone, cognitive, or neuropsychological assessment and screening. After systematic searching of the PubMed and EBSCO databases, 139 relevant articles were retained.ResultsIn total, 17 unique cognitive screening measures, 20 cognitive batteries, and 6 single-task measures were identified as being developed or adapted specifically for telephone administration. Tables summarizing the identified cognitive assessments, information on diagnostic accuracy, and comparisons to face-to-face cognitive assessment are included in supplementary materials.ConclusionsOverall, literature suggests that TBCA is a viable modality for identifying cognitive impairment in various populations. However, the mode of assessment selected clinically should reflect an understanding of the purpose, evidence, and limitations for various tests and populations. Most identified measures were developed for research application to support gross cognitive characterization and to help determine when more comprehensive testing was needed. While TBCA is not meant to replace gold-standard, face-to-face evaluation, if appropriately utilized, it can expand scope of practice, particularly when barriers to standard neuropsychological assessment occur.
| S-EPMC7665291 | biostudies-literature
Project description:ImportanceMeasurement of delirium severity has been recognized as highly important for tracking prognosis, monitoring response to treatment, and estimating burden of care for patients both during and after hospitalization. Rather than simply rating delirium as present or absent, the ability to quantify its severity would enable development and monitoring of more effective treatment approaches for the condition.ObjectivesTo present a comprehensive review of delirium severity instruments, conduct a methodologic quality rating of the original validation study of the most commonly used instruments, and select a group of top-rated instruments.Evidence reviewThis systematic review was conducted using literature from Embase, PsycINFO, PubMed, Web of Science, and Cumulative Index to Nursing and Allied Health Literature, from January 1, 1974, through March 31, 2017, with the key words delirium, severity, tests, measures, and intensity. Inclusion criteria were original articles assessing delirium severity and using a delirium-specific severity instrument. Final listings of articles were supplemented with hand searches of reference listings to ensure completeness. At least 2 reviewers independently completed each step of the review process: article selection, data extraction, and methodologic quality assessment of relevant articles using a validated rating scale. All discrepancies between raters were resolved by consensus.FindingsOf 9409 articles identified, 228 underwent full text review, and we identified 42 different instruments of delirium severity. Eleven of the 42 tools were multidomain, delirium-specific instruments providing a quantitative rating of delirium severity; these instruments underwent a methodologic quality review. Applying prespecified criteria related to frequency of use, methodologic quality, construct or predictive validity, and broad domain coverage, an expert panel used an iterative modified Delphi process to select 6 final high-quality instruments meeting these criteria: the Confusion Assessment Method-Severity Score, Confusional State Examination, Delirium-O-Meter, Delirium Observation Scale, Delirium Rating Scale, and Memorial Delirium Assessment Scale.Conclusions and relevanceThe 6 instruments identified may enable accurate measurement of delirium severity to improve clinical care for patients with this condition. This work may stimulate increased usage and head-to-head comparison of these instruments.
| S-EPMC6382582 | biostudies-literature
Project description:Although many instruments are used to assess the families of people with diabetes, their measurement properties have not been systematically reviewed. We aimed to identify and evaluate the psychometric properties of the instruments used to assess family functioning in adults with diabetes.MethodsA systematic literature review, according to the JBI systematic reviews of measurement properties, was conducted using different databases, including gray literature. PROSPERO registration number: CRD42021239733. Two independent reviewers searched, screened, and assessed the risk of bias among the articles according to the COSMIN methodology. The quality of each included instrument was assessed using the updated criteria for good measurement properties.ResultsEighty-one studies were included, and thirty-one eligible instruments were identified. The psychometric properties frequently assessed were structural validity, internal consistency, and construct validity.ConclusionsAlthough 31 instruments were included, none of their psychometric properties were scored as "very good". From the instruments scored as adequate on development and content validity, five stood out for their quality appraisal.. The development of new instruments is not recommended. More studies should be conducted on the existing instruments to assess the less commonly evaluated psychometric properties. Using valid instruments to develop and evaluate interventions is essential to promote health literacy and the effectiveness of diabetes management.
| S-EPMC9859216 | biostudies-literature
Project description:UNLABELLED:We systematically reviewed the literature on the performance of osteoporosis absolute fracture risk assessment instruments. Relatively few studies have evaluated the calibration of instruments in populations separate from their development cohorts, and findings are mixed. Many studies had methodological limitations making susceptibility to bias a concern. INTRODUCTION:The aim of this study was to systematically review the literature on the performance of osteoporosis clinical fracture risk assessment instruments for predicting absolute fracture risk, or calibration, in populations other than their derivation cohorts. METHODS:We performed a systematic review, and MEDLINE, Embase, Cochrane Library, and multiple other literature sources were searched. Inclusion and exclusion criteria were applied and data extracted, including information about study participants, study design, potential sources of bias, and predicted and observed fracture probabilities. RESULTS:A total of 19,949 unique records were identified for review. Fourteen studies met inclusion criteria. There was substantial heterogeneity among included studies. Six studies assessed the WHO's Fracture Risk Assessment (FRAX) instrument in five separate cohorts, and a variety of risk assessment instruments were evaluated in the remainder of the studies. Approximately half found good instrument calibration, with observed fracture probabilities being close to predicted probabilities for different risk categories. Studies that assessed the calibration of FRAX found mixed performance in different populations. A similar proportion of studies that evaluated simple risk assessment instruments (?5 variables) found good calibration when compared with studies that assessed complex instruments (>5 variables). Many studies had methodological features making them susceptible to bias. CONCLUSIONS:Few studies have evaluated the performance or calibration of osteoporosis fracture risk assessment instruments in populations separate from their development cohorts. Findings are mixed, and many studies had methodological limitations making susceptibility to bias a possibility, raising concerns about use of these tools outside of the original derivation cohorts. Further studies are needed to assess the calibration of instruments in different populations prior to widespread use.
| S-EPMC3962543 | biostudies-literature
Project description:ObjectivesWe (1) collected instruments that assess health-related quality of life (HRQoL), activities of daily living (ADL) and social participation during follow-up after polytrauma, (2) described their use and (3) investigated other relevant patient-reported outcomes (PROs) assessed in the studies.DesignSystematic Review using the Preferred Reporting Items for Systematic Review and Meta-Analysis guideline.Data sourcesMEDLINE, Embase, CINAHL, PsycINFO, CENTRAL, as well as the trials registers ClinicalTrials.gov and WHO ICTRP were searched from January 2005 to April 2018.Eligibility criteriaAll original empirical research published in English or German including PROs of patients aged 18-75 years with an Injury Severity Score≥16 and/or an Abbreviated Injury Scale≥3. Studies with defined injuries or diseases (e.g. low-energy injuries) and some text types (e.g. grey literature and books) were excluded. Systematic reviews and meta-analyses were excluded, but references screened for appropriate studies.Data extraction and synthesisData extraction, narrative content analysis and a critical appraisal (e.g. UK National Institute for Health and Care Excellence) were performed by two reviewers independently.ResultsThe search yielded 3496 hits; 54 publications were included. Predominantly, HRQoL was assessed, with Short Form-36 Health Survey applied most frequently. ADL and (social) participation were rarely assessed. The methods most used were postal surveys and single assessments of PROs, with a follow-up period of one to one and a half years. Other relevant PRO areas reported were function, mental disorders and pain.ConclusionsThere is a large variation in the assessment of PROs after polytrauma, impairing comparability of outcomes. First efforts to standardise the collection of PROs have been initiated, but require further harmonisation between central players. Additional knowledge on rarely reported PRO areas (e.g. (social) participation, social networks) may lead to their consideration in health services provision.Prospero registration numberCRD42017060825.
| S-EPMC8679059 | biostudies-literature
Project description:ObjectivesFormative peer assessment focuses on learning and development of the student learning process. This implies that students are taking responsibility for assessing the work of their peers by giving and receiving feedback to each other. The aim was to compile research about formative peer assessment presented in higher healthcare education, focusing on the rationale, the interventions, the experiences of students and teachers and the outcomes of formative assessment interventions.DesignA scoping review.Data sourcesSearches were conducted until May 2019 in PubMed, Cumulative Index to Nursing and Allied Health Literature, Education Research Complete and Education Research Centre. Grey literature was searched in Library Search, Google Scholar and Science Direct.Eligibility criteriaStudies addressing formative peer assessment in higher education, focusing on medicine, nursing, midwifery, dentistry, physical or occupational therapy and radiology published in peer-reviewed articles or in grey literature.Data extractions and synthesisOut of 1452 studies, 37 met the inclusion criteria and were critically appraised using relevant Critical Appraisal Skills Programme, Joanna Briggs Institute and Mixed Methods Appraisal Tool tools. The pertinent data were analysed using thematic analysis.ResultThe critical appraisal resulted in 18 included studies with high and moderate quality. The rationale for using formative peer assessment relates to giving and receiving constructive feedback as a means to promote learning. The experience and outcome of formative peer assessment interventions from the perspective of students and teachers are presented within three themes: (1) organisation and structure of the formative peer assessment activities, (2) personal attributes and consequences for oneself and relationships and (3) experience and outcome of feedback and learning.ConclusionHealthcare education must consider preparing and introducing students to collaborative learning, and thus develop well-designed learning activities aligned with the learning outcomes. Since peer collaboration seems to affect students' and teachers' experiences of formative peer assessment, empirical investigations exploring collaboration between students are of utmost importance.
| S-EPMC7875268 | biostudies-literature
Project description:Peer review is the gold standard for scientific communication, but its ability to guarantee the quality of published research remains difficult to verify. Recent modeling studies suggest that peer review is sensitive to reviewer misbehavior, and it has been claimed that referees who sabotage work they perceive as competition may severely undermine the quality of publications. Here we examine which aspects of suboptimal reviewing practices most strongly impact quality, and test different mitigating strategies that editors may employ to counter them. We find that the biggest hazard to the quality of published literature is not selfish rejection of high-quality manuscripts but indifferent acceptance of low-quality ones. Bypassing or blacklisting bad reviewers and consulting additional reviewers to settle disagreements can reduce but not eliminate the impact. The other editorial strategies we tested do not significantly improve quality, but pairing manuscripts to reviewers unlikely to selfishly reject them and allowing revision of rejected manuscripts minimize rejection of above-average manuscripts. In its current form, peer review offers few incentives for impartial reviewing efforts. Editors can help, but structural changes are more likely to have a stronger impact.
| S-EPMC5633172 | biostudies-other
Project description:BackgroundOlder people are commonly prescribed complex multi-drug regimens while also experiencing declines in the cognitive and physical abilities required for medication management, leading to increased risk of medication errors and need for assisted living. The purpose of this study was to review published instruments designed to assess patients' capacity to self-administer medications.MethodsSearches of Medline, EMBASE, CINAHL, PsycINFO, International Pharmaceutical Abstracts, Health and Psychosocial Instruments, Google, and reference lists of identified publications were conducted to identify English-language articles describing development and validation of instruments designed to assess patients' capacity to self-administer medications. Methodological quality of validation studies was rated independently against published criteria by two reviewers and reliability and validity data were reviewed.ResultsThirty-two instruments were identified, of which 14 met pre-defined inclusion criteria. Instruments fell into two categories: those that used patients' own medications as the basis for assessment and those that used a simulated medication regimen. The quality of validation studies was generally low to moderate and few instruments were subjected to reliability testing. Most instruments had some evidence of construct validity, through associations with tests of cognitive function, health literacy, activities of daily living or measures of medication management or adherence. Only one instrument had sensitivity and specificity data with respect to prediction of medication-related outcomes such as adherence to therapy. Only three instruments had validity data from more than one independent research group.ConclusionA number of performance-based instruments exist to assess patients' capacity to manage their own medications. These may be useful for identifying physical and cognitive barriers to successful medication management, but further studies are needed to determine whether they are able to accurately and reliably predict medication outcomes.
| S-EPMC2719637 | biostudies-literature
Project description:BackgroundSickle cell disease (SCD) is a chronic condition associated with high mortality and morbidity. It is characterized by acute clinical symptoms such as painful vaso-occlusive crises, which can impair health-related quality of life (HRQL). This study was conducted to identify validated patient-reported outcome (PRO) instruments for use in future trials of potential treatments for SCD.MethodsA systematic literature review (SLR) was performed using MEDLINE and EMBASE to identify United States (US)-based studies published in English between 1997 and 2017 that reported on validated PRO instruments used in randomized controlled trials and real-world settings. The COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist was used to assess the quality of PRO instruments.ResultsThe SLR included 21 studies assessing the psychometric properties of 24 PRO instruments. Fifteen of those instruments were developed and validated for adults and 10 for children (one instrument was used in both children and young adults aged up to 21 years). Only five of the 15 adult instruments and three of the 10 pediatric instruments were developed specifically for SCD. For most instruments, there were few or no data on validation conducted in SCD development cohorts. Of the 24 PRO instruments identified, 16 had strong internal reliability (Cronbach's α ≥0.80). There was often insufficient information to assess the content validity, construct validity, responsiveness, or test-retest reliability of the instruments identified for both child and adult populations. No validated PRO instruments measuring caregiver burden in SCD were identified.ConclusionsThe evidence on the psychometric properties of PRO instruments was limited. However, the results of this SLR provide key information on such tools to help inform the design of future clinical trials for patients with SCD in the US.
| S-EPMC5963009 | biostudies-literature