Dataset Information

Inter-rater reliability of post-arrest cerebral performance category (CPC) scores.

ABSTRACT: Cerebral Performance Category (CPC) scores are often an outcome measure for post-arrest neurologic function, collected worldwide to compare performance, evaluate therapies, and formulate recommendations. At most institutions, no formal training is offered in their determination, potentially leading to misclassification.We identified 171 patients at 2 hospitals between 5/10/2005 and 8/31/2012 with two CPC scores at hospital discharge recorded independently - in an in-house quality improvement database and as part of a national registry. Scores were abstracted retrospectively from the same electronic medical record by two separate non-clinical researchers. These scores were compared to assess inter-rater reliability and stratified based on whether the score was concordant or discordant among reviewers to determine factors related to discordance.Thirty-nine CPC scores (22.8%) were discordant (kappa: 0.66), indicating substantial agreement. When dichotomized into "favorable" neurologic outcome (CPC 1-2)/"unfavorable" neurologic outcome (CPC 3-5), 20 (11.7%) scores were discordant (kappa: 0.70), also indicating substantial agreement. Patients discharged home (as opposed to nursing/other care facility) and patients with suspected cardiac etiology of arrest were statistically more likely to have concordant scores. For the quality improvement database, patients with discordant scores had a statistically higher median CPC score than those with concordant scores. The registry had statistically lower median CPC score (CPC 1) than the quality improvement database (CPC 2); p<0.01 for statistical significance.CPC scores have substantial inter-rater reliability, which is reduced in patients who have worse outcomes, have a non-cardiac etiology of arrest, and are discharged to a location other than home.

SUBMITTER: Grossestreuer AV

PROVIDER: S-EPMC5990009 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inter-rater reliability of post-arrest cerebral performance category (CPC) scores.

Grossestreuer Anne V AV Abella Benjamin S BS Sheak Kelsey R KR Cinousis Marisa J MJ Perman Sarah M SM Leary Marion M Wiebe Douglas J DJ Gaieski David F DF

Resuscitation 20160917

<h4>Purpose</h4>Cerebral Performance Category (CPC) scores are often an outcome measure for post-arrest neurologic function, collected worldwide to compare performance, evaluate therapies, and formulate recommendations. At most institutions, no formal training is offered in their determination, potentially leading to misclassification.<h4>Materials and methods</h4>We identified 171 patients at 2 hospitals between 5/10/2005 and 8/31/2012 with two CPC scores at hospital discharge recorded independ ...[more]

PMID: 27650863

Similar Datasets

Project description:Background: Arterial Doppler Ultrasound waveform (DW) analysis allows the detection and evaluation of lower extremity peripheral artery disease. The high heterogeneity of the reported description of DW is reduced by the use of classification. However, the reliability of these classifications is either unknown or low to moderate and practices of vascular caregivers regarding the use of these classifications remain unknown. Aims: This study aims to assess the inter-observer reliability of the Saint-Bonnet classification, a 13-category DW classification. The secondary objective was to determine the utilization rate of the most common classifications and the ability of these classifications to describe DW. Methods: A national survey was conducted among all vascular physicians of French society of vascular medicine. They were invited by email to describe on a website 20 DW without and with the display of the Saint-Bonnet classification. The reliability of this classification was estimated by Fleiss' Kappa expressed with [95% confidence interval]. A semantic analysis allowed us to classify the physicians' responses according to the terms used. Finally we have evaluated for each classification the rate of misuse, i.e., the addition of a complementary term to the defined categories. Results: One hundred and ten physicians participated and only 5% of these were familiar with Saint-Bonnet classification. Fifty-four percent of vascular physicians used no classification at all. Vascular physicians used the Spronk (four-category), Descotes (five-category) and Saint-Bonnet (13-category) classifications for respectively, 31, 10, and 5%. Kappa coefficient of Fleiss (κ) was 0.546 [0.544-0.547] (p < 0.001). Reliability by category ranges from κ of 0.075 to 0.864. In multivariate analysis, the use of a classification was associated with fewer years of experience and was dependent on geographic location. Misuse rate by classification was 88, 82, and 5% using Spronk, Descotes and Saint-Bonnet classifications respectively. Conclusion: The reliability of Saint-Bonnet classification is weak to moderate by vascular physicians who are not familiar with its use. However, unlike the other classifications, it seems to be sufficiently precise so that the user does not need to complete its description. There is a significant heterogeneity in the use of arterial Doppler classifications in France.

Project description:AimDespite the impact of spasticity, there is a lack of objective, clinically reliable and valid tools for its assessment. This study aims to evaluate the reliability of various performance- and spasticity-related parameters collected with a manually controlled instrumented spasticity assessment in four lower limb muscles in children with cerebral palsy (CP).MethodThe lateral gastrocnemius, medial hamstrings, rectus femoris and hip adductors of 12 children with spastic CP (12.8 years, ±4.13 years, bilateral/unilateral involvement n=7/5) were passively stretched in the sagittal plane at incremental velocities. Muscle activity, joint motion, and torque were synchronously recorded using electromyography, inertial sensors, and a force/torque load-cell. Reliability was assessed on three levels: (1) intra- and (2) inter-rater within session, and (3) intra-rater between session.ResultsParameters were found to be reliable in all three analyses, with 90% containing intra-class correlation coefficients >0.6, and 70% of standard error of measurement values <20% of the mean values. The most reliable analysis was intra-rater within session, followed by intra-rater between session, and then inter-rater within session. The Adds evaluation had a slightly lower level of reliability than that of the other muscles.ConclusionsLimited intrinsic/extrinsic errors were introduced by repeated stretch repetitions. The parameters were more reliable when the same rater, rather than different raters performed the evaluation. Standardisation and training should be further improved to reduce extrinsic error when different raters perform the measurement. Errors were also muscle specific, or related to the measurement set-up. They need to be accounted for, in particular when assessing pre-post interventions or longitudinal follow-up. The parameters of the instrumented spasticity assessment demonstrate a wide range of applications for both research and clinical environments in the quantification of spasticity.

Project description:Physical performance measures can be used to predict functional decline and increased dependency in older persons. However, few studies have assessed the feasibility or reliability of such measures in hospitalized older patients. Here we assessed the feasibility and inter-rater reliability of four simple measures of physical performance in acutely admitted older medical patients.During the first 24 hours of hospitalization, the following were assessed twice by different raters in 52 (? 65 years) patients admitted for acute medical illness: isometric hand grip strength, 4-meter gait speed, 30-s chair stand and Cumulated Ambulation Score. Relative reliability was expressed as weighted kappa for the Cumulated Ambulation Score or as intra-class correlation coefficient (ICC1,1) and lower limit of the 95%-confidence interval (LL95%) for grip strength, gait speed, and 30-s chair stand. Absolute reliability was expressed as the standard error of measurement and the smallest real difference as a percentage of their respective means (SEM% and SRD%).The primary reasons for admission of the 52 included patients were infectious disease and cardiovascular illness. The mean± SD age was 78±8.3 years, and 73.1% were women. All patients performed grip strength and Cumulated Ambulation Score testing, 81% performed the gait speed test, and 54% completed the 30-s chair stand test (46% were unable to rise without using the armrests). No systematic bias was found between first and second tests or between raters. The weighted kappa for the Cumulated Ambulation Score was 0.76 (0.60-0.92). The ICC1,1 values were as follows: grip strength, 0.95 (LL95% 0.92); gait speed, 0.92 (LL95% 0.73), and 30-s chair stand, 0.82 (LL95% 0.67). The SEM% values for grip strength, gait speed, and 30-s chair stand were 8%, 7%, and 18%, and the SRD95% values were 22%, 17%, and 49%.In acutely admitted older medical patients, grip strength, gait speed, and the Cumulated Ambulation Score measurements were feasible and showed high inter-rater reliability when administered by different raters. The feasibility and inter-rater reliability of the 30-s chair stand were moderate, complicating the use of the 30-s chair stand in acutely admitted older medical patients. However, the predefined modified version of the chair stand test was both feasible and with high inter-rater reliability in this population.

Project description:BackgroundThe statistical validities of the official Japanese classifications of activities of daily living (ADLs), including bedriddenness ranks (BR) and cognitive function scores (CFS), have yet to be assessed. To this aim, we evaluated the ability of BR and CFS to assess ADLs using inter-rater reliability and criterion-related validity.MethodsNew inpatients aged ≥75 years were enrolled in this hospital-based prospective observational study. BR and CFS were assessed once by an attending nurse, and then by a social worker/medical clerk. We evaluated inter-rater reliability between different professions by calculating the concordance rate, kappa coefficient, Cronbach's α, and intraclass correlation coefficient. We also estimated the relationship of the Barthel Index and Katz Index with the BR and CFS using Spearman's correlation coefficients.ResultsFor the 271 patients enrolled, BR at the first assessment revealed 66 normal, 10 of J1, 15 of J2, 18 of A1, 31 of A2, 37 of B1, 35 of B2, 22 of C1, and 32 of C2. The concordance rate between the two BR assessments was 68.6%, with a kappa coefficient of 0.61, Cronbach's α of 0.91, and an intraclass correlation coefficient of 0.83, thus showing good inter-rater reliability. BR was negatively correlated with the Barthel Index (r = - 0.848, p < 0.001) and Katz Index (r = - 0.820, p < 0.001), showing justifiable criterion-related validity. Meanwhile, CFS at the first assessment revealed 92 normal, 47 of 1, 19 of 2a, 30 of 2b, 60 of 3a, 8 of 3b, 8 of 4, and 0 of M. The concordance rate between the two CFS assessments was 70.1%, with a kappa coefficient of 0.62, Cronbach's α of 0.87, and an intraclass correlation coefficient 0.78, thus also showing good inter-rater reliability. CFS was negatively correlated with the Barthel Index (r = - 0.667, p < 0.001) and Katz Index (r = - 0.661, p < 0.001), showing justifiable criterion-related validity.ConclusionsBR and CFS could be reliable and easy-to-use grading scales of ADLs in acute clinical practice or large-scale screening, with high inter-rater reliabilities among different professions and significant correlations with well-established, though complicated to use, instruments to assess ADLs.Trial registrationUMIN000041051 (2020/7/10).

Project description:ObjectivesEvaluation of the inter-rater reliability of clinical assessment methods for pelvic floor muscles and diastasis recti abdominis post partum.DesignA multicentre inter-rater reliability study.SettingThree primary care rehabilitation centres in Sweden.ParticipantsA total of 222 participants were recruited via advertising at Swedish maternity care units and social media. Eligibility for participation included female gender, ≥18 years, at maximum 3 months after childbirth. Exclusion criteria were chronic pelvic girdle pain and/or low back pain and/or pelvic floor tear grade III/IV. At each centre, 2 physiotherapists, with training and experience in pelvic floor assessment, assessed the 222 women according to a standardised protocol in random order.Outcome measuresInter-rater reliability of the assessment of pelvic floor muscle function (involuntary and voluntary contraction and voluntary relaxation) and diastasis recti abdominis (width, depth and bulging).ResultsVaginal palpation of maximal voluntary contraction revealed a kappa value of 0.69 (95% CI 0.62 to 0.76). Assessments of involuntary contraction and voluntary relaxation yielded inconsistent results, with slight-to-moderate weighted kappa values ranging from 0.10 to 0.51. After 2 months of training in applying this method, diastasis recti abdominis width measured at the umbilicus by calliper yielded an intraclass correlation coefficient value of 0.83 (95% CI 0.76 to 0.87). Assessments of diastasis recti abdominis depth and bulging showed moderate kappa values, with reservation for some inconsistency between the centres.ConclusionsVaginal palpation of pelvic floor muscle strength is a reliable method for the postpartum muscle assessment. Additional research is needed to identify reliable assessment method for other pelvic floor muscle functions like involuntary contraction and voluntary relaxation. With some training, a calliper is a reliable instrument for measuring the postpartum diastasis recti abdominis width. This study provides novel thoughts about how to measure diastasis recti abdominis depth and bulging.Trial registration numberNCT03703804.

Project description:BackgroundTo build capacity in medicines management, the Uganda Ministry of Health introduced a nationwide supervision, performance assessment and recognition strategy (SPARS) in 2012. Medicines management supervisors (MMS) assess performance using 25 indicators to identify problems, focus supervision, and monitor improvement in medicines stock and storage management, ordering and reporting, and prescribing and dispensing. Although the indicators are well-recognized and used internationally, little was known about the reliability of these indicators. An initial assessment of inter-rater reliability (IRR), which measures agreement among raters (i.e., MMS), showed poor IRR; subsequently, we implemented efforts to improve IRR. The aim of this study was to assess IRR for SPARS indicators at two subsequent time points to determine whether IRR increased following efforts to improve reproducibility.MethodsIRR was assessed in 2011 and again after efforts to improve IRR in 2012 and 2013. Efforts included targeted training, providing detailed guidelines and job aids, and refining indicator definitions and response categories. In the assessments, teams of three MMS measured 24 SPARS indicators in 26 facilities. We calculated IRR as a team agreement score (i.e., percent of the MMS teams in which all three MMS had the same score). Two sample tests for proportions were used to compare IRR scores for each indicator, domain, and overall for the initial assessment and the following two assessments. We also compared the IRR scores for indicators classified as simple (binary) versus complex (multi-component). Logistic regression was used to identify supervisor group characteristics associated with domain-specific and overall IRR scores.ResultsInitially only five (21%) indicators had acceptable reproducibility, defined as an IRR score ≥ 75%. At the initial assessment, prescribing quality indicators had the lowest and stock management indicators had the highest IRR. By the third IRR assessment, 12 (50%) indicators had acceptable reproducibility, and the overall IRR score improved from 57% to 72%. The IRR of simple indicators was consistently higher than that of complex indicators in the three assessment periods. We found no correlation between IRR scores and MMS experience or professional background.ConclusionsAssessments of indicator reproducibility are needed to improve IRR. Using simple indicators is recommended.

Project description:Behavioral development in domestic dogs has been investigated for predicting suitability for service dog work or for matching with the "right" families as well as for identifying predispositions to behavioral problems. Findings from the scientific literature seem to confirm that conducting behavioral tests at 7 weeks of age is too early to reliably predict the temperament and personality of a dog. However, this period for domestic dogs is sensitive for early life learning and conditions during this time could have important consequences in adulthood. The aims of this study were to evaluate inter-rater reliability of a simple standardized test and to investigate which factors influence the behavioral reaction of puppies. 105 seven-week old puppies were exposed to five subtests: social attraction, following, retrieving, sudden appearance, noise. During each task, the behaviour of each pup was scored on a 3-5 point scale that reflected the suitability of the pup's reaction to the task. Scores were evaluated for a single subtest and for two aggregate indicators (i.e. response to a person: social attraction subtest and following subtest and response to object and noise: retrieving subtest, sudden appearance subtest and noise subtest). Three assessors independently scored the dogs' reactions for each task. Inter-rater reliability of the three assessors were analyzed with Fleiss' Kappa and Kendall's coefficient, which showed a high inter-rater reliability in 4 of 5 tasks. The ordered logistic regression was carried out to obtain a proportional odds model that was used to model the relationship between sex, litter size, stimulating environment, parity of mother, adequate maternal behavior and high scores. Litter size and maternal parity were associated with test performance in response to a person. The variance of effect of litter was high in response to object and noise. Taken together, our results suggest that using this scoring system there is sufficient inter-rater reliability in the test and litter size and mother experience influences task performances related to dog-human interaction.

Project description:Background: There has been a groundswell of national support for transparent tracking and dissemination of PhD career outcomes. In 2017, individuals from multiple institutions and professional organizations met to create the Unified Career Outcomes Taxonomy (UCOT 2017), a three-tiered taxonomy to help institutions uniformly classify career outcomes of PhD graduates. Early adopters of UCOT 2017, noted ambiguity in some categories of the career taxonomy, raising questions about its consistent application within and across institutions. Methods: To test and evaluate the consistency of UCOT 2017, we calculated inter-rater reliability across two rounds of iterative refinement of the career taxonomy, classifying over 800 PhD alumni records via nine coders. Results: We identified areas of discordance in the taxonomy, and progressively refined UCOT 2017 and an accompanying Guidance Document to improve inter-rater reliability across all three tiers of the career taxonomy. However, differing interpretations of the classifications, especially for faculty classifications in the third tier, resulted in continued discordance among the coders. We addressed this discordance with clarifying language in the Guidance Document, and proposed the addition of a flag system for identification of the title, rank, and prefix of faculty members. This labeling system provides the additional benefit of highlighting the granularity and the intersectionality of faculty job functions, while maintaining the ability to sort by - and report data on - faculty and postdoctoral trainee roles, as is required by some national and federal reporting guidelines. We provide specific crosswalk guidance for how a user may choose to incorporate our suggestions while maintaining the ability to report in accordance with UCOT 2017. Conclusions: Our findings underscore the importance of detailed guidance documents, coder training, and periodic collaborative review of career outcomes taxonomies as PhD careers evolve in the global workforce. Implications for coder-training and use of novice coders are also discussed.

Dataset Information

Inter-rater reliability of post-arrest cerebral performance category (CPC) scores.

Publications

Inter-rater reliability of post-arrest cerebral performance category (CPC) scores.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets