Dataset Information

Inter-rater agreement of the Quality of Life-Alzheimer's Disease (QoL-AD) self-rating and proxy rating scale: secondary analysis of RightTimePlaceCare data.

ABSTRACT:

Background

To assess the quality of life of people with dementia, measures are required for self-rating by the person with dementia, and for proxy rating by others. The Quality of Life in Alzheimer's Disease scale (QoL-AD) is available in two versions, QoL-AD-SR (self-rating) and QoL-AD-PR (proxy rating). The aim of our study was to analyse the inter-rater agreement between self- and proxy ratings, in terms of both the total score and the items, including an analysis specific to care setting, and to identify factors associated with this agreement.

Methods

Cross-sectional QoL-AD data from the 7th Framework European RightTimePlaceCare study were analysed. A total of 1330 cases were included: n?=?854 receiving home care and n?=?476 receiving institutional long-term nursing care. The proxy raters were informal carers (home care) and best-informed professional carers (institutional long-term nursing care). Inter-rater agreement was investigated using Bland-Altman plots for the QoL-AD total score and by weighted kappa statistics for single items. Associations were investigated by regression analysis.

Results

The overall QoL-AD assessment of those with dementia revealed a mean value of 33.2 points, and the proxy ratings revealed a mean value of 29.8 points. The Bland-Altman plots revealed a poor agreement between self- and proxy ratings for the overall sample and for both care settings. With one exception (item 'Marriage' weighted kappa 0.26), the weighted kappa values for the single QoL-AD items were below 0.20, indicating poor agreement. Home care setting, dementia-related behavioural and psychological symptoms, and the functional status of the person with dementia, along with the caregiver burden, were associated with the level of agreement. Only the home care setting was associated with an increase larger than the predefined acceptable difference between self- and proxy ratings.

Conclusions

Proxy quality of life ratings from professional and informal carers appear to be lower than the self-ratings of those with dementia. QoL-AD-SR and QoL-AD-PR are therefore not interchangeable, as the inter-rater agreement differs distinctly. Thus, a proxy rating should be judged as a complementary perspective for a self-assessment of quality of life by those with dementia, rather than as a valid substitute.

SUBMITTER: Romhild J

PROVIDER: S-EPMC6022444 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Inter-rater agreement of the Quality of Life-Alzheimer's Disease (QoL-AD) self-rating and proxy rating scale: secondary analysis of RightTimePlaceCare data.

Römhild Josephine J Fleischer Steffen S Meyer Gabriele G Stephan Astrid A Zwakhalen Sandra S Leino-Kilpi Helena H Zabalegui Adelaida A Saks Kai K Soto-Martin Maria M Sutcliffe Caroline C Rahm Hallberg Ingalill I Berg Almuth A

Health and quality of life outcomes 20180628 1

<h4>Background</h4>To assess the quality of life of people with dementia, measures are required for self-rating by the person with dementia, and for proxy rating by others. The Quality of Life in Alzheimer's Disease scale (QoL-AD) is available in two versions, QoL-AD-SR (self-rating) and QoL-AD-PR (proxy rating). The aim of our study was to analyse the inter-rater agreement between self- and proxy ratings, in terms of both the total score and the items, including an analysis specific to care set ...[more]

PMID: 29954384

Similar Datasets

Project description:Studies published in general and specialty medical journals have the potential to improve emergency medicine (EM) practice, but there can be delayed awareness of this evidence because emergency physicians (EPs) are unlikely to read most of these journals. Also, not all published studies are intended for or ready for clinical practice application. The authors developed "Best Evidence in Emergency Medicine" (BEEM) to ameliorate these problems by searching for, identifying, appraising, and translating potentially practice-changing studies for EPs. An initial step in the BEEM process is the BEEM rater scale, a novel tool for EPs to collectively evaluate the relative clinical relevance of EM-related studies found in more than 120 journals. The BEEM rater process was designed to serve as a clinical relevance filter to identify those studies with the greatest potential to affect EM practice. Therefore, only those studies identified by BEEM raters as having the highest clinical relevance are selected for the subsequent critical appraisal process and, if found methodologically sound, are promoted as the best evidence in EM.The primary objective was to measure inter-rater reliability (IRR) of the BEEM rater scale. Secondary objectives were to determine the minimum number of EP raters needed for the BEEM rater scale to achieve acceptable reliability and to compare performance of the scale against a previously published evidence rating system, the McMaster Online Rating of Evidence (MORE), in an EP population.The authors electronically distributed the title, conclusion, and a PubMed link for 23 recently published studies related to EM to a volunteer group of 134 EPs. The volunteers answered two demographic questions and rated the articles using one of two randomly assigned seven-point Likert scales, the BEEM rater scale (n = 68) or the MORE scale (n = 66), over two separate administrations. The IRR of each scale was measured using generalizability theory.The IRR of the BEEM rater scale ranged between 0.90 (95% confidence interval [CI] = 0.86 to 0.93) to 0.92 (95% CI = 0.89 to 0.94) across administrations. Decision studies showed a minimum of 12 raters is required for acceptable reliability of the BEEM rater scale. The IRR of the MORE scale was 0.82 to 0.84.The BEEM rater scale is a highly reliable, single-question tool for a small number of EPs to collectively rate the relative clinical relevance within the specialty of EM of recently published studies from a variety of medical journals. It compares favorably with the MORE system because it achieves a high IRR despite simply requiring raters to read each article's title and conclusion.

Project description:Objectives To explore agreement among healthcare professionals assessing eligibility for work disability benefits.Design Systematic review and narrative synthesis of reproducibility studies.Data sources Medline, Embase, and PsycINFO searched up to 16 March 2016, without language restrictions, and review of bibliographies of included studies.Eligibility criteria Observational studies investigating reproducibility among healthcare professionals performing disability evaluations using a global rating of working capacity and reporting inter-rater reliability by a statistical measure or descriptively. Studies could be conducted in insurance settings, where decisions on ability to work include normative judgments based on legal considerations, or in research settings, where decisions on ability to work disregard normative considerations. : Teams of paired reviewers identified eligible studies, appraised their methodological quality and generalisability, and abstracted results with pretested forms. As heterogeneity of research designs and findings impeded a quantitative analysis, a descriptive synthesis stratified by setting (insurance or research) was performed.Results From 4562 references, 101 full text articles were reviewed. Of these, 16 studies conducted in an insurance setting and seven in a research setting, performed in 12 countries, met the inclusion criteria. Studies in the insurance setting were conducted with medical experts assessing claimants who were actual disability claimants or played by actors, hypothetical cases, or short written scenarios. Conditions were mental (n=6, 38%), musculoskeletal (n=4, 25%), or mixed (n=6, 38%). Applicability of findings from studies conducted in an insurance setting to real life evaluations ranged from generalisable (n=7, 44%) and probably generalisable (n=3, 19%) to probably not generalisable (n=6, 37%). Median inter-rater reliability among experts was 0.45 (range intraclass correlation coefficient 0.86 to ?-0.10). Inter-rater reliability was poor in six studies (37%) and excellent in only two (13%). This contrasts with studies conducted in the research setting, where the median inter-rater reliability was 0.76 (range 0.91-0.53), and 71% (5/7) studies achieved excellent inter-rater reliability. Reliability between assessing professionals was higher when the evaluation was guided by a standardised instrument (23 studies, P=0.006). No such association was detected for subjective or chronic health conditions or the studies' generalisability to real world evaluation of disability (P=0.46, 0.45, and 0.65, respectively).Conclusions Despite their common use and far reaching consequences for workers claiming disabling injury or illness, research on the reliability of medical evaluations of disability for work is limited and indicates high variation in judgments among assessing professionals. Standardising the evaluation process could improve reliability. Development and testing of instruments and structured approaches to improve reliability in evaluation of disability are urgently needed.

Project description:Objectives: It has been recommended that clinical trials of Traditional Chinese Medicine (TCM) would be more ecologically valid if its characteristic mode of diagnostic reasoning were integrated into their design. In that context, however, it is also widely held that demonstrating a high level of agreement on initial TCM diagnoses is necessary for the replicability that the biomedical paradigm requires for the conclusions from such trials. Our aim was to review, summarize, and critique quantitative experimental studies of inter-rater agreement in TCM, and some of their underlying assumptions. Design: Systematic electronic searches were conducted for articles that reported a quantitative measure of inter-rater agreement across a number of rating choices based on examinations of human subjects in person by TCM practitioners, and published in English language peer-reviewed journals. Publications in languages other than English were not included, nor those appearing in other than peer-reviewed journals. Predefined categories of information were extracted from full texts by two investigators working independently. Each article was scored for methodological quality. Outcome measures: Design features across all studies and levels of inter-rater agreement across studies that reported the same type of outcome statistic were compared. Results: Twenty-one articles met inclusion criteria. Fourteen assessed inter-rater agreement on TCM diagnoses, two on diagnostic signs found upon traditional TCM examination, and five on novel rating schemes derived from TCM theory and practice. Raters were students of TCM colleges or graduates of TCM training programs with 3 or more years experience and licensure. Type of outcome statistic varied. Mean rates of pairwise agreement averaged 57% (median 65, range 19-96) across the 9 studies reporting them. Mean Cohen's kappa averaged 0.34 (median 0.34, range 0.07-0.59) across the seven studies reporting them. Meta-analysis was not possible due to variations in study design and outcome statistics. High risks of bias and confounding, and deficits in statistical reporting were common. Conclusions: With a few exceptions, the levels of agreement were low to moderate. Most studies had significant deficits of both methodology and reporting. Results overall suggest a few design features that might contribute to higher levels of agreement. These should be studied further with better experimental controls and more thorough reporting of outcomes. In addition, methods of complex systems analysis should be explored to more adequately model the relationship between clinical outcomes, and the series of diagnoses and treatments that are the norm in actual TCM practice.

Project description:BackgroundNo specific scale to measure Quality of Life in Alzheimer's Disease in Nursing Homes (QoL-AD NH) exists in French. We aimed to translate and culturally adapt the QoL-AD NH participant scale into a French version and evaluate its psychometric properties with residents in French nursing homes (EHPAD).MethodsFirst, the QoL-AD NH was cross-culturally adapted into French according to guidelines. Secondly, a convenience group of residents with mild to moderate dementia answered the Folstein's test and the QoL-AD NH. They also answered the Dementia Quality of Life and the Geriatric Depression Scale to test convergent and divergent validity. Known-group validity was tested with a comparison group of residents without dementia. Exploratory Structural Equation Modeling (ESEM) was used after Exploratory Factor Analysis (EFA) to identify factors and measure invariance across age and mental state groups. Reliability (internal consistency, McDonald's omega and test-retest) were also measured.ResultsFollowing successful adaptation of the QoL-AD NH, 174 residents (mean age 86.6) from 7 nursing homes with mild to moderate dementia participated in the validation study. We retained a 3-factor model of the scale after ESEM identifying: "Intra & interpersonal environment-related QoL", "Self-functioning-related QoL" and "Perceived current health-related QoL" that were invariant across age and mental state groups. The QoL-AD NH had acceptable convergent (ρ range 0.24-0.53) and divergent validity (ρ range - 0.43 to - 0.57) and good known-group validity with 33 residents without dementia (t(205) = 2.70, p = .007). For reliability, the results revealed very good and adequate internal consistency (α = 0.86 for total scale and ≥ 0.71 for subscales). All total omega values exceeded the threshold 0.70. The hierarchical omega was 0.50, supporting the multidimensionality of the scale. Hierarchical omega subscale values exceeded the minimal level 0.50 except for the third factor, although reliable, would deserve more items. Test-retest was good with ICC (3,1) = 0.76.ConclusionsThe QoL-AD NH French participant version has globally good reliability and validity for evaluating residents' quality of life. However, further studies must rework and confirm the factor structure, test sensitivity to change and responsiveness.

Project description:OBJECTIVE:Surgical site infection (SSI) is the most common nosocomial infection, in vascular surgery patients, who experience a high rate of readmission. Facilitating transition from hospital to outpatient care with digital image-based wound monitoring has the potential to detect and to enable treatment of SSI at an early stage. In this study, we evaluated whether smartphone digital images can supplant in-person evaluation of postoperative vascular surgery wounds. METHODS:We developed a wound assessment checklist using previously validated criteria. We recruited adults who underwent a vascular surgical procedure between 2014 and 2015, involving an incision of at least 3 cm, from a high-volume academic vascular surgery service. Vascular surgery care providers evaluated wounds in person using the assessment checklist; a different group of providers evaluated wounds by a smartphone digital image. Inter-rater agreement coefficients for wound characteristics and treatment plan were calculated within and between the in-person group and the digital image group; the sensitivity and specificity of digital images relative to in-person evaluation were determined. RESULTS:We assessed a total of 80 wounds. Regardless of modality, inter-rater agreement was poor when wounds were evaluated for the presence of ecchymosis and redness; moderate for cellulitis; and high for the presence of a drain, necrosis, or dehiscence. As expected, the presence of drainage was more readily observed in person. Inter-rater agreement was high for both in-person and image-based assessment with respect to course of treatment, with near-perfect agreement for treatments ranging from antibiotics to surgical débridement to hospital readmission. No difference in agreement emerged when raters evaluated poor-quality compared with high-quality images. For most parameters, specificity was higher than sensitivity for image-based compared with "gold standard" in-person assessment. CONCLUSIONS:Using smartphone digital images is a valid method for evaluating postoperative vascular surgery wounds and is comparable to in-person evaluation with regard to most wound characteristics. The inter-rater reliability for determining treatment recommendations was universally high. Remote wound monitoring and assessment may play an integral role in future transitional care models to decrease readmission for SSI in vascular or other surgical patients. These findings will inform smartphone implementation in the clinical care setting as wound images transition from informal clinical communication to becoming part of the care standard.

Dataset Information

Inter-rater agreement of the Quality of Life-Alzheimer's Disease (QoL-AD) self-rating and proxy rating scale: secondary analysis of RightTimePlaceCare data.

Background

Methods

Results

Conclusions

Publications

Inter-rater agreement of the Quality of Life-Alzheimer's Disease (QoL-AD) self-rating and proxy rating scale: secondary analysis of RightTimePlaceCare data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure