Dataset Information

Intraclass correlation - A discussion and demonstration of basic features.

ABSTRACT: A re-analysis of intraclass correlation (ICC) theory is presented together with Monte Carlo simulations of ICC probability distributions. A partly revised and simplified theory of the single-score ICC is obtained, together with an alternative and simple recipe for its use in reliability studies. Our main, practical conclusion is that in the analysis of a reliability study it is neither necessary nor convenient to start from an initial choice of a specified statistical model. Rather, one may impartially use all three single-score ICC formulas. A near equality of the three ICC values indicates the absence of bias (systematic error), in which case the classical (one-way random) ICC may be used. A consistency ICC larger than absolute agreement ICC indicates the presence of non-negligible bias; if so, classical ICC is invalid and misleading. An F-test may be used to confirm whether biases are present. From the resulting model (without or with bias) variances and confidence intervals may then be calculated. In presence of bias, both absolute agreement ICC and consistency ICC should be reported, since they give different and complementary information about the reliability of the method. A clinical example with data from the literature is given.

SUBMITTER: Liljequist D

PROVIDER: S-EPMC6645485 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Intraclass correlation - A discussion and demonstration of basic features.

Liljequist David D Elfving Britt B Skavberg Roaldsen Kirsti K

PloS one 20190722 7

A re-analysis of intraclass correlation (ICC) theory is presented together with Monte Carlo simulations of ICC probability distributions. A partly revised and simplified theory of the single-score ICC is obtained, together with an alternative and simple recipe for its use in reliability studies. Our main, practical conclusion is that in the analysis of a reliability study it is neither necessary nor convenient to start from an initial choice of a specified statistical model. Rather, one may impa ...[more]

PMID: 31329615

Similar Datasets

Project description:Intraclass correlation (ICC) is a reliability metric that gauges similarity when, for example, entities are measured under similar, or even the same, well-controlled conditions, which in MRI applications include runs/sessions, twins, parent/child, scanners, sites, and so on. The popular definitions and interpretations of ICC are usually framed statistically under the conventional ANOVA platform. Here, we provide a comprehensive overview of ICC analysis in its prior usage in neuroimaging, and we show that the standard ANOVA framework is often limited, rigid, and inflexible in modeling capabilities. These intrinsic limitations motivate several improvements. Specifically, we start with the conventional ICC model under the ANOVA platform, and extend it along two dimensions: first, fixing the failure in ICC estimation when negative values occur under degenerative circumstance, and second, incorporating precision information of effect estimates into the ICC model. These endeavors lead to four modeling strategies: linear mixed-effects (LME), regularized mixed-effects (RME), multilevel mixed-effects (MME), and regularized multilevel mixed-effects (RMME). Compared to ANOVA, each of these four models directly provides estimates for fixed effects and their statistical significances, in addition to the ICC estimate. These new modeling approaches can also accommodate missing data and fixed effects for confounding variables. More importantly, we show that the MME and RMME approaches offer more accurate characterization and decomposition among the variance components, leading to more robust ICC computation. Based on these theoretical considerations and model performance comparisons with a real experimental dataset, we offer the following general-purpose recommendations. First, ICC estimation through MME or RMME is preferable when precision information (i.e., weights that more accurately allocate the variances in the data) is available for the effect estimate; when precision information is unavailable, ICC estimation through LME or the RME is the preferred option. Second, even though the absolute agreement version, ICC(2,1), is presently more popular in the field, the consistency version, ICC(3,1), is a practical and informative choice for whole-brain ICC analysis that achieves a well-balanced compromise when all potential fixed effects are accounted for. Third, approaches for clear, meaningful, and useful result reporting in ICC analysis are discussed. All models, ICC formulations, and related statistical testing methods have been implemented in an open source program 3dICC, which is publicly available as part of the AFNI suite. Even though our work here focuses on the whole-brain level, the modeling strategy and recommendations can be equivalently applied to other situations such as voxel, region, and network levels.

Project description:BACKGROUND: In multicenter studies, center-specific variations in measurements may arise for various reasons, such as low interrater reliability, differences in equipment, deviations from the protocol, sociocultural characteristics, and differences in patient populations due to e.g. local referral patterns. The aim of this research is to derive measures for the degree of clustering. We present a method to detect heavily clustered variables and to identify physicians with outlying measurements. METHODS: We use regression models with fixed effects to account for patient case-mix and a random cluster intercept to study clustering by physicians. We propose to use the residual intraclass correlation (RICC), the proportion of residual variance that is situated at the cluster level, to detect variables that are influenced by clustering. An RICC of 0 indicates that the variance in the measurements is not due to variation between clusters. We further suggest, where appropriate, to evaluate RICC in combination with R2, the proportion of variance that is explained by the fixed effects. Variables with a high R2 may have benefits that outweigh the disadvantages of clustering in terms of statistical analysis. We apply the proposed methods to a dataset collected for the development of models for ovarian tumor diagnosis. We study the variability in 18 tumor characteristics collected through ultrasound examination, 4 patient characteristics, and the serum marker CA-125 measured by 40 physicians on 2407 patients. RESULTS: The RICC showed large variation between variables: from 2.2% for age to 25.1% for the amount of fluid in the pouch of Douglas. Seven variables had an RICC above 15%, indicating that a considerable part of the variance is due to systematic differences at the physician level, rather than random differences at the patient level. Accounting for differences in ultrasound machine quality reduced the RICC for a number of blood flow measurements. CONCLUSIONS: We recommend that the degree of data clustering is addressed during the monitoring and analysis of multicenter studies. The RICC is a useful tool that expresses the degree of clustering as a percentage. Specific applications are data quality monitoring and variable screening prior to the development of a prediction model.

Project description:Background Reasonable personalized surgical design can achieve good treatment results for Helveston syndrome in one surgery, reducing the psychological and economic burden on patients. This article aims to explore the clinical characteristics of Helveston syndrome and the clinical effectiveness and feasibility of individualized surgical design. Methods In this retrospective case series study, 28 patients who underwent strabismus correction for Helveston syndrome at the Affiliated Hospital of Yunnan University from June 2018 to December 2020 with complete follow-up data were enrolled. Preoperatively, all patients received standard assessment of vision, intraocular pressure, slit lamp, fundus and refractive status, excluding other eye diseases, as well as detailed special examination of strabismus. These patients were divided into two groups according to the surgical modality: the horizontal muscle surgery alone group and the horizontal muscle surgery combined with superior oblique muscle surgery (combined surgery) group. We used SPSS software for data analysis and compared the postoperative eye position, eye movement, success rate, and reoperation rate between these two groups. Clinical measurement data were compared and analyzed with Fisher’s exact test for count data, the t-test for normally distributed measurement data, and the Mann-Whitney U test for non-normally distributed measurement data. P<0.05 was considered statistically significant. Results This study included a total of 28 patients with Helveston syndrome, including 20 males and 8 females. The average age at the time of surgery is 12.04 ± 8.67 years (range, 4–43 years). The postoperative A-pattern degree was significantly greater in the group undergoing horizontal muscle surgery alone [6.23±1.31 prism diopters (PD); range, 0–10 PD] than in the group undergoing combined surgery (0.53±0.32 PD; range, 0–4 PD; P=0.002). Superior oblique muscle overactivity was significantly reduced in the combined surgery group (0.20±0.11+; range, 0–1+) compared to the horizontal muscle surgery alone group (1.31±0.26+; range, 0–2+; P=0.002). However, there was no significant difference in success rate or reoperation rate between the two groups. Additionally, after combining the recession of the superior rectus muscle with the horizontal muscle, the number of A-pattern degrees was greatly reduced. Conclusions Helveston syndrome can be improved using a personalized surgical design according to the degree of external strabismus A-pattern, superior oblique muscle overaction, and dissociated vertical deviation (DVD) degree, which improves the success rate of single surgery.

Dataset Information

Intraclass correlation - A discussion and demonstration of basic features.

Publications

Intraclass correlation - A discussion and demonstration of basic features.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets