Methods of assessing categorical agreement between correlated screening tests in clinical studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhou TJ
PROVIDER: S-EPMC8299998 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In clinical studies, it is often of interest to see the diagnostic agreement among clinicians on certain symptoms. Previous work has focused on the agreement between two clinicians under two different conditions or the agreement among multiple clinicians under one condition. Few have discussed the agreement study with a design where multiple clinicians examine the same group of patients under two different conditions. In this paper, we use the intraclass kappa statistic for assessing nominal scale agreement with such a design. We derive an explicit variance formula for the difference of correlated kappa statistics and conduct hypothesis testing for the equality of kappa statistics. Simulation studies show that the method performs well with realistic sample sizes and may be superior to a method that did not take into account the measurement dependence structure. The practical utility of the method is illustrated on data from an eosinophilic esophagitis (EoE) study.
| S-EPMC9631095 | biostudies-literature
Project description:Background/aimsTo estimate the level of agreement and positivity rates of latent tuberculosis infection (LTBI) tests prior to the use of tumor necrosis factor (TNF) inhibitors in relation to underlying rheumatic diseases and endemic tuberculosis levels.MethodsThe Ovid-Medline, Embase, and Cochrane Libraries were searched for articles before October 2013 involving LTBI screening in rheumatic patients, including rheumatoid arthritis (RA), ankylosing spondylitis (AS), juvenile idiopathic arthritis (JIA), and psoriatic arthritis.ResultsIn pooled analyses, 5,224 rheumatic patients had undergone both a tuberculin skin test (TST) and an interferon-gamma release assay (IGRA) before TNF inhibitors use. The positivity of TST, QuantiFERON-TB Gold In Tube (QFT-GIT), and T-SPOT.TB (T-SPOT) tests were estimated to be 29%, 17%, and 18%, respectively. The agreement percentage between the TST and QFT-GIT, and between the TST and T-SPOT were 73% and 75%. Populations from low-to-moderate endemic TB presented with slightly less agreement (71% between TST and QFT-GIT, and 74% between TST and T-SPOT) than patients from high endemic countries (73% between TST and QFT-GIT, and 81% between TST and T-SPOT). By underlying disease stratification, a lower level of agreement between TST and QFT-GIT was found among AS (64%) than among JIA (77%) and RA patients (73%).ConclusionWe reaffirm the current evidence for accuracy of LTBI test done by TST and IGRA among rheumatic patients is inconsistent. Our stratified analysis suggests different screening strategies might be needed in clinical settings considering the endemic status in the patient's country of origin and the precise nature of underlying diseases.
| S-EPMC6234384 | biostudies-literature
Project description:IntroductionThe correlation between monofilament testing, symptom surveys, and electrodiagnostic studies for the diagnosis of axonal polyneuropathy has not been well studied. This investigation was done to assess the agreement between these procedures in a non-random sample of volunteers.MethodsThe procedures evaluated included electrodiagnostic tests of the sural nerve, monofilament testing of the great toe, a symptom survey, and a body diagram. Kappa coefficients and sensitivity and specificity, using nerve conduction as a "gold standard," were used to determine the agreement between various combinations of procedures.ResultsPoor agreement (kappa values -0.12-0.44) and sensitivity (sensitivity <30%) were found for all combinations of symptoms and monofilament results in comparison with sural peak latency and amplitude.ConclusionsOverall, the results demonstrated a low discriminatory power for the screening procedures for identifying persons with impaired sural nerve function. The results highlight the need for further development and evaluation of screening methods for distal neuropathy in population-based studies.
| S-EPMC3335757 | biostudies-literature
Project description:Over the last few years, miRNA microarray platforms have provided great insights into the biological mechanisms underlying the onset and development of several diseases. However, only a few studies have evaluated the concordance between different microarray platforms using methods that took into account measurement error in the data. In this work, we propose the use of a modified version of the Bland-Altman plot to assess agreement between microarray platforms. To this aim, two samples, one renal tumor cell line and a pool of 20 different human normal tissues, were profiled using three different miRNA platforms (Affymetrix, Agilent, Illumina) on triplicate arrays. Intra-platform reliability was assessed by calculating pair-wise concordance correlation coefficients (CCC) between technical replicates and overall concordance correlation coefficient (OCCC) with bootstrap percentile confidence intervals, which revealed moderate-to-good repeatability of all platforms for both samples. Modified Bland-Altman analysis revealed good patterns of concordance for Agilent and Illumina, whereas Affymetrix showed poor-to-moderate agreement for both samples considered. The proposed method is useful to assess agreement between array platforms by modifying the original Bland-Altman plot to let it account for measurement error and bias correction and can be used to assess patterns of concordance between other kinds of arrays other than miRNA microarrays.
| S-EPMC4979053 | biostudies-literature
Project description:Methods commonly used clinically to assess cardiac function in patients with heart failure include ejection fraction (EF), exercise treadmill testing (ETT), and symptom evaluation. Although these approaches are useful in evaluating patients with heart failure, there are at times substantial mismatches between individual assessments. For example, ETT results are often discordant with EF, and patients with minimal symptoms sometimes have surprisingly low EFs. To better define the relationship of these methods of assessment, we studied 56 patients with heart failure with reduced EF (HFrEF) who underwent measurement of ETT duration, EF by echocardiography, quantitative symptom evaluation, and LV peak dP/dt (rate of left ventricular pressure development and decline, measured invasively). Correlations were determined among these four tests in order to assess the relationship of EF, ETT, and symptoms against LV peak dP/dt. In addition, we sought to determine whether EF, ETT, and symptoms correlated with each other. Overall, correlations were poor. Only 15 of 63 total correlations (24%) were significant (p < 0.05). EF correlated most closely with LV peak -dP/dt. Linear regression analysis indicated that EF, ETT, and symptoms taken together predicted LV peak dP/dt better than any one measure alone. We conclude that clinical tests used to assess LV function in patients with HFrEF may not be as accurate or correlate as well as expected. All three clinical measures considered together may be the best representation of cardiac function in HFrEF patients currently available.
| S-EPMC4990285 | biostudies-literature
Project description:BackgroundSeveral criteria have been used to assess agreement between replicate slide readings of malaria parasite density. Such criteria may be based on percent difference, or absolute difference, or a combination. Neither the rationale for choosing between these types of criteria, nor that for choosing the magnitude of difference which defines acceptable agreement, are clear. The current paper seeks a procedure which avoids the disadvantages of these current options and whose parameter values are more clearly justified.Methods and resultsVariation of parasite density within a slide is expected, even when it has been prepared from a homogeneous sample. This places lower limits on sensitivity and observer agreement, quantified by the Poisson distribution. This means that, if a criterion of fixed percent difference criterion is used for satisfactory agreement, the number of discrepant readings is over-estimated at low parasite densities. With a criterion of fixed absolute difference, the same happens at high parasite densities. For an ideal slide, following the Poisson distribution, a criterion based on a constant difference in square root counts would apply for all densities. This can be back-transformed to a difference in absolute counts, which, as expected, gives a wider range of acceptable agreement at higher average densities. In an example dataset from Tanzania, observed differences in square root counts correspond to a 95% limits of agreement of -2,800 and +2,500 parasites/microl at average density of 2,000 parasites/microl, and -6,200 and +5,700 parasites/microl at 10,000 parasites/microl. However, there were more outliers beyond those ranges at higher densities, meaning that actual coverage of these ranges was not a constant 95%, but decreased with density. In a second study, a trial of microscopist training, the corresponding ranges of agreement are wider and asymmetrical: -8,600 to +5,200/microl, and -19,200 to +11,700/microl, respectively. By comparison, the optimal limits of agreement, corresponding to Poisson variation, are +/- 780 and +/- 1,800 parasites/microl, respectively. The focus of this approach on the volume of blood read leads to other conclusions. For example, no matter how large a volume of blood is read, some densities are too low to be reliably detected, which in turn means that disagreements on slide positivity may simply result from within-slide variation, rather than reading errors.ConclusionsThe proposed method defines limits of acceptable agreement in a way which allows for the natural increase in variability with parasite density. This includes defining the levels of between-reader variability, which are consistent with random variation: disagreements within these limits should not trigger additional readings. This approach merits investigation in other settings, in order to determine both the extent of its applicability, and appropriate numerical values for limits of agreement.
| S-EPMC2817705 | biostudies-literature
Project description:Measurement of respiratory muscles strength such as maximal inspiratory pressure (MIP) and maximal expiratory pressure (MEP) are used to detect, diagnose and treat respiratory weakness. However, devices used for these measurements are not widely available and are costly. Currently, the use of a digital manometer is recommended. In industry, several inexpensive devices are available, but these have not been validated for clinical use. Our objective was to determine the agreement between maximal respiratory pressures obtained with a clinical digital manometer and that with a non-clinical digital manometer in healthy volunteers. We assessed the height, weight, lung function, MIP, and MEP of healthy volunteers. To compare pressures obtained by each type of digital manometer, a parallel approach configuration was used. The agreement was measured with the Intraclass Coefficient Correlation (ICC) and the Bland-Altman plot. Twenty-seven participants (14 men) were recruited with a median age of 22 (range: 21-23) years. Each participant underwent three measurements to give a total of 81 measurements. The mean MIPs were 90.8 ± 26.4 (SEM 2.9) and 91.1 ± 26.4 (SEM 2.9) cmH2O for the clinical and non-clinical digital manometers, respectively. The mean MEPs were 113.8 ± 40.4 (SEM 4.5) and 114.5 ± 40.5 (SEM 4.5) cmH2O for the clinical and non-clinical digital manometers, respectively. We obtained an ICC of 0.998 (IC 0.997-0.999) for MIP and 0.999 (IC 0.998-0.999) for MEP. There is a high agreement in the values obtained for MIP and MEP between clinical and non-clinical digital manometers in healthy volunteers. Further validation at lower pressures and safety profiling among human subjects is needed.
| S-EPMC6812781 | biostudies-literature
Project description:In this study, we have further extended the methodology proposed, first, by Lin et al. (2002) and, later, extended by Stevens et al. (2017, 2018), on the coverage probability/probability of agreement, by relaxing the strong parametric assumptions regarding the distribution of the latent trait and developing inference methods allowing to compute both pointwise and simultaneous confidence bands. The methodology requires repeated measurements by at least one of the two measurement methods and accommodates heteroscedastic measurement errors. It performs often very well even when one has only one measurement by one of the two measurement methods and at least five repeated measurements from the other. It circumvents some of the deficiencies of the Bland & Altman limits of agreement method and provides a more direct assessment of the agreement level.
| S-EPMC9814023 | biostudies-literature
Project description:BackgroundChronic lung disease of prematurity (CLD) is the most prevalent complication of preterm birth and indicates an increased likelihood of long-term pulmonary complications. The accurate diagnosis of this condition is critical for long-term health management. Numerous definitions define CLD with different clinical parameters and radiology findings, making diagnosis of the disease ambiguous and potentially inaccurate.Methods95 patients were identified for this study, as determined by the diagnosis or confirmation of CLD in the impression of the radiologist's report on chest x-ray. Pulmonary function and complications were recorded at multiple benchmark timeframes within each patient's first few months of life and used for determining eligibility under each definition.ResultsEach clinical definition of CLD had a high sensitivity for patients identified to have CLD by radiologists, correctly fitting over 90% of patients. Most patients included required invasive mechanical ventilation or positive pressure ventilation at 36 weeks postmenstrual age, indicating patients with radiographically confirmed CLD tended to have more severe disease. Radiologists tended to diagnose CLD before 36 weeks postmenstrual age, a timepoint used by multiple standard clinical definitions, with cases called earlier fitting under a larger percentage of definitions than those called later.ConclusionsRadiologists tend to diagnose CLD in young patients with severe respiratory compromise, and can accurately diagnose the condition before developmental milestones for clinical definitions are met.
| S-EPMC10996264 | biostudies-literature
Project description:In the analysis of pooled data from multiple studies involving a biomarker exposure, the biomarker measurements can vary across laboratories and usually require calibration to a reference assay prior to pooling. Previous researches consider the measurements from a reference laboratory as the gold standard, even though measurements in the reference laboratory are not necessarily closer to the underlying truth in reality. In this paper we do not treat any laboratory measurements as the gold standard, and we develop two statistical methods, the exact calibration and cut-off calibration methods, for the analysis of aggregated categorical biomarker data. We compare the performance of both methods for estimating the biomarker-disease relationship under a random sample or controls-only calibration design. Our findings include: (1) the exact calibration method provides significantly less biased estimates and more accurate confidence intervals than the other method; (2) the cut-off calibration method could yield estimates with minimal bias and valid confidence intervals under small measurement errors and/or small exposure effects; (3) controls-only calibration design can result in additional bias, but the bias is minimal if the exposure effects and/or disease prevalences are small. Finally, we illustrate the methods in an application evaluating the relationship between circulating vitamin D levels and colorectal cancer risk in a pooling project.
| S-EPMC7903924 | biostudies-literature