Methods of assessing categorical agreement between correlated screening tests in clinical studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhou TJ
PROVIDER: S-EPMC8299998 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND/AIMS:To estimate the level of agreement and positivity rates of latent tuberculosis infection (LTBI) tests prior to the use of tumor necrosis factor (TNF) inhibitors in relation to underlying rheumatic diseases and endemic tuberculosis levels. METHODS:The Ovid-Medline, Embase, and Cochrane Libraries were searched for articles before October 2013 involving LTBI screening in rheumatic patients, including rheumatoid arthritis (RA), ankylosing spondylitis (AS), juvenile idiopathic arthritis (JIA), and psoriatic arthritis. RESULTS:In pooled analyses, 5,224 rheumatic patients had undergone both a tuberculin skin test (TST) and an interferon-gamma release assay (IGRA) before TNF inhibitors use. The positivity of TST, QuantiFERON-TB Gold In Tube (QFT-GIT), and T-SPOT.TB (T-SPOT) tests were estimated to be 29%, 17%, and 18%, respectively. The agreement percentage between the TST and QFT-GIT, and between the TST and T-SPOT were 73% and 75%. Populations from low-to-moderate endemic TB presented with slightly less agreement (71% between TST and QFT-GIT, and 74% between TST and T-SPOT) than patients from high endemic countries (73% between TST and QFT-GIT, and 81% between TST and T-SPOT). By underlying disease stratification, a lower level of agreement between TST and QFT-GIT was found among AS (64%) than among JIA (77%) and RA patients (73%). CONCLUSION:We reaffirm the current evidence for accuracy of LTBI test done by TST and IGRA among rheumatic patients is inconsistent. Our stratified analysis suggests different screening strategies might be needed in clinical settings considering the endemic status in the patient's country of origin and the precise nature of underlying diseases.
| S-EPMC6234384 | biostudies-literature
Project description:IntroductionThe correlation between monofilament testing, symptom surveys, and electrodiagnostic studies for the diagnosis of axonal polyneuropathy has not been well studied. This investigation was done to assess the agreement between these procedures in a non-random sample of volunteers.MethodsThe procedures evaluated included electrodiagnostic tests of the sural nerve, monofilament testing of the great toe, a symptom survey, and a body diagram. Kappa coefficients and sensitivity and specificity, using nerve conduction as a "gold standard," were used to determine the agreement between various combinations of procedures.ResultsPoor agreement (kappa values -0.12-0.44) and sensitivity (sensitivity <30%) were found for all combinations of symptoms and monofilament results in comparison with sural peak latency and amplitude.ConclusionsOverall, the results demonstrated a low discriminatory power for the screening procedures for identifying persons with impaired sural nerve function. The results highlight the need for further development and evaluation of screening methods for distal neuropathy in population-based studies.
| S-EPMC3335757 | biostudies-literature
Project description:Over the last few years, miRNA microarray platforms have provided great insights into the biological mechanisms underlying the onset and development of several diseases. However, only a few studies have evaluated the concordance between different microarray platforms using methods that took into account measurement error in the data. In this work, we propose the use of a modified version of the Bland-Altman plot to assess agreement between microarray platforms. To this aim, two samples, one renal tumor cell line and a pool of 20 different human normal tissues, were profiled using three different miRNA platforms (Affymetrix, Agilent, Illumina) on triplicate arrays. Intra-platform reliability was assessed by calculating pair-wise concordance correlation coefficients (CCC) between technical replicates and overall concordance correlation coefficient (OCCC) with bootstrap percentile confidence intervals, which revealed moderate-to-good repeatability of all platforms for both samples. Modified Bland-Altman analysis revealed good patterns of concordance for Agilent and Illumina, whereas Affymetrix showed poor-to-moderate agreement for both samples considered. The proposed method is useful to assess agreement between array platforms by modifying the original Bland-Altman plot to let it account for measurement error and bias correction and can be used to assess patterns of concordance between other kinds of arrays other than miRNA microarrays.
| S-EPMC4979053 | biostudies-literature
Project description:Methods commonly used clinically to assess cardiac function in patients with heart failure include ejection fraction (EF), exercise treadmill testing (ETT), and symptom evaluation. Although these approaches are useful in evaluating patients with heart failure, there are at times substantial mismatches between individual assessments. For example, ETT results are often discordant with EF, and patients with minimal symptoms sometimes have surprisingly low EFs. To better define the relationship of these methods of assessment, we studied 56 patients with heart failure with reduced EF (HFrEF) who underwent measurement of ETT duration, EF by echocardiography, quantitative symptom evaluation, and LV peak dP/dt (rate of left ventricular pressure development and decline, measured invasively). Correlations were determined among these four tests in order to assess the relationship of EF, ETT, and symptoms against LV peak dP/dt. In addition, we sought to determine whether EF, ETT, and symptoms correlated with each other. Overall, correlations were poor. Only 15 of 63 total correlations (24%) were significant (p < 0.05). EF correlated most closely with LV peak -dP/dt. Linear regression analysis indicated that EF, ETT, and symptoms taken together predicted LV peak dP/dt better than any one measure alone. We conclude that clinical tests used to assess LV function in patients with HFrEF may not be as accurate or correlate as well as expected. All three clinical measures considered together may be the best representation of cardiac function in HFrEF patients currently available.
| S-EPMC4990285 | biostudies-literature
Project description:BackgroundSeveral criteria have been used to assess agreement between replicate slide readings of malaria parasite density. Such criteria may be based on percent difference, or absolute difference, or a combination. Neither the rationale for choosing between these types of criteria, nor that for choosing the magnitude of difference which defines acceptable agreement, are clear. The current paper seeks a procedure which avoids the disadvantages of these current options and whose parameter values are more clearly justified.Methods and resultsVariation of parasite density within a slide is expected, even when it has been prepared from a homogeneous sample. This places lower limits on sensitivity and observer agreement, quantified by the Poisson distribution. This means that, if a criterion of fixed percent difference criterion is used for satisfactory agreement, the number of discrepant readings is over-estimated at low parasite densities. With a criterion of fixed absolute difference, the same happens at high parasite densities. For an ideal slide, following the Poisson distribution, a criterion based on a constant difference in square root counts would apply for all densities. This can be back-transformed to a difference in absolute counts, which, as expected, gives a wider range of acceptable agreement at higher average densities. In an example dataset from Tanzania, observed differences in square root counts correspond to a 95% limits of agreement of -2,800 and +2,500 parasites/microl at average density of 2,000 parasites/microl, and -6,200 and +5,700 parasites/microl at 10,000 parasites/microl. However, there were more outliers beyond those ranges at higher densities, meaning that actual coverage of these ranges was not a constant 95%, but decreased with density. In a second study, a trial of microscopist training, the corresponding ranges of agreement are wider and asymmetrical: -8,600 to +5,200/microl, and -19,200 to +11,700/microl, respectively. By comparison, the optimal limits of agreement, corresponding to Poisson variation, are +/- 780 and +/- 1,800 parasites/microl, respectively. The focus of this approach on the volume of blood read leads to other conclusions. For example, no matter how large a volume of blood is read, some densities are too low to be reliably detected, which in turn means that disagreements on slide positivity may simply result from within-slide variation, rather than reading errors.ConclusionsThe proposed method defines limits of acceptable agreement in a way which allows for the natural increase in variability with parasite density. This includes defining the levels of between-reader variability, which are consistent with random variation: disagreements within these limits should not trigger additional readings. This approach merits investigation in other settings, in order to determine both the extent of its applicability, and appropriate numerical values for limits of agreement.
| S-EPMC2817705 | biostudies-literature
Project description:Measurement of respiratory muscles strength such as maximal inspiratory pressure (MIP) and maximal expiratory pressure (MEP) are used to detect, diagnose and treat respiratory weakness. However, devices used for these measurements are not widely available and are costly. Currently, the use of a digital manometer is recommended. In industry, several inexpensive devices are available, but these have not been validated for clinical use. Our objective was to determine the agreement between maximal respiratory pressures obtained with a clinical digital manometer and that with a non-clinical digital manometer in healthy volunteers. We assessed the height, weight, lung function, MIP, and MEP of healthy volunteers. To compare pressures obtained by each type of digital manometer, a parallel approach configuration was used. The agreement was measured with the Intraclass Coefficient Correlation (ICC) and the Bland-Altman plot. Twenty-seven participants (14 men) were recruited with a median age of 22 (range: 21-23) years. Each participant underwent three measurements to give a total of 81 measurements. The mean MIPs were 90.8 ± 26.4 (SEM 2.9) and 91.1 ± 26.4 (SEM 2.9) cmH2O for the clinical and non-clinical digital manometers, respectively. The mean MEPs were 113.8 ± 40.4 (SEM 4.5) and 114.5 ± 40.5 (SEM 4.5) cmH2O for the clinical and non-clinical digital manometers, respectively. We obtained an ICC of 0.998 (IC 0.997-0.999) for MIP and 0.999 (IC 0.998-0.999) for MEP. There is a high agreement in the values obtained for MIP and MEP between clinical and non-clinical digital manometers in healthy volunteers. Further validation at lower pressures and safety profiling among human subjects is needed.
| S-EPMC6812781 | biostudies-literature
Project description:In the analysis of pooled data from multiple studies involving a biomarker exposure, the biomarker measurements can vary across laboratories and usually require calibration to a reference assay prior to pooling. Previous researches consider the measurements from a reference laboratory as the gold standard, even though measurements in the reference laboratory are not necessarily closer to the underlying truth in reality. In this paper we do not treat any laboratory measurements as the gold standard, and we develop two statistical methods, the exact calibration and cut-off calibration methods, for the analysis of aggregated categorical biomarker data. We compare the performance of both methods for estimating the biomarker-disease relationship under a random sample or controls-only calibration design. Our findings include: (1) the exact calibration method provides significantly less biased estimates and more accurate confidence intervals than the other method; (2) the cut-off calibration method could yield estimates with minimal bias and valid confidence intervals under small measurement errors and/or small exposure effects; (3) controls-only calibration design can result in additional bias, but the bias is minimal if the exposure effects and/or disease prevalences are small. Finally, we illustrate the methods in an application evaluating the relationship between circulating vitamin D levels and colorectal cancer risk in a pooling project.
| S-EPMC7903924 | biostudies-literature
Project description:BACKGROUND:Data standardization is essential in electronic health records (EHRs) for both clinical practice and retrospective research. However, it is still not easy to standardize EHR data because of nonidentical duplicates, typographical errors, or inconsistencies. To overcome this drawback, standardization efforts have been undertaken for collecting data in a standardized format as well as for curating the stored data in EHRs. To perform clinical big data research, the stored data in EHR should be standardized, starting from laboratory results, given their importance. However, most of the previous efforts have been based on labor-intensive manual methods. OBJECTIVE:We aimed to develop an automatic standardization method for eliminating the noises of categorical laboratory data, grouping, and mapping of cleaned data using standard terminology. METHODS:We developed a method called standardization algorithm for laboratory test-categorical result (SALT-C) that can process categorical laboratory data, such as pos +, 250 4+ (urinalysis results), and reddish (urinalysis color results). SALT-C consists of five steps. First, it applies data cleaning rules to categorical laboratory data. Second, it categorizes the cleaned data into 5 predefined groups (urine color, urine dipstick, blood type, presence-finding, and pathogenesis tests). Third, all data in each group are vectorized. Fourth, similarity is calculated between the vectors of data and those of each value in the predefined value sets. Finally, the value closest to the data is assigned. RESULTS:The performance of SALT-C was validated using 59,213,696 data points (167,938 unique values) generated over 23 years from a tertiary hospital. Apart from the data whose original meaning could not be interpreted correctly (eg, ** and _^), SALT-C mapped unique raw data to the correct reference value for each group with accuracy of 97.6% (123/126; urine color tests), 97.5% (198/203; (urine dipstick tests), 95% (53/56; blood type tests), 99.68% (162,291/162,805; presence-finding tests), and 99.61% (4643/4661; pathogenesis tests). CONCLUSIONS:The proposed SALT-C successfully standardized the categorical laboratory test results with high reliability. SALT-C can be beneficial for clinical big data research by reducing laborious manual standardization efforts.
| S-EPMC6740165 | biostudies-literature
Project description:The Clinical Global Impressions of Improvement (CGI-I) scale is widely used in clinical trials to monitor clinically meaningful change during treatment. Although it is standard practice in research to have independent evaluators (IEs) complete the CGI-I, this approach is not practical in school and community settings. Few studies have explored the potential utility of other informants, such as youth and parents. Therefore, this study aimed to investigate agreement between IEs and both adolescents and parents in CGI-I improvement ratings in the context of a randomized controlled trial of cognitive-behavioral therapy for social anxiety disorder, as delivered by psychologists and school counselors. Multilevel growth models indicated that IEs were generally more conservative in their ratings of positive treatment response across time and treatment conditions, though greater agreement was observed between parents and IEs by post-intervention and 5-month follow-up. Possible explanations for these findings and suggestions for alternative approaches are discussed.
| S-EPMC5440211 | biostudies-literature
Project description:ObjectivesRadiomics utilizes quantitative image features (QIFs) to characterize tumor phenotype. In practice, radiological images are obtained from different vendors' equipment using various imaging acquisition settings. Our objective was to assess the inter-setting agreement of QIFs computed from CT images by varying two parameters, slice thickness and reconstruction algorithm.Materials and methodsCT images from an IRB-approved/HIPAA-compliant study assessing thirty-two lung cancer patients were included for the analysis. Each scan's raw data were reconstructed into six imaging series using combinations of two reconstruction algorithms (Lung[L] and Standard[S]) and three slice thicknesses (1.25mm, 2.5mm and 5mm), i.e., 1.25L, 1.25S, 2.5L, 2.5S, 5L and 5S. For each imaging-setting, 89 well-defined QIFs were computed for each of the 32 tumors (one tumor per patient). The six settings led to 15 inter-setting comparisons (combinatorial pairs). To reduce QIF redundancy, hierarchical clustering was done. Concordance correlation coefficients (CCCs) were used to assess inter-setting agreement of the non-redundant feature groups. The CCC of each group was assessed by averaging CCCs of QIFs in the group.ResultsTwenty-three non-redundant feature groups were created. Across all feature groups, the best inter-setting agreements (CCCs>0.8) were 1.25S vs 2.5S, 1.25L vs 2.5L, and 2.5S vs 5S; the worst (CCCs<0.51) belonged to 1.25L vs 5S and 2.5L vs 5S. Eight of the feature groups related to size, shape, and coarse texture had an average CCC>0.8 across all imaging settings.ConclusionsVarying degrees of inter-setting disagreements of QIFs exist when features are computed from CT images reconstructed using different algorithms and slice thicknesses. Our findings highlight the importance of harmonizing imaging acquisition for obtaining consistent QIFs to study tumor imaging phonotype.
| S-EPMC5199063 | biostudies-literature