Illustrating potential effects of alternate control populations on real-world evidence-based statistical analyses.
Ontology highlight
ABSTRACT: Objective
Case-control study designs are commonly used in retrospective analyses of real-world evidence (RWE). Due to the increasingly wide availability of RWE, it can be difficult to determine whether findings are robust or the result of testing multiple hypotheses.Materials and methods
We investigate the potential effects of modifying cohort definitions in a case-control association study between depression and type 2 diabetes mellitus. We used a large (>75 million individuals) de-identified administrative claims database to observe the effects of minor changes to the requirements of glucose and hemoglobin A1c tests in the control group.Results
We found that small permutations to the criteria used to define the control population result in significant shifts in both the demographic structure of the identified cohort as well as the odds ratio of association. These differences remain present when testing against age- and sex-matched controls.Discussion
Analyses of RWE need to be carefully designed to avoid issues of multiple testing. Minor changes to control cohorts can lead to significantly different results and have the potential to alter even prospective studies through selection bias.Conclusion
We believe this work offers strong support for the need for robust guidelines, best practices, and regulations around the use of observational RWE for clinical or regulatory decision-making.
SUBMITTER: Huang Y
PROVIDER: S-EPMC8206406 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:The aim of this article is to help develop a common understanding of the current status, challenges, and future perspectives of real-world data (RWD) and real-world evidence (RWE) in Japan. RWD and RWE are very widely used terms, but standardized definitions are lacking. Given broad and growing applications of RWD/RWE from the perspective of clinical development and medical affairs, the PhRMA Japan Medical Affairs Committee Working Group 1 have proposed the following definitions: "RWD are the data relating to patient health status and/or the delivery of health care routinely collected from a variety of sources" and "RWE is the evidence derived from analysis of RWD." The key challenges for RWD and RWE in Japan include restricted access and linkage of RWD, as well as a lack of universally accepted methodological approaches, which reduces the potential for patient and healthcare benefits. These challenges for RWD/RWE are by no means unique to Japan and similar challenges exist for countries in Europe and the USA. The quality of data and analysis, study design, and the transparency of reporting should be discussed more to ensure credibility and acceptance by decision-makers as the demand for RWD and RWE increases. The future developments around Japan's RWD and RWE are expected to include improved RWD access, data linkage, and increased acceptance by decision-makers, all supported by innovative technology. Improvements in RWD access and database linkage will enable both public and private sectors to assemble more comprehensive health information in Japan.
| S-EPMC8605941 | biostudies-literature
Project description:Randomized controlled trials (RCTs) are regarded as the most reputable source of evidence. In some studies, factors beyond the intervention itself may contribute to the measured effect, an occurrence known as heterogeneity of treatment effect (HTE). If the RCT population differs from the real-world population on factors that induce HTE, the trials effect will not replicate. The RCTs eligibility criteria should identify the sub-population in which its evidence will replicate. However, the extent to which the eligibility criteria identify the appropriate population is unknown, which raises concerns for generalizability. We compared reported data from RCTs with real-world data from the electronic health records of a large, academic medical center that was curated according to RCT eligibility criteria. Our results show fundamental differences between the RCT population and our observational cohorts, which suggests that eligibility criteria may be insufficient for identifying the applicable real-world population in which RCT evidence will replicate.
| S-EPMC7214444 | biostudies-literature
Project description: Not available
| S-EPMC5797980 | biostudies-literature
Project description:Randomized controlled trials (RCTs) are preferred by payers for health technology assessments and coverage decisions. However, the inclusion of a highly selective patient population and the rigorously controlled conditions in RCTs may not be reflective of real-world clinical practice. Real-world evidence (RWE) obtained from an analysis of real-world data (RWD) from observational studies can bridge gaps in evidence not addressed by RCTs and is thus valuable to public and private payers for decision-making. Through a broad literature search to obtain insights into payers' experience, we found that payers have concerns about real-world studies with respect to data quality, poor internal validity, potential bias, and lack of meaningful endpoints. However, they valued RWE to fill evidence gaps not addressed by RCTs, such as high-quality, real-world, long-term effectiveness and safety data; head-to-head drug comparisons; cost analyses for tiering formulary placement; medication use and adherence patterns; identification of relevant responder and non-responder patient subpopulations; and patient-reported outcomes (PROs). RWE can be used to assess clinically meaningful endpoints and gauge the impact of interventions on the quality of healthcare. Here, we review how payers use or can use RWD on the comparative effectiveness and safety of treatments, PROs, medication adherence and persistence, prescribing patterns, healthcare resource utilization, and patient characteristics and/or biomarkers associated with treatment response when making health technology assessments and payer coverage decisions across therapeutic areas.
| S-EPMC7895868 | biostudies-literature
Project description:There is no direct evidence comparing the 2 most commonly prescribed direct oral anticoagulants, apixaban and rivaroxaban, used for stroke prevention in nonvalvular atrial fibrillation (NVAF). A number of network meta-analyses (NMAs) of randomized control trials and real-world evidence (RWE) studies comparing the efficacy, effectiveness, and safety of apixaban and rivaroxaban have been published; however, a comprehensive evidence review across the available body of evidence is lacking. In this study, we aimed to systematically review and evaluate the clinical outcomes of apixaban and rivaroxaban using a combination of data gleaned from both NMAs and RWE studies. The review identified 21 NMAs and 5 RWE studies. The data demonstrated that apixaban was associated with fewer major bleeding events compared to rivaroxaban. There was no difference in the efficacy/effectiveness profiles between these treatments. Bleeding is a serious complication of anticoagulation therapy for the management of NVAF, and is associated with increased rates of hospitalization, morbidity, mortality, and health-care expenditure. The majority of studies in this comprehensive evidence review suggests that apixaban has a lower risk of major bleeding events compared to rivaroxaban in patients with NVAF.
| S-EPMC7098208 | biostudies-literature
Project description:BACKGROUND:The burden of disease due to cancer remains substantial. Since the value of real-world evidence has also been recognised by regulatory agencies, we established a Research Ethics Committee (REC) approved research database for cancer patients (Reference: 18/NW/0297). CONSTRUCTION AND CONTENT:Guy's Cancer Cohort introduces the concept of opt-out consent processes for research in a subset of oncology patients diagnosed and treated at a large NHS Trust in the UK. From April 2016 until March 2017, 1388 eligible patients visited Guy's and St Thomas' NHS Foundation Trust (GSTT) for breast cancer management. For urological cancers this number was 1757 and for lung cancer 677. The Cohort consists of a large repository of routinely collected clinical data recorded both retrospectively and prospectively. The database contains detailed clinical information collected at various timepoints across the treatment pathway inclusive of diagnostic data, and data on disease progression, recurrence and survival. CONCLUSIONS:Guy's Cancer Cohort provides a valuable infrastructure to answer a wide variety of research questions of a clinical, mechanistic, and supportive care nature. Clinical research using this database will result in improved patient safety and experience. Guy's Cancer Cohort promotes collaborative research and will accept applications for the release of anonymised datasets for research purposes.
| S-EPMC7077127 | biostudies-literature
Project description:Much has been written about real-world evidence (RWE), a concept that offers an understanding of the effects of healthcare interventions using routine clinical data. The reflection of diverse real-world practices is a double-edged sword that makes RWE attractive but also opens doors to several biases that need to be minimised both in the design and analytical phases of non-experimental studies. Additionally, it is critical to ensure that researchers who conduct these studies possess adequate methodological expertise and ability to accurately implement these methods. Critical design elements to be considered should include a clearly defined research question using a causal inference framework, choice of a fit-for-purpose data source, inclusion of new users of a treatment with comparators that are as similar as possible to that group, accurately classifying person-time and deciding censoring approaches. Having taken measures to minimise bias 'by design', the next step is to implement appropriate analytical techniques (for example propensity scores) to minimise the remnant potential biases. A clear protocol should be provided at the beginning of the study and a report of the results after, including caveats to consider. We also point the readers to readings on some novel analytical methods as well as newer areas of application of RWE. While there is no one-size-fits-all solution to evaluating RWE studies, we have focused our discussion on key methods and issues commonly encountered in comparative observational cohort studies with the hope that readers are better equipped to evaluate non-experimental studies that they encounter in the future. Graphical abstract.
| S-EPMC7448554 | biostudies-literature
Project description:BackgroundAmong self-care measures, the self-monitoring of blood glucose (SMBG) is a critical component for checking blood glucose levels. In addition, there is growing evidence suggesting that digital technologies are being adopted as an additional method for health care systems to increase patient contact. However, for patients with non-insulin-treated diabetes mellitus type 2 (DMT2), the value of SMBG was inconsistent among studies, and the evidence for digital technologies from real-world clinical practice is still limited.ObjectiveOur study aimed to assess patients with non-insulin-treated DMT2 who were receiving care from a single clinic and analyze whether the use of a diabetes management app and SMBG behavior would affect glycemic control in a real-world clinical setting.MethodsWe collaborated with a large clinic focused on diabetes care in Taiwan that had been using the Health2Sync mobile app and web-based Patient Management Platform to collect the data. The patients were divided into 2 groups (app-engaged-user group and only-data-uploader group) according to different activities in the app, and blood glucose was recorded every month from 1 to 6 months after registration in the app. A sample of 420 patients was included in the analysis, and a linear mixed model was built to investigate which factors affected the patients' blood glucose percentage change.ResultsUsing the mixed model coefficient estimates, we found that the percentage change was significantly negative when the only-data-uploader group was set as the baseline (t=-3.873, df=1.81 × 104; P<.001 for the patients of the app-engaged-user group). We found that for patients with shorter diabetes duration, their blood glucose decreased more than patients with longer diabetes duration (t=2.823, df=1.71 × 104; P=.005 for the number of years of diabetes duration). In addition, we found that for younger patients, their blood glucose decreased more than older patients (t=2.652, df=1.71 × 104; P=.008 for the age of the patients). Furthermore, the patients with an education level of junior high school or lower saw a significantly greater decrease in blood glucose percentage change than the patients with an education level of senior high school or higher (t=4.996, df=1.72 × 104; P<.001 for the patients with an education level of senior high school or higher). We also found that the count of blood glucose measured enlarged the decrease along the interaction months (t=-8.266, df=1.97 × 104; P<.001 for the nth month × the count of blood glucose in the nth month). Lastly, the gender of the patients did not significantly affect the percentage change (t=0.534, df=1.74 × 104; P=.59 for female patients).ConclusionsOur analysis showed the following: the blood glucose percentage change of the patients in the app-engaged-user group dropped more than that in the only-data-uploader group; shorter diabetes duration is associated with a steeper decrease in the patients' blood glucose percentage change; the percentage decrease in blood glucose change in younger patients is greater than older patients; the blood glucose percentage change of the patients with an education level of junior high school or lower dropped more than those with an education level of senior high school or higher; and the more frequently the patients test SMBG each month, the greater the decrease in the patients' blood glucose percentage. Further studies can be performed to consider the differences in daily behaviors such as exercise and diet across the patients and whether these factors could have vital effects on glycemic control.
| S-EPMC9244648 | biostudies-literature
Project description:There is growing interest in regulatory use of randomized pragmatic trials and noninterventional real-world (RW) studies of effectiveness and safety, but there is no agreed-on framework for assessing when this type of evidence is sufficiently reliable. Rather than impose a clinical trial-like paradigm on RW evidence, like blinded treatments or complete, source-verified data, the framework for assessing the utility of RW evidence should be grounded in the context of specific study objectives, clinical events that are likely to be detected in routine care, and the extent to which systematic error (bias) is likely to impact effect estimation. Whether treatment is blinded should depend on how well the outcome can be measured objectively. Qualification of a data source should be based on (1) numbers of patients of interest available for study; (2) if "must-have" data are likely to be recorded, and if so, how and where; (3) the accessibility of systematic follow-up data for the time period of interest; and (4) the potential for systematic errors (bias) in data collection and the likely magnitude of any such bias. Accessible data may not be representative of an entire population, but still may provide reliable evidence about the experience of typical patients treated under conditions of conventional care. Similarly, RW data that falls short of optimal length of follow-up or study size may still be useful in terms of its ability to provide evidence for regulators for subgroups of special interest. Developing a framework to qualify RW evidence in the context of a particular study purpose and data asset will enable broader regulatory use of RW data for approval of new molecular entities and label changes. Reliable information about diverse populations and settings should also help us move closer to more affordable, effective health care.
| S-EPMC5944086 | biostudies-other
Project description:The New World Screwworm fly, Cochliomyia hominivorax, is a major pest of livestock in South America and Caribbean. However, few genomic resources have been available for this species. A genome of 534?Mb was assembled from long read PacBio DNA sequencing of DNA from a highly inbred strain. Analysis of molecular evolution identified 40 genes that are likely under positive selection. Developmental RNA-seq analysis identified specific genes associated with each stage. We identify and analyze the expression of genes that are likely important for host-seeking behavior (chemosensory), development of larvae in open wounds in warm-blooded animals (heat shock protein, immune response) and for building transgenic strains for genetic control programs including gene drive (sex determination, germline). This study will underpin future experiments aimed at understanding the parasitic lifestyle of the screwworm fly and greatly facilitate future development of strains for efficient systems for genetic control of screwworm.
| S-EPMC7403345 | biostudies-literature