Dataset Information

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data.

ABSTRACT:

Objective

To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).

Materials and methods

We started with 3 widely cited DQ literature-2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)-and expanded our review systematically to cover relevant articles published up to February 2020. We extracted DQ dimensions and assessment methods from these studies, mapped their relationships, and organized a synthesized summarization of existing DQ dimensions and assessment methods. We reviewed the data checks employed by the PCORnet and mapped them to the synthesized DQ dimensions and methods.

Results

We analyzed a total of 3 reviews, 20 DQ frameworks, and 226 DQ studies and extracted 14 DQ dimensions and 10 assessment methods. We found that completeness, concordance, and correctness/accuracy were commonly assessed. Element presence, validity check, and conformance were commonly used DQ assessment methods and were the main focuses of the PCORnet data checks.

Discussion

Definitions of DQ dimensions and methods were not consistent in the literature, and the DQ assessment practice was not evenly distributed (eg, usability and ease-of-use were rarely discussed). Challenges in DQ assessments, given the complex and heterogeneous nature of real-world data, exist.

Conclusion

The practice of DQ assessment is still limited in scope. Future work is warranted to generate understandable, executable, and reusable DQ measures.

SUBMITTER: Bian J

PROVIDER: S-EPMC7727392 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data.

Bian Jiang J Lyu Tianchen T Loiacono Alexander A Viramontes Tonatiuh Mendoza TM Lipori Gloria G Guo Yi Y Wu Yonghui Y Prosperi Mattia M George Thomas J TJ Harle Christopher A CA Shenkman Elizabeth A EA Hogan William W

Journal of the American Medical Informatics Association : JAMIA 20201201 12

<h4>Objective</h4>To synthesize data quality (DQ) dimensions and assessment methods of real-world data, especially electronic health records, through a systematic scoping review and to assess the practice of DQ assessment in the national Patient-centered Clinical Research Network (PCORnet).<h4>Materials and methods</h4>We started with 3 widely cited DQ literature-2 reviews from Chan et al (2010) and Weiskopf et al (2013a) and 1 DQ framework from Kahn et al (2016)-and expanded our review systemat ...[more]

PMID: 33166397

Similar Datasets

Project description:BackgroundWhile randomized controlled trials (RCTs) are based on strict inclusion/exclusion criteria, non-interventional studies (NISs) might provide additional information to guide management in patients more representative to the real-world setting. The aim of this study was to compare baseline characteristics of patients receiving intravitreal treatment in the NIS OCEAN with those from published RCTs.MethodsThe ongoing OCEAN study enrolled patients treated with ranibizumab for neovascular age-related macular degeneration (nAMD), diabetic macular oedema (DME) or branch/central retinal vein occlusion (B/CRVO). Baseline patient characteristics were compared by indication within the OCEAN cohort. Furthermore, the characteristics were set in reference to those of published RCTs in the same indications. Confidence intervals (CIs) were calculated and assessed for statistically significant differences as indicated by non-overlapping CIs.ResultsPatient characteristics in the NIS OCEAN were evaluated for 3,614 patients with nAMD, 1,211 with DME, 204 with BRVO and 121 with CRVO. Between these groups, significant differences in mean age, gender distributions, and mean baseline VA were seen, reflecting known differences between the indications. Compared to the patient characteristics of published RCTs (trials selected by literature search: nAMD: 13 RCTs, DME: 9, RVO: 5), the OCEAN patients' mean age was significantly higher in every indication. The gender distributions across the trials were comparable, with only few differences between OCEAN and the RCTs. Regarding the mean baseline VA, notable differences were found in nAMD and in DME, with VA significantly higher in some RCTs and lower in others.ConclusionsThe described differences underline the complementarity of NISs and RCTs. OCEAN covers a broader spectrum and more variability of patients than do RCTs. As baseline values may have impact on the treatment response (ceiling effect), there is an ongoing need for research in all patient subgroups. Country-specific assessments of patient populations can better reflect the real-world situation. NISs can deliver insights that RCTs may not, as NISs can include non-typical patients, patients with comorbidities, a broader age spectrum and patients of various disease stages.Trial registrationThe NIS OCEAN was registered on www.clinicaltrials.gov (identifier: NCT02194803 ).

Project description:BackgroundDigital technologies are transforming the health care system. A large part of information is generated as real-world data (RWD). Data from electronic health records and digital biomarkers have the potential to reveal associations between the benefits and adverse events of medicines, establish new patient-stratification principles, expose unknown disease correlations, and inform on preventive measures. The impact for health care payers and providers, the biopharmaceutical industry, and governments is massive in terms of health outcomes, quality of care, and cost. However, a framework to assess the preliminary quality of RWD is missing, thus hindering the conduct of population-based observational studies to support regulatory decision-making and real-world evidence.ObjectiveTo address the need to qualify RWD, we aimed to build a web application as a tool to translate characterization of some quality parameters of RWD into a metric and propose a standard framework for evaluating the quality of the RWD.MethodsThe RWD-Cockpit systematically scores data sets based on proposed quality metrics and customizable variables chosen by the user. Sleep RWD generated de novo and publicly available data sets were used to validate the usability and applicability of the web application. The RWD quality score is based on the evaluation of 7 variables: manageability specifies access and publication status; complexity defines univariate, multivariate, and longitudinal data; sample size indicates the size of the sample or samples; privacy and liability stipulates privacy rules; accessibility specifies how the data set can be accessed and to what granularity; periodicity specifies how often the data set is updated; and standardization specifies whether the data set adheres to any specific technical or metadata standard. These variables are associated with several descriptors that define specific characteristics of the data set.ResultsTo address the need to qualify RWD, we built the RWD-Cockpit web application, which proposes a framework and applies a common standard for a preliminary evaluation of RWD quality across data sets-molecular, phenotypical, and social-and proposes a standard that can be further personalized by the community retaining an internal standard. Applied to 2 different case studies-de novo-generated sleep data and publicly available data sets-the RWD-Cockpit could identify and provide researchers with variables that might increase quality.ConclusionsThe results from the application of the framework of RWD metrics implemented in the RWD-Cockpit application suggests that multiple data sets can be preliminarily evaluated in terms of quality using the proposed metrics. The output scores-quality identifiers-provide a first quality assessment for the use of RWD. Although extensive challenges remain to be addressed to set RWD quality standards, our proposal can serve as an initial blueprint for community efforts in the characterization of RWD quality for regulated settings.

Project description:IntroductionIn this literature review we evaluated the real-world clinical effectiveness of switching Japanese diabetic patients from their current insulin regimen to insulin degludec (IDeg).MethodsStudies were identified from Japanese Diabetes Society (JDS) abstracts (2014-2015) and PubMed (2012 onwards). Inclusion criteria were: Japanese population, >15 participants, and studies switching patients from basal or basal-bolus insulin regimens to IDeg. Randomized controlled trials and case reports were excluded. Weighted mean changes in safety and effectiveness endpoints were calculated using the number of patients in each study.ResultsIn total, 81 JDS abstracts and seven manuscripts met the search criteria, representing 4238 patients [1028 with type 1 diabetes (T1D), 602 with type 2 diabetes (T2D), 2608 with unspecified or mixed diabetes]. Glycated hemoglobin (HbA1c) was reported in 93% of studies, with an improvement in 84% of these (51% significant, 33% numerical), no change in 12%, and worsening in 4% (3% numerical, 1% significant). Across all studies, the weighted mean absolute change in HbA1c was -0.3% (-2.7 mmol/mol). Basal insulin dose was reported in 58% of studies and was lower in 60% of these (30% significant, 30% numerical), numerically unchanged in 26%, and higher in 14% (2% significant, 12% numerical). The weighted mean change in basal insulin dose was -4.8% and -3.0% for all studies and for studies with only significant results, respectively. The weighted mean change in basal dose based on all studies was -8.9, -5.5, and -2.9% for the T1D, T2D, and unspecified patient populations, respectively. Hypoglycemia was recorded in 31% of the studies. After switching treatment to IDeg, 55% of studies reported decreased hypoglycemia, 29% no change, and 16% an increase. Quality of life (QoL) was measured in 11% of studies, of which 82% reported improved QoL after switching, and 18% reported no change in QoL.ConclusionSwitching from a conventional basal insulin to IDeg has the potential to improve HbA1c with a lower insulin dose. Switching to IDeg may also provide a reduced risk of hypoglycemia and improvement in QoL.FundingNovo Nordisk.

Dataset Information

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data.

Objective

Materials and methods

Results

Discussion

Conclusion

Publications

Assessing the practice of data quality evaluation in a national clinical data research network through a systematic scoping review in the era of real-world data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets