A comparative effectiveness study of eSource used for data capture for a clinical research registry.
Ontology highlight
ABSTRACT: Objective
This pilot study compared eSource-enabled versus traditional manual data transcription (non-eSource methods) for the collection of clinical registry information. The primary study objective was to compare the time spent completing registry forms using eSource versus non-eSource methods The secondary objectives were to compare data quality associated with these two data capture methods and the flexibility of the workflows. This study directly addressed fundamental questions relating to eSource adoption: what time-savings can be realized, and to what extent does eSource improve data quality.Materials and methods
The study used time and motion methods to compare eSource versus non-eSource data capture workflows for a single center OB/GYN registry. Direct observation by industrial engineers using specialized computer software captured keystrokes, mouse clicks and video recordings of the study team in their normal work environment completing real-time data collection.Results
The overall average data capture time was reduced with eSource versus non-eSource methods (difference, 151s per case; eSource, 1603s; non-eSource, 1754s; p=0.051). The average data capture time for the demographic data was reduced (difference, 79s per case; eSource, 133s; non-eSource, 213s; p<0.001). This represents a 37% time reduction (95% confidence interval 27% to 47%). eSourced data field transcription errors were also reduced (eSource, 0%; non-eSource, 9%).Conclusion
The use of eSource versus traditional data transcription was associated with a significant reduction in data entry time and data quality errors. Further studies in other settings are needed to validate these results.
SUBMITTER: Nordo AH
PROVIDER: S-EPMC5942198 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ImportanceResearchers often analyze cancer registry data to assess for differences in survival among cancer treatments. However, the retrospective, nonrandomized design of these analyses raises questions about study validity.ObjectiveTo examine the extent to which comparative effectiveness analyses using observational cancer registry data produce results concordant with those of randomized clinical trials.Design, setting, and participantsIn this comparative effectiveness study, a total of 141 randomized clinical trials referenced in the National Comprehensive Cancer Network Clinical Practice Guidelines for 8 common solid tumor types were identified. Data on participants within the National Cancer Database (NCDB) diagnosed between 2004 and 2014, matching the eligibility criteria of the randomized clinical trial, were obtained. The present study was conducted from August 1, 2017, to September 10, 2019. The trials included 85 118 patients, and the corresponding NCDB analyses included 1 344 536 patients. Three Cox proportional hazards regression models were used to determine hazard ratios (HRs) for overall survival, including univariable, multivariable, and propensity score-adjusted models. Multivariable and propensity score analyses controlled for potential confounders, including demographic, comorbidity, clinical, treatment, and tumor-related variables.Main outcomes and measuresThe main outcome was concordance between the results of randomized clinical trials and observational cancer registry data. Hazard ratios with an NCDB analysis were considered concordant if the NDCB HR fell within the 95% CI of the randomized clinical trial HR. An NCDB analysis was considered concordant if both the NCDB and clinical trial P values for survival were nonsignificant (P ≥ .05) or if they were both significant (P < .05) with survival favoring the same treatment arm in the NCDB and in the randomized clinical trial.ResultsAnalyses using the NCDB-produced HRs for survival were concordant with those of 141 randomized clinical trials in 79 univariable analyses (56%), 98 multivariable analyses (70%), and 90 propensity score models (64%). The NCDB analyses produced P values concordant with randomized clinical trials in 58 univariable analyses (41%), 65 multivariable analyses (46%), and 63 propensity score models (45%). No clinical trial characteristics were associated with concordance between NCDB analyses and randomized clinical trials, including disease site, type of clinical intervention, or severity of cancer.Conclusions and relevanceThe findings of this study suggest that comparative effectiveness research using cancer registry data often produces survival outcomes discordant with those of randomized clinical trial data. These findings may help provide context for clinicians and policy makers interpreting observational comparative effectiveness research in oncology.
| S-EPMC9009816 | biostudies-literature
Project description:BackgroundThe technological complexities and broad operational scope of eSource impede coordinated, inter-organizational action on advancing at-scale solutions.MethodsWe introduce an architectural framework for articulating technological considerations across organizations. The architecture neither implies nor endorses solution implementations; rather, it proposes solution functionality based upon principles and good clinical practices.ResultsKey technology considerations include patterns of anticipated use, implications to the current state of clinical trial operations, and the need for new technologies (i.e., IoT, Big Data, Predictive Analytics).ConclusionTechnology considerations drive implications beyond technology-influencing regulatory, process, and ethical realms of clinical research.
| S-EPMC7458892 | biostudies-literature
Project description:Direct extraction and use of electronic health record (EHR) data is a long-term and multifaceted endeavor that includes design, development, implementation and evaluation of methods and tools for semi-automating tasks in the research data collection process, including, but not limited to, medical record abstraction (MRA). A systematic mapping of study data elements was used to measure the coverage of the Health Level Seven (HL7®) Fast Healthcare Interoperability Resources (FHIR®) standard for a federally sponsored, pragmatic cardiovascular randomized controlled trial (RCT) targeting adults. We evaluated site-level implementations of the HL7® FHIR® standard to investigate study- and site-level differences that could affect coverage and offer insight into the feasibility of a FHIR-based eSource solution for multicenter clinical research.
| S-EPMC9137245 | biostudies-literature
Project description:ObjectiveLinkages between registries and administrative data may provide a valuable resource for comparative effectiveness research. However, personal identifiers that uniquely identify individuals are not always available. Here we describe methods to link a de-identified arthritis registry and US Medicare data. The linked data set was also used to evaluate the generalizability of the registry to the US Medicare population.MethodsRheumatoid arthritis (RA) patients participating in the Consortium of Rheumatology Researchers of North America (CORRONA) registry were linked to Medicare data restricted to rheumatology claims or claims for RA. Deterministic linkage was performed using age, sex, provider identification number, and geographic location of the CORRONA site. We then searched for visit dates in Medicare matching visit dates in CORRONA, requiring ≥1 exact matching date. Linkage accuracy was quantified as a positive predictive value in a subcohort (n = 1,581) with more precise identifiers.ResultsCORRONA participants with self-reported Medicare (n = 11,001) were initially matched to 30,943 Medicare beneficiaries treated by CORRONA physicians. A total of 8,431 CORRONA participants matched on ≥1 visit; 5,317 matched uniquely on all visits. The number of patients who linked and linkage accuracy (from the subcohort) were high for patients with >2 visits (n = 3,458, 98% accuracy), exactly 2 visits (n = 822, 96% accuracy), and 1 visit (n = 1,037, 79% accuracy) that matched exactly on calendar date. Demographics and comorbidity profiles of registry participants were similar to nonparticipants, except participants were more likely to take disease-modifying antirheumatic drugs and biologic agents.ConclusionLinkage between a national, de-identified outpatient arthritis registry and Medicare data on multiple nonunique identifiers appears feasible and valid.
| S-EPMC4245366 | biostudies-literature
Project description:Electronic health information routinely collected during health care delivery and reimbursement can help address the need for evidence about the real-world effectiveness, safety, and quality of medical care. Often, distributed networks that combine information from multiple sources are needed to generate this real-world evidence.We provide a set of field-tested best practices and a set of recommendations for data quality checking for comparative effectiveness research (CER) in distributed data networks.Explore the requirements for data quality checking and describe data quality approaches undertaken by several existing multi-site networks.There are no established standards regarding how to evaluate the quality of electronic health data for CER within distributed networks. Data checks of increasing complexity are often used, ranging from consistency with syntactic rules to evaluation of semantics and consistency within and across sites. Temporal trends within and across sites are widely used, as are checks of each data refresh or update. Rates of specific events and exposures by age group, sex, and month are also common.Secondary use of electronic health data for CER holds promise but is complex, especially in distributed data networks that incorporate periodic data refreshes. The viability of a learning health system is dependent on a robust understanding of the quality, validity, and optimal secondary uses of routinely collected electronic health data within distributed health data networks. Robust data quality checking can strengthen confidence in findings based on distributed data network.
| S-EPMC4306391 | biostudies-literature
Project description:Good decisions depend on an accurate understanding of the comparative effectiveness of decision alternatives. The best way to convey data needed to support these comparisons is unknown.To determine how well 5 commonly used data presentation formats convey comparative effectiveness information.The study was an Internet survey using a factorial design. Participants consisted of 279 members of an online survey panel. Study participants compared outcomes associated with 3 hypothetical screening test options relative to 5 possible outcomes with probabilities ranging from 2 per 5000 (0.04%) to 500 per 1000 (50%). Data presentation formats included a table, a "magnified" bar chart, a risk scale, a frequency diagram, and an icon array. Outcomes included the number of correct ordinal judgments regarding the more likely of 2 outcomes, the ratio of perceived versus actual relative likelihoods of the paired outcomes, the intersubject consistency of responses, and perceived clarity.The mean number of correct ordinal judgments was 12 of 15 (80%), with no differences among data formats. On average, there was a 3.3-fold difference between perceived and actual likelihood ratios (95% confidence interval = 3.0-3.6). Comparative judgments based on flowcharts, icon arrays, and tables were all significantly more accurate and consistent than those based on risk scales and bar charts (P < 0.001). The most clearly perceived formats were the table and the flowchart. Low subjective numeracy was associated with less accurate and more variable data interpretations and lower perceived clarity for icon displays, bar charts, and flow diagrams.None of the data presentation formats studied can reliably provide patients, especially those with low subjective numeracy, with an accurate understanding of comparative effectiveness information.
| S-EPMC3515653 | biostudies-literature
Project description:BackgroundTo assess the completeness of reporting, research transparency practices, and risk of selection and immortal bias in observational studies using routinely collected data for comparative effectiveness research.MethodWe performed a meta-research study by searching PubMed for comparative effectiveness observational studies evaluating therapeutic interventions using routinely collected data published in high impact factor journals from 01/06/2018 to 30/06/2020. We assessed the reporting of the study design (i.e., eligibility, treatment assignment, and the start of follow-up). The risk of selection bias and immortal time bias was determined by assessing if the time of eligibility, the treatment assignment, and the start of follow-up were synchronized to mimic the randomization following the target trial emulation framework.ResultSeventy-seven articles were identified. Most studies evaluated pharmacological treatments (69%) with a median sample size of 24,000 individuals. In total, 20% of articles inadequately reported essential information of the study design. One-third of the articles (n = 25, 33%) raised some concerns because of unclear reporting (n = 6, 8%) or were at high risk of selection bias and/or immortal time bias (n = 19, 25%). Only five articles (25%) described a solution to mitigate these biases. Six articles (31%) discussed these biases in the limitations section.ConclusionReporting of essential information of study design in observational studies remained suboptimal. Selection bias and immortal time bias were common methodological issues that researchers and physicians should be aware of when interpreting the results of observational studies using routinely collected data.
| S-EPMC8608432 | biostudies-literature
Project description:Comparative effectiveness research (CER) calls for substantial changes in the way clinical research is conducted, interpreted and practically applied in the USA, in order to produce useful clinical evidence. Departing from classic efficacy and effectiveness research, the evolving CER paradigm requires structural and substantive innovations that address three basic questions: what works? for whom? and in whose hands? Addressing these questions will require fundamental changes in the approach to clinical research that include: the use of active treatments (or comparators) versus placebos in the comparisons of treatments, innovative or 'alternative' research methods, the specification and a priori design of studies to account for important subgroups, accounting for the nested nature of healthcare delivery in design and analysis of CER, the simultaneous study of multiple treatments or treatment modalities, the study of multiple outcomes (benefits and harms) for each treatment compared, and the reassessment of the value of different study designs in the hierarchy of collective 'evidence'. In order to aid individual providers and patients in making informed, personalized treatment decisions, guided by the best evidence possible, CER studies must generalize to a broad range of subgroups reflecting the spectrum of patients, providers and health systems that populate real-world practice settings. Without expansion in the scope, conduct and subsequent interpretation of clinical research reflected in the issues outlined above, CER will fall short of its potential for informing evidence-based practice and personalized medicine. The current paradigm for conducting, interpreting and applying clinical research does not meet the needs of optimal generalizability and application to individual physician-patient efforts to identify the most effective treatment, and therefore does not support the basic requirements of CER. The proposed changes should neither require decades nor exorbitant budgets to achieve. Using two examples, prostate cancer, and comparisons of single medications, we illustrated how the proposed changes in clinical research, matching strategy to each application, might be addressed.
| S-EPMC3619728 | biostudies-literature
Project description:The Chronic Hepatitis Cohort Study (CHeCS) is a longitudinal observational study of risks and benefits of treatments and care in patients with chronic hepatitis B (HBV) and C (HCV) infection from four US health systems. We hypothesized that comparative effectiveness methods-including a centralized data management system and an adaptive approach for cohort selection-would improve cohort selection while controlling data quality and reducing the cost.Cohort selection and data collection were performed primarily via the electronic health record (EHR); cases were confirmed via chart abstraction. Two parallel sources fed data to a centralized data management system: direct EHR data collection with common data elements, and chart abstraction via electronic data capture. An adaptive Classification and Regression Tree (CART) identified a set of electronic variables to improve case ascertainment accuracy.Over 16 million patient records were collected on 23 case report forms in 2006-2008. The vast majority of data (99.2%) were collected electronically from EHR; only 0.8% was collected via chart abstraction. Initial electronic criteria identified 12,144 chronic hepatitis patients; 10,098 were confirmed via chart abstraction with positive predictive values (PPV) 79 and 83% for HBV and HCV, respectively. CART-optimized models significantly increased PPV to 88 for HBV and 95% for HCV.CHeCS is a comparative effectiveness research project that leverages electronic centralized data collection and adaptive cohort identification approaches to enhance study efficiency. The adaptive CART model significantly improved the positive predictive value of cohort identification methods.
| S-EPMC5719869 | biostudies-other
Project description:BackgroundThe FeverApp registry uses ecological momentary assessment (EMA) to collect parental data on pediatric fever for scientific research. The mobile app FeverApp educates parents on safe fever management and serves as a fever diary.ObjectiveThe focus of this study was to evaluate the completeness and concordance of the EMA-based FeverApp registry with regard to its data quality from a multilevel perspective.MethodsStructured descriptions of fever episodes by health care professionals from an office were used as reference. The number of children, their sociodemographic data, and agreement of fever episodes, with maximum temperature, intake of antipyretics and antibiotics, and physician visits, were compared with the entries in the corresponding physician's reference records. The data quality indicators for completeness, meaning the extent to which the necessary data for the registry has actually been submitted, and concordance, which is the correspondence of the value of a data element with a reference source, were chosen to analyze whether EMA may be a suitable method for this kind of registry.ResultsIn both data sources, 1012 children were available for comparison over 16 months. The completeness of gender (1012/1012, 100%) and date of birth (1004/1012, 99.2%) information was high, and the mismatches were 0.69% (7/1012) and 1.19% (12/1012), respectively, between the sources. Of these 1012 children, 668 (66%) registered fever episodes in FeverApp. They relate to 534 families with 953 fever episodes in the reference records and 1452 episodes in the FeverApp registry. Of the 534 families, 183 (34.3%) refrained from visiting the office during fever episodes but nevertheless documented them in FeverApp. Largest part (766/1452, 52.75%) episodes were recorded exclusively in the FeverApp registry by 371 (371/534, 69.5%) families. The remaining 686 (47.2%) episodes of 391 (58.5%) children from 351 (65.7%) families were comparable with the reference data source in terms of physician visits, medication, and temperature. The completeness ranged, depending on the kind of variable, from 11.5% to 65% in the registry and from 7.6% to 42.6% in the office. The 953 fever episodes reported by the reference office consisted of 681 (71.5%) acute and 272 (28.5%) past episodes. In FeverApp, most past (262/272, 96.3%) but less acute (424/681, 62.3%) episodes have been entered. The concordance rates were varied: 90.2% for antibiotic use, 66.6% for antipyretic use, 61.7% for physician visits, and 16% for the highest temperature during the fever episode.ConclusionsBoth sources delivered only partial data, and the rates of completeness and concordance depended on the kind of variable. However, the FeverApp registry showed higher documentation and precision rates than professional records for all considered variables. Therefore, EMA may play a unique supplement for research in ambulatory care. FeverApp could support pediatric offices, especially during the pandemic.
| S-EPMC9669892 | biostudies-literature