Dataset Information

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations.

ABSTRACT:

Objective

To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.

Methods

We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommendations for study-specific fitness-for-use assessments.

Results

Of 15 invited trial teams, 7 interviews were conducted. Of 31 invited trials, 15 responded to the survey. Most respondents had prior experience using RWD (93%). Major concerns about EHR data were data reliability, missingness or incompleteness of EHR elements, variation in data quality across study sites, and presence of implausible or incorrect values. Although many PCTs conducted fitness-for-use activities (eg, data quality assessments, 11/14, 79%), less than a quarter did so before choosing a data source. Fitness-for-use activities, findings, and resulting study design changes were not often publically documented. Overall costs and personnel costs were barriers to fitness-for-use assessments.

Discussion

These results support three recommendations for PCTs that use EHR data for endpoint ascertainment. Trials should detail the rationale and plan for study-specific fitness-for-use activities, conduct study-specific fitness-for-use assessments early in the prestudy phase to inform study design changes before the trial begins, and share results of fitness-for-use assessments and description of relevant challenges and facilitators.

Conclusion

These recommendations can help researchers and end-users of real-world evidence improve characterization of RWD reliability and relevance in the PCT-specific context.

SUBMITTER: Raman SR

PROVIDER: S-EPMC9006695 | biostudies-literature | 2022 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations.

Raman Sudha R SR O'Brien Emily C EC Hammill Bradley G BG Nelson Adam J AJ Fish Laura J LJ Curtis Lesley H LH Marsolo Keith K

Journal of the American Medical Informatics Association : JAMIA 20220401 5

<h4>Objective</h4>To empirically explore how pragmatic clinical trials (PCTs) that used real-world data (RWD) assessed study-specific fitness-for-use.<h4>Methods</h4>We conducted interviews and surveys with PCT teams who used electronic health record (EHR) data to ascertain endpoints. The survey cataloged key concerns about RWD, activities used to assess data fitness-for-use, and related barriers encountered by study teams. Patterns and commonalities across trials were used to develop recommenda ...[more]

PMID: 35171985

Similar Datasets

Project description:Objectives The objective was to compare case ascertainment, agreement, validity, and missing values for clinical research data obtained, processed, and linked electronically from electronic health records (EHR) compared to "manual" data processing and record abstraction in a cohort of out-of-hospital trauma patients.Methods This was a secondary analysis of two sets of data collected for a prospective, population-based, out-of-hospital trauma cohort evaluated by 10 emergency medical services (EMS) agencies transporting to 16 hospitals, from January 1, 2006, through October 2, 2007. Eighteen clinical, operational, procedural, and outcome variables were collected and processed separately and independently using two parallel data processing strategies by personnel blinded to patients in the other group. The electronic approach included EHR data exports from EMS agencies, reformatting, and probabilistic linkage to outcomes from local trauma registries and state discharge databases. The manual data processing approach included chart matching, data abstraction, and data entry by a trained abstractor. Descriptive statistics, measures of agreement, and validity were used to compare the two approaches to data processing.Results During the 21-month period, 418 patients underwent both data processing methods and formed the primary cohort. Agreement was good to excellent (kappa = 0.76 to 0.97; intraclass correlation coefficient [ICC] = 0.49 to 0.97), with exact agreement in 67% to 99% of cases and a median difference of zero for all continuous and ordinal variables. The proportions of missing out-of-hospital values were similar between the two approaches, although electronic processing generated more missing outcomes (87 of 418, 21%, 95% confidence interval [CI] = 17% to 25%) than the manual approach (11 of 418, 3%, 95% CI = 1% to 5%). Case ascertainment of eligible injured patients was greater using electronic methods (n = 3,008) compared to manual methods (n = 629).Conclusions In this sample of out-of-hospital trauma patients, an all-electronic data processing strategy identified more patients and generated values with good agreement and validity compared to traditional data collection and processing methods.

Project description:The use of electronic health records has garnered interest as an approach for conducting innovative outcome research and producing real-world evidence at a reduced cost compared to traditional clinical trials. The study aimed to evaluate the utility of deidentified EHR data from a multicenter research network to identify characteristics associated with treatment escalation (TE) in newly diagnosed pediatric ulcerative colitis patients. EHR data (01/2010-12/2021) from 13 Midwest healthcare systems (Greater Plains Collaborative) were collected for pediatric ulcerative colitis patients. We identified standard treatments, excluded missing initial therapy data, and analyzed the TE and time-to-TE outcomes. The clinical and laboratory characteristics at baseline were extracted. Logistic and Cox models were used, and the missing risk factors were imputed. Machine-learning Bayesian additive regression trees were also utilized to create partial dependence plots for assessing the associations between risk factors and clinical outcomes. A total of 502 eligible pediatric patients (aged 4-17 years) who initiated standard treatment were identified. Among them, 205 out of 502 (41%) experienced TE, with a median (P25, P75) duration of 63 (9, 237) days after the initial treatment. Additionally, 20 out of 509 (4%) patients underwent colectomy (COL) with a median (P25, P75) duration of 80 (3, 205) days. Both multivariable logistic regression and Cox proportional hazards regression demonstrated moderate discriminative power in predicting TE and time-to-TE, respectively. Common positive predictors for both TE and time-to-TE included a high monocyte proportion and elevated platelet counts. Conversely, BMI z-score, albumin, hemoglobin levels, and lymphocyte proportion were negatively associated with both TE and time-to-TE. This study demonstrates that multicenter EHR data can be used to identify a trial-comparable study sample of potentially larger size and to identify clinically meaningful endpoints for conducting outcome analysis and generating real-world evidence.

Project description:BACKGROUND:Designing appropriate studies for evaluating complex interventions, such as electronic health solutions to support integrated care, remains a methodological challenge. With the many moving parts of complex interventions, it is not always clear how program activities are connected to anticipated and unanticipated outcomes. Exploratory trials can be used to uncover determinants (or mechanisms) to inform content theory that underpins complex interventions before designing a full evaluation plan. OBJECTIVE:A multimethod exploratory trial of the electronic patient-reported outcome (ePRO) tool was conducted to uncover contexts, processes and outcome variables, and the mechanisms that link these variables before full-scale evaluation. ePRO is a mobile app and portal designed to support goal-oriented care in interdisciplinary primary health care practices (clinical-level integration). This paper offers evaluation findings and methodological insight on how to use exploratory trial data to identify relevant context, process, and outcome variables, as well as central (necessary to achieving outcomes) versus peripheral (less critical and potentially context dependent) mechanisms at play. METHODS:The 4-month trial was conducted in 2 primary health care practices in Toronto, Canada. The patients were randomized into control and intervention groups and compared pre and post on quality of life and activation outcome measures. Semistructured interviews were conducted with providers and patients in the intervention group. Narrative analysis was used to uncover dominant mechanisms that inform the intervention's content theory (how context and process variables are linked to outcomes). RESULTS:Overall, 7 providers, 1 administrator, and 16 patients (7-control, 9-intervention) participated in the study. This study uncovered many complex and nuanced context, process, and outcome variables at play in the intervention. Narrative analysis of patient and provider interviews revealed dominant story lines that help to tease apart central and peripheral mechanisms driving the intervention. Provider and patient story lines centered around fitting the new intervention into everyday work and life of patients and providers and meaningfulness of the intervention. These themes were moderated by patient-provider relationships going into and throughout the intervention, their comfort with technology, and the research process. CONCLUSIONS:Identifying dominant story lines using narrative analysis helps to identify the most relevant context and process variables likely to influence study outcomes. Normalization process theory emerges as a useful theory to uncover underlying mechanisms because of its emphasis on the social production and normalization of technological, processual, and social aspects of work; all found to be critical to our intervention. The number of complex, overlapping influencing variables suggests that complex interventions such as ePRO require us to pay careful attention to central versus peripheral mechanisms that will influence study outcomes. The narrative methods presented here are shown to be useful in uncovering these mechanisms and help to guide subsequent larger evaluation studies.

Project description:BackgroundLinkage of electronic healthcare records is becoming increasingly important for research purposes. However, linkage error due to mis-recorded or missing identifiers can lead to biased results. We evaluated the impact of linkage error on estimated infection rates using two different methods for classifying links: highest-weight (HW) classification using probabilistic match weights and prior-informed imputation (PII) using match probabilities.MethodsA gold-standard dataset was created through deterministic linkage of unique identifiers in admission data from two hospitals and infection data recorded at the hospital laboratories (original data). Unique identifiers were then removed and data were re-linked by date of birth, sex and Soundex using two classification methods: i) HW classification - accepting the candidate record with the highest weight exceeding a threshold and ii) PII-imputing values from a match probability distribution. To evaluate methods for linking data with different error rates, non-random error and different match rates, we generated simulation data. Each set of simulated files was linked using both classification methods. Infection rates in the linked data were compared with those in the gold-standard data.ResultsIn the original gold-standard data, 1496/20924 admissions linked to an infection. In the linked original data, PII provided least biased results: 1481 and 1457 infections (upper/lower thresholds) compared with 1316 and 1287 (HW upper/lower thresholds). In the simulated data, substantial bias (up to 112%) was introduced when linkage error varied by hospital. Bias was also greater when the match rate was low or the identifier error rate was high and in these cases, PII performed better than HW classification at reducing bias due to false-matches.ConclusionsThis study highlights the importance of evaluating the potential impact of linkage error on results. PII can help incorporate linkage uncertainty into analysis and reduce bias due to linkage error, without requiring identifiers.

Dataset Information

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations.

Objective

Methods

Results

Discussion

Conclusion

Publications

Evaluating fitness-for-use of electronic health records in pragmatic clinical trials: reported practices and recommendations.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets