Dataset Information

Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future.

ABSTRACT:

Background

Randomised controlled trials (RCTs) are generally regarded as the "gold standard" for providing quantifiable evidence around the effectiveness and cost-effectiveness of new healthcare technologies. In order to perform the economic evaluations associated with RCTs, there is a need for accessible and good quality resource-use data; for the purpose of discussion here, data that best reflect the care received. Traditionally, researchers have developed questionnaires for resource-use data collection. However, the evolution of routinely collected electronic data within care services provides new opportunities for collecting data without burdening patients or caregivers (e.g. clinicians). This paper describes the potential strengths and limitations of each data collection method and then discusses aspects for consideration before choosing which method to use.

Main text

We describe electronic data sources (large observational datasets, commissioning data, and raw data extraction) that may be suitable data sources for informing clinical trials and the current status of self-reported instruments for measuring resource-use. We assess the methodological risks and benefits, and compare the two methodologies. We focus on healthcare resource-use; however, many of the considerations have relevance to clinical questions. Patient self-report forms a pragmatic and cheap method that is largely under the control of the researcher. However, there are known issues with the validity of the data collected, loss to follow-up may be high, and questionnaires suffer from missing data. Routinely collected electronic data may be more accurate and more practical if large numbers of patients are involved. However, datasets often incur a cost and researchers are bound by the time for data approval and extraction by the data holders.

Conclusions

Owing to the issues associated with electronic datasets, self-reported methods may currently be the preferred option. However, electronic hospital data are relatively more accessible, informative, standardised, and reliable. Therefore in trials where secondary care constitutes a major driver of patient care, detailed electronic data may be considered superior to self-reported methods; with the caveat of requiring data sharing agreements with third party providers and potentially time-consuming extraction periods. Self-reported methods will still be required when a 'societal' perspective (e.g. quantifying informal care) is desirable for the intended analysis.

SUBMITTER: Franklin M

PROVIDER: S-EPMC6325715 | biostudies-literature | 2019 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future.

Franklin Matthew M Thorn Joanna J

BMC medical research methodology 20190109 1

<h4>Background</h4>Randomised controlled trials (RCTs) are generally regarded as the "gold standard" for providing quantifiable evidence around the effectiveness and cost-effectiveness of new healthcare technologies. In order to perform the economic evaluations associated with RCTs, there is a need for accessible and good quality resource-use data; for the purpose of discussion here, data that best reflect the care received. Traditionally, researchers have developed questionnaires for resource-u ...[more]

PMID: 30626337

Similar Datasets

Project description:BackgroundHealth economic evaluations rely on the accurate measurement of health service resource use in order to calculate costs. These are usually measured with patient completed questionnaires using instruments such as the Client Service Receipt Inventory (CSRI). These rely on participants' recall and can be burdensome to complete. Health service activity data are routinely captured by electronic databases.The aim was to test methods for obtaining these data and compare with those data collected using the CSRI, within a feasibility study of an enhanced rehabilitation intervention following hip fracture (Fracture in the Elderly Multidisciplinary Rehabilitation: FEMuR).MethodsPrimary care activity including prescribing data was obtained from the Secure Anonymised Information Linkage (SAIL) Databank and secondary care activity (Emergency Department attendances, out-patient visits and in-patient days) directly from Betsi Cadwaladr University Health Board (BCUHB), North Wales, UK. These data were compared with patient responses from the CSRI using descriptive statistics and the intraclass correlation coefficient (ICC).ResultsIt was possible to compare health service resource use data for 49 out of 61 participants in the FEMuR study. For emergency department (ED) attendances, records matched in 23 (47%) cases, 21 (43%) over-reported on electronic records compared with CSRI and five participants (10%) under-reported, with an overall ICC of 0.42. For out-patient episodes, records matched in only six cases, 28 participants over-reported on electronic records compared with CSRI and 15 (12%) under-reported, with an overall ICC of only 0.27. For in-patient days, records matched exactly in only five cases (10%), but if an error margin of 7 days was allowed, then agreement rose to 39 (66%) cases, and the overall ICC for all data was 0.88.It was only possible to compare prescribing data for 12 participants. For prescribing data, the SAIL data reported 117 out of 118 items (99%) and the CSRI only 89 (79%) items.ConclusionsThe use of routinely collected data has the potential to improve the efficiency of trials and other studies. Although the methodology to make the data available has been demonstrated, the data obtained was incomplete and the validity of using this method remains to be demonstrated.Trial registrationTrial registration: ISRCTN22464643 Registered 21 July 2014.

Project description:BackgroundIn low- and middle-income countries, budget impact is an important criterion for funding new interventions, particularly for large public health investments such as new vaccines. However, budget impact analyses remain less frequently conducted and less well researched than cost-effectiveness analyses.ObjectiveThe objective of this study was to fill the gap in research on budget impact analyses by assessing (1) the quality of stand-alone budget impact analyses, and (2) the feasibility of extending cost-effectiveness analyses to capture budget impact.MethodsWe developed a budget impact analysis checklist and scoring system for budget impact analyses, which we then adapted for cost-effectiveness analyses, based on current International Society for Pharmacoeconomics and Outcomes Research Task Force recommendations. We applied both budget impact analysis and cost-effectiveness analysis checklists and scoring systems to examine the extent to which existing economic evaluations provide sufficient evidence about budget impact to enable decision making. We used rotavirus vaccination as an illustrative case in which low- and middle-income countries uptake has been limited despite demonstrated cost effectiveness. A systematic literature review was conducted to identify economic evaluations of rotavirus vaccine in low- and middle-income countries published between January 2000 and February 2017. We critically appraised the quality of budget impact analyses, and assessed the extension of cost-effectiveness analyses to provide useful budget impact information.ResultsSix budget impact analyses and 60 cost-effectiveness analyses were identified. Budget impact analyses adhered to most International Society for Pharmacoeconomics and Outcomes Research recommendations, with key exceptions being provision of undiscounted financial streams for each budget period and model validation. Most cost-effectiveness analyses could not be extended to provide useful budget impact information; cost-effectiveness analyses also rarely presented undiscounted annual costs, or estimated financial streams during the first years of programme scale-up.ConclusionsCost-effectiveness analyses vastly outnumber budget impact analyses of rotavirus vaccination, despite both being critical for policy decision making. Straightforward changes to the presentation of cost-effectiveness analyses results could facilitate their adaptation into budget impact analyses.

Project description:BackgroundIndependent inquiries have identified that appropriate staffing in maternity units is key to enabling quality care and minimising harm, but optimal staffing levels can be difficult to achieve when there is a shortage of midwives. The services provided and how they are staffed (total staffing, skill-mix and deployment) have been changing, and the effects of workforce changes on care quality and outcomes have not been assessed. This study aims to explore the association between daily midwifery staffing levels and the rate of reported harmful incidents affecting mothers and babies.MethodsWe conducted a cross-sectional analysis of daily reports of clinical incidents in maternity inpatient areas matched with inpatient staffing levels for three maternity services in England, using data from April 2015 to February 2020. Incidents resulting in harm to mothers or babies was the primary outcome measure. Staffing levels were calculated from daily staffing rosters, quantified in Hours Per Patient Day (HPPD) for midwives and maternity assistants. Understaffing was defined as staffing below the mean for the service. A negative binomial hierarchical model was used to assess the relationship between exposure to low staffing and reported incidents involving harm.ResultsThe sample covered 106,904 maternal admissions over 46 months. The rate of harmful incidents in each of the three services ranged from 2.1 to 3.0 per 100 admissions across the study period. Understaffing by registered midwives was associated with an 11% increase in harmful incidents (adjusted IRR 1.110, 95% CI 1.002,1.229). Understaffing by maternity assistants was not associated with an increase in harmful incidents (adjusted IRR 0.919, 95% 0.813,1.039). Analysis of specific types of incidents showed no statistically significant associations, but most of the point estimates were in the direction of increased incidents when services were understaffed.ConclusionWhen there is understaffing by registered midwives, more harmful incidents are reported but understaffing by maternity assistants is not associated with higher risk of harms. Adequate registered midwife staffing levels are crucial for maintaining safety. Changes in the profile of maternity service workforces need to be carefully scrutinised to prevent mothers and babies being put at risk of avoidable harm.

Project description:Importance:Inpatient violence remains a significant problem despite existing risk assessment methods. The lack of robustness and the high degree of effort needed to use current methods might be mitigated by using routinely registered clinical notes. Objective:To develop and validate a multivariable prediction model for assessing inpatient violence risk based on machine learning techniques applied to clinical notes written in patients' electronic health records. Design, Setting, and Participants:This prognostic study used retrospective clinical notes registered in electronic health records during admission at 2 independent psychiatric health care institutions in the Netherlands. No exclusion criteria for individual patients were defined. At site 1, all adults admitted between January 2013 and August 2018 were included, and at site 2 all adults admitted to general psychiatric wards between June 2016 and August 2018 were included. Data were analyzed between September 2018 and February 2019. Main Outcomes and Measures:Predictive validity and generalizability of prognostic models measured using area under the curve (AUC). Results:Clinical notes recorded during a total of 3189 admissions of 2209 unique individuals at site 1 (mean [SD] age, 34.0 [16.6] years; 1536 [48.2%] male) and 3253 admissions of 1919 unique individuals at site 2 (mean [SD] age, 45.9 [16.6] years; 2097 [64.5%] male) were analyzed. Violent outcome was determined using the Staff Observation Aggression Scale-Revised. Nested cross-validation was used to train and evaluate models that assess violence risk during the first 4 weeks of admission based on clinical notes available after 24 hours. The predictive validity of models was measured at site 1 (AUC = 0.797; 95% CI, 0.771-0.822) and site 2 (AUC = 0.764; 95% CI, 0.732-0.797). The validation of pretrained models in the other site resulted in AUCs of 0.722 (95% CI, 0.690-0.753) at site 1 and 0.643 (95% CI, 0.610-0.675) at site 2; the difference in AUCs between the internally trained model and the model trained on other-site data was significant at site 1 (AUC difference = 0.075; 95% CI, 0.045-0.105; P < .001) and site 2 (AUC difference = 0.121; 95% CI, 0.085-0.156; P < .001). Conclusions and Relevance:Internally validated predictions resulted in AUC values with good predictive validity, suggesting that automatic violence risk assessment using routinely registered clinical notes is possible. The validation of trained models using data from other sites corroborates previous findings that violence risk assessment generalizes modestly to different populations.

Dataset Information

Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future.

Background

Main text

Conclusions

Publications

Self-reported and routinely collected electronic healthcare resource-use data for trial-based economic evaluations: the current state of play in England and considerations for the future.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets