Dataset Information

Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols.

ABSTRACT: Qualitative evidence suggests patient-reported outcome (PRO) information is frequently absent from clinical trial protocols, potentially leading to inconsistent PRO data collection and risking bias. Direct evidence regarding PRO trial protocol content is lacking. The aim of this study was to systematically evaluate the PRO-specific content of UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme trial protocols.We conducted an electronic search of the NIHR HTA programme database (inception to August 2013) for protocols describing a randomised controlled trial including a primary/secondary PRO. Two investigators independently reviewed the content of each protocol, using a specially constructed PRO-specific protocol checklist, alongside the 'Standard Protocol Items: Recommendations for Interventional Trials' (SPIRIT) checklist. Disagreements were resolved through discussion with a third investigator. 75 trial protocols were included in the analysis. Protocols included a mean of 32/51 (63%) SPIRIT recommendations (range 16-41, SD 5.62) and 11/33 (33%) PRO-specific items (range 4-18, SD 3.56). Over half (61%) of the PRO items were incomplete. Protocols containing a primary PRO included slightly more PRO checklist items (mean 14/33 (43%)). PRO protocol content was not associated with general protocol completeness; thus, protocols judged as relatively 'complete' using SPIRIT were still likely to have omitted a large proportion of PRO checklist items.The PRO components of HTA clinical trial protocols require improvement. Information on the PRO rationale/hypothesis, data collection methods, training and management was often absent. This low compliance is unsurprising; evidence shows existing PRO guidance for protocol developers remains difficult to access and lacks consistency. Study findings suggest there are a number of PRO protocol checklist items that are not fully addressed by the current SPIRIT statement. We therefore advocate the development of consensus-based supplementary guidelines, aimed at improving the completeness and quality of PRO content in clinical trial protocols.

SUBMITTER: Kyte D

PROVIDER: S-EPMC4198237 | biostudies-literature | 2014

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols.

Kyte Derek D Duffy Helen H Fletcher Benjamin B Gheorghe Adrian A Mercieca-Bebber Rebecca R King Madeleine M Draper Heather H Ives Jonathan J Brundage Michael M Blazeby Jane J Calvert Melanie M

PloS one 20141015 10

<h4>Background</h4>Qualitative evidence suggests patient-reported outcome (PRO) information is frequently absent from clinical trial protocols, potentially leading to inconsistent PRO data collection and risking bias. Direct evidence regarding PRO trial protocol content is lacking. The aim of this study was to systematically evaluate the PRO-specific content of UK National Institute for Health Research (NIHR) Health Technology Assessment (HTA) programme trial protocols.<h4>Methods and findings</ ...[more]

PMID: 25333349

Similar Datasets

Project description:IntroductionFailure to incorporate key patient-reported outcome (PRO) content in trial protocols affects the quality and interpretability of the collected data, contributing to research waste. Our group developed evidence-based training specifically addressing PRO components of protocols. We aimed to assess whether 2-day educational workshops improved the PRO completeness of protocols against consensus-based minimum standards provided in the SPIRIT-PRO Extension in 2018.MethodAnnual workshops were conducted 2011-2017. Participants were investigators/trialists from cancer clinical trials groups. Although developed before 2018, workshops covered 15/16 SPIRIT-PRO items. Participant feedback immediately post-workshop and, retrospectively, in November 2017 was summarised descriptively. Protocols were evaluated against SPIRIT-PRO by two independent raters for workshop protocols (developed post-workshop by participants) and control protocols (contemporaneous non-workshop protocols). SPIRIT-PRO items were assessed for completeness (0 = not addressed, 10 = fully addressed). Mann-Whitney U tests assessed whether workshop protocols scored higher than controls by item and overall.ResultsParticipants (n = 107) evaluated the workshop positively. In 2017, 16/41 survey responders (39%) reported never applying in practice; barriers included role restrictions (14/41, 34%) and lack of time (5/41, 12%). SPIRIT-PRO overall scores did not differ between workshop (n = 13, median = 3.81/10, interquartile range = 3.24) and control protocols (n = 9, 3.51/10 (2.14)), (p = 0.35). Workshop protocols scored higher than controls on two items: 'specify PRO concepts/domains' (p = 0.05); 'methods for handling missing data' (p = 0.044).ConclusionAlthough participants were highly satisfied with these workshops, the completeness of PRO protocol content generally did not improve. Additional knowledge translation efforts are needed to assist protocol writers address SPIRIT-PRO guidance and avoid research waste that may eventuate from sub-optimal PRO protocol content.

Project description:PURPOSE:The CONSORT extension for patient reported outcomes (PROs) aims to improve reporting, but guidance on the optimal integration with clinical data is lacking. This study examines in detail the reporting of PROs and clinical data from randomized controlled trials (RCTs) in gastro-intestinal cancer to inform design and reporting of combined PRO and clinical data from trials to improve the 'take home' message for clinicians to use in practice. MATERIALS AND METHODS:The case study was undertaken in gastro-intestinal cancer trials. Well-conducted RCTs reporting PROs with validated instruments were identified and categorized into those combining PRO and clinical data in a single paper, or those separating data into linked primary and supplemental papers. Qualitative methods were developed to examine reporting of the critical interpretation of the trial results (trial exegesis) in the papers in relation of the PRO and clinical outcomes and applied to each publication category. Results were used to inform recommendations for practice. RESULTS:From 1917 screened abstracts, 49 high quality RCTs were identified reported in 36 combined and 15 linked primary and supplemental papers. In-depth analysis of manuscript text identified three categories for understanding trial exegesis: where authors reported a "detailed", "general", or absent PRO rationale and integrated interpretation of clinical and PRO results. A total of 11 (30%) and 6 (16%) combined papers reported "detailed" PRO rationale and integrated interpretation of results although only 2 (14%) and 1 (7%) primary papers achieved the same standard respectively. Supplemental papers provide better information with 11 (73%) and 3 (20%) achieving "detailed" rationale and integrated interpretation of results. Supplemental papers, however, were published a median of 20 months after the primary RCT data in lower impact factor journals (median 16.8 versus 5.2). CONCLUSION:It is recommended that single papers, with detailed PRO rationale and integrated PRO and clinical data are published to optimize trial exegesis. Further work to examine whether this improves the use of PRO data to inform practice is needed.

Project description:BackgroundEvidence suggests there are inconsistencies in patient-reported outcome (PRO) assessment and reporting in clinical trials, which may limit the use of these data to inform patient care. For trials with a PRO endpoint, routine inclusion of key PRO information in the protocol may help improve trial conduct and the reporting and appraisal of PRO results; however, it is currently unclear exactly what PRO-specific information should be included. The aim of this review was to summarize the current PRO-specific guidance for clinical trial protocol developers.Methods and findingsWe searched the MEDLINE, EMBASE, CINHAL and Cochrane Library databases (inception to February 2013) for PRO-specific guidance regarding trial protocol development. Further guidance documents were identified via Google, Google scholar, requests to members of the UK Clinical Research Collaboration registered clinical trials units and international experts. Two independent investigators undertook title/abstract screening, full text review and data extraction, with a third involved in the event of disagreement. 21,175 citations were screened and 54 met the inclusion criteria. Guidance documents were difficult to access: electronic database searches identified just 8 documents, with the remaining 46 sourced elsewhere (5 from citation tracking, 27 from hand searching, 7 from the grey literature review and 7 from experts). 162 unique PRO-specific protocol recommendations were extracted from included documents. A further 10 PRO recommendations were identified relating to supporting trial documentation. Only 5/162 (3%) recommendations appeared in ≥50% of guidance documents reviewed, indicating a lack of consistency.ConclusionsPRO-specific protocol guidelines were difficult to access, lacked consistency and may be challenging to implement in practice. There is a need to develop easily accessible consensus-driven PRO protocol guidance. Guidance should be aimed at ensuring key PRO information is routinely included in appropriate trial protocols, in order to facilitate rigorous collection/reporting of PRO data, to effectively inform patient care.

Project description:BackgroundPatient-reported outcomes (PROs) are captured within cancer trials to help future patients and their clinicians make more informed treatment decisions. However, variability in standards of PRO trial design and reporting threaten the validity of these endpoints for application in clinical practice.MethodsWe systematically investigated a cohort of randomized controlled cancer trials that included a primary or secondary PRO. For each trial, an evaluation of protocol and reporting quality was undertaken using standard checklists. General patterns of reporting where also explored.ResultsProtocols (101 sourced, 44.3%) included a mean (SD) of 10 (4) of 33 (range = 2-19) PRO protocol checklist items. Recommended items frequently omitted included the rationale and objectives underpinning PRO collection and approaches to minimize/address missing PRO data. Of 160 trials with published results, 61 (38.1%, 95% confidence interval = 30.6% to 45.7%) failed to include their PRO findings in any publication (mean 6.43-year follow-up); these trials included 49 568 participants. Although two-thirds of included trials published PRO findings, reporting standards were often inadequate according to international guidelines (mean [SD] inclusion of 3 [3] of 14 [range = 0-11]) CONSORT PRO Extension checklist items). More than one-half of trials publishing PRO results in a secondary publication (12 of 22, 54.5%) took 4 or more years to do so following trial closure, with eight (36.4%) taking 5-8 years and one trial publishing after 14 years.ConclusionsPRO protocol content is frequently inadequate, and nonreporting of PRO findings is widespread, meaning patient-important information may not be available to benefit patients, clinicians, and regulators. Even where PRO data are published, there is often considerable delay and reporting quality is suboptimal. This study presents key recommendations to enhance the likelihood of successful delivery of PROs in the future.

Project description:BackgroundPatient-reported outcomes (PROs) are commonly collected in clinical trials and should provide impactful evidence on the effect of interventions on patient symptoms and quality of life. However, it is unclear how PRO impact is currently realised in practice. In addition, the different types of impact associated with PRO trial results, their barriers and facilitators, and appropriate impact metrics are not well defined. Therefore, our objectives were: i) to determine the range of potential impacts from PRO clinical trial data, ii) identify potential PRO impact metrics and iii) identify barriers/facilitators to maximising PRO impact; and iv) to examine real-world evidence of PRO trial data impact based on Research Excellence Framework (REF) impact case studies.MethodsTwo independent investigators searched MEDLINE, EMBASE, CINAHL+, HMIC databases from inception until December 2018. Articles were eligible if they discussed research impact in the context of PRO clinical trial data. In addition, the REF 2014 database was systematically searched. REF impact case studies were included if they incorporated PRO data in a clinical trial.ResultsThirty-nine publications of eleven thousand four hundred eighty screened met the inclusion criteria. Nine types of PRO trial impact were identified; the most frequent of which centred around PRO data informing clinical decision-making. The included publications identified several barriers and facilitators around PRO trial design, conduct, analysis and report that can hinder or promote the impact of PRO trial data. Sixty-nine out of two hundred nine screened REF 2014 case studies were included. 12 (17%) REF case studies led to demonstrable impact including changes to international guidelines; national guidelines; influencing cost-effectiveness analysis; and influencing drug approvals.ConclusionsPRO trial data may potentially lead to a range of benefits for patients and society, which can be measured through appropriate impact metrics. However, in practice there is relatively limited evidence demonstrating directly attributable and indirect real world PRO-related research impact. In part, this is due to the wider challenges of measuring the impact of research and PRO-specific issues around design, conduct, analysis and reporting. Adherence to guidelines and multi-stakeholder collaboration is essential to maximise the use of PRO trial data, facilitate impact and minimise research waste.Trial registrationSystematic Review registration PROSPERO CRD42017067799.

Project description:BackgroundThe quality of life of patients at all stages of hematological malignancy is greatly affected by the disease and its treatment. There is a wide range of health-related quality of life (HRQoL) issues important to these patients. Any new instrument developed to measure HRQoL of such patients should be content valid, i.e., the items should be comprehensively relevant to the patients and their health condition. The aim of the present study was to examine content validity of a hematological malignancy specific patient reported outcome measure (HM-PRO) developed for use in routine clinical practice.MethodsFollowing literature review and semi-structured interviews, the generated themes and sub-themes were discussed to develop the prototype version of the HM-PRO. A 4-step approach was used for content validation: initial testing and cognitive interviewing; item rating; content validity panel meeting; final field testing and cognitive interviewing. Additional questions related to patients' perception of recall period and preferred sentence structure (i.e., question or statement) of the items were also asked during cognitive interviews.ResultsThe content analysis of 129 transcribed semi-structured interviews resulted in the prototype version of the instrument consisting of 58 items grouped into two parts: Part A (impact/HRQoL - 34 items) and Part B (signs and symptoms - 24 items). The initial testing showed intra-class correlation coefficient (ICC) of >0.8 for both Part A and Part B. Item rating for language clarity, completeness, relevance, and response scale by experts and patients showed content validity index for scales average >0.8 for both Part A and Part B, except 0.64 for relevance for Part A by the patient panel. The final testing of the revised version of the instrument showed the Cronbach's alpha value of 0.91 for Part A and 0.76 for Part B, suggesting high internal consistency, and ICC of 0.91 for Part A and 0.76 for Part B. The recall period of "today" for Part-A and "last 3 days" for Part-B were the patients' preferred "recall period." Furthermore, the patients expressed preference to the HM-PRO items as statements.ConclusionThe findings of this study confirm that the HM-PRO possesses a strong content validity, includes all the issues important to patients and is easy to read, understand and respond to spontaneously.

Project description:ObjectiveThe Lupus Foundation of America Rapid Evaluation of Activity in Lupus (LFA-REAL) system is a novel and simple SLE disease activity instrument, consisting of a tandem clinician-reported (ClinRO) and patient-reported (PRO) outcome measure. The aim of this study was to compare the LFA-REAL system with other SLE activity measures in the phase III trial of ustekinumab in patients with active SLE.MethodsThis was a prespecified analysis of data from a randomised, double-blind, placebo-controlled, parallel-group trial conducted at 140 sites in 20 countries. Correlations were evaluated between the LFA-REAL ClinRO and PRO with a panel of clinician-reported and patient-reported disease activity measures commonly used in SLE clinical trials at baseline, week 24 and week 52. All p values are reported as nominal.ResultsTrial participants included 516 patients with SLE with a mean (SD) age of 43.5 (8.9), of whom 482 (93.4%) were female. The LFA-REAL ClinRO correlated with Physician Global Assessment (r=0.39, 0.65 and 0.74, p<0.001), British Isles Lupus Assessment Group Index (r=0.43, 0.67 and 0.73, p<0.001) and SLE Disease Activity Index-2000 (r=0.35, 0.60 and 0.62, p<0.001). The LFA-REAL ClinRO arthralgia/arthritis score correlated well with active joint counts (r=0.54, 0.73 and 0.68, p<0.001) and the mucocutaneous global score correlated strongly with Cutaneous Lupus Erythematosus Disease Area and Severity Index total activity (r=0.57, 0.77 and 0.81, p<0.001). The LFA-REAL PRO demonstrated a moderate correlation with Functional Assessment of Chronic Illness Therapy-Fatigue (r=-0.60, -0.55 and -0.58, p<0.001), Lupus QoL physical health (r=-0.42, -0.47 and -0.46, p<0.001), SF-36v2 vitality (r=-0.40, -0.43 and -0.58, p<0.001) and SF-36v2 Physical Component Summary (r=-0.45, -0.53 and -0.53, p<0.001). The LFA-REAL ClinRO and PRO showed a moderate correlation with each other (r=0.32, 0.45 and 0.50, p<0.001).ConclusionsThe LFA-REAL ClinRO and PRO showed varied levels of correlations (weak to strong) with existing physician-based lupus disease activity measures and patient-reported outcome instruments, respectively and were able to more accurately capture organ-specific mucocutaneous and musculoskeletal manifestations. More analyses are needed to determine areas in which patient-reported outcomes are most similar or different to physician-reported end points and the basis for differences.

Project description:BackgroundGout is a common, monosodium urate crystal-driven inflammatory arthritis. Besides its clinical manifestations, patients often also suffer from pain, physical impairment, emotional distress and work productivity loss, as a result of the disease. Patient-reported outcome measures (PROMs) are commonly used to assess these consequences of the disease. However, current instrument endorsements for measuring such outcomes in acute and chronic gout clinical settings are based on limited psychometric evidence. The objective of this systematic literature review was to identify currently available PROMs for gout, and to critically evaluate their content and psychometric properties, in order to evaluate the current status regarding PROMs for use in gout patients.MethodsSystematic literature searches were performed in the PubMed and EMBASE databases. The methodological quality of included papers was appraised using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) checklist, and evaluation of measurement properties (reliability, responsiveness, construct validity, floor and ceiling effects) was done in accordance with published quality criteria. Item content was appraised by linking health concepts to the International Classification of Functioning Disability and Health (ICF) framework.ResultsIn total, 13 PROMs were identified, of which three were targeted specifically at gout patients. The majority of the PROMs were rated positively for content validity. For most instruments, limited evidence was available for construct validity and reliability. Instruments to assess pain scored well on responsiveness and floor and ceiling effects, but not much is known about their reliability in gout.ConclusionsThe physical functioning subscale of the SF-36v2 (Short Form-36 item version 2) is the only PROM that had sufficient supporting evidence for all its psychometric properties. Many of the commonly used PROMs in gout are currently not yet well supported and more studies on their measurement properties are needed among both acute and chronic gout populations.

Project description:BackgroundAssessment of patient-reported outcomes (PROs) provides valuable information to inform patient-centered care, but may also reveal 'PRO alerts': psychological distress or physical symptoms that may require an immediate response. Ad-hoc management of PRO alerts in clinical trials may result in suboptimal patient care or potentially bias trial results. To gain greater understanding of current practice in PRO alert management we conducted a national survey of personnel involved in clinical trials with a PRO endpoint.Methods and findingsWe conducted a national cross-sectional survey of 767 UK-based research nurses, data managers/coordinators, trial managers and chief/principal investigators involved in clinical trials using PROs. Respondents were self-selected volunteers from a non-randomised sample of eligible individuals recruited via 55 UK Clinical Research Collaboration Registered Clinical Trials Units and 19 Comprehensive Local Research Networks. Questions centred on the proportion of trial personnel encountering alerts, how staff responded to PRO alerts and whether current guidance was deemed sufficient to support research personnel. We undertook descriptive analyses of the quantitative data and directed thematic analysis of free-text comments. 20% of research nurses did not view completed PRO questionnaires and were not in a position to discover alerts, 39-50% of the remaining respondent group participants reported encountering PRO alerts. Of these, 83% of research nurses and 54% of data managers/trial coordinators reported taking action to assist the trial participant, but less than half were able to record the intervention in the trial documentation. Research personnel reported current PRO alert guidance/training was insufficient.ConclusionsResearch personnel are intermittently exposed to PRO alerts. Some intervene to help trial participants, but are not able to record this intervention in the trial documentation, risking co-intervention bias. Other staff do not check PRO information during the trial, meaning alerts may remain undiscovered, or do not respond to alerts if they are inadvertently encountered; both of which may impact on patient safety. Guidance is needed to support PRO alert management that protects the interests of trial participants whilst avoiding potential bias.

Dataset Information

Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols.

Publications

Systematic evaluation of the patient-reported outcome (PRO) content of clinical trial protocols.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets