Dataset Information

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

ABSTRACT: BACKGROUND:Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT) -based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events (AEs), and explore the utility of this method in patient-reported outcome (PRO) and health-related quality of life (HRQOL) research. METHODS:Data were derived from a National Cancer Institute-sponsored study examining the validity of a measurement system (PRO-CTCAE) for patient self-reporting of AEs in cancer patients receiving treatment (N?=?940). AEs included 13 multiple attribute patient-reported symptoms that had corresponding unidimensional clinician AE grades. A Bayesian IRT Model was fitted to calculate the latent grading thresholds between raters. The posterior mean values of the model-fitted item responses were calculated to represent model-based AE grades obtained from patients and clinicians. RESULTS:Model-based AE grades showed a general pattern of clinician underestimation relative to patient-graded AEs. However, the magnitude of clinician underestimation was associated with AE severity, such that clinicians' underestimation was more pronounced for moderate/very severe model-estimated AEs, and less so with mild AEs. CONCLUSIONS:The Bayesian IRT approach reconciles multiple symptom attributes and elaborates on the patterns of clinician-patient non-concordance beyond that provided by traditional metrics. This IRT-based technique may be used as a supplemental tool to detect and characterize nuanced differences in patient-, clinician-, and proxy-based ratings of HRQOL and patient-centered outcomes. TRIAL REGISTRATION:ClinicalTrials.gov NCT01031641 . Registered 1 December 2009.

SUBMITTER: Atkinson TM

PROVIDER: S-EPMC6279753 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Atkinson Thomas M TM Reeve Bryce B BB Dueck Amylou C AC Bennett Antonia V AV Mendoza Tito R TR Rogak Lauren J LJ Basch Ethan E Li Yuelin Y

Journal of patient-reported outcomes 20181204 1

<h4>Background</h4>Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT) -based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events ...[more]

PMID: 30515599

Similar Datasets

Project description:BackgroundIncorporating patient reporting of symptomatic adverse events (AEs) is important in evaluating safety and tolerability in cancer clinical trials. The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed to assess the frequency, severity, and/or interference of patient-reported symptomatic AEs. The objective of this study was to identify, based on oncologist and patient interviews, a relevant subset of symptomatic AEs from the PRO-CTCAE that can be used to optimize patient reporting of symptomatic AEs in hepatocellular carcinoma (HCC) clinical trials.MethodsQualitative and quantitative data on HCC diagnosis, treatment, symptoms, and side effects were collected from patients. Using a numerical rating scale, medical oncologists specializing in HCC rated the importance of (1) 34 symptomatic AEs (grade ≥ 2) identified in past sponsor HCC clinical trials, (2) each PRO-CTCAE symptomatic AE item to the patient, and (3) each PRO-CTCAE symptomatic AE item with respect to patient safety or tolerability. Patients completed the PRO-CTCAE items and were debriefed on the importance of each PRO-CTCAE symptomatic AE to them.ResultsFive medical oncologists from the United States, Spain, Taiwan, Korea, and Hong Kong with 14 to 30 years of experience, and 17 patients with HCC and Child-Pugh class A or B cirrhosis status completed interviews. Medical oncologists rated the following symptomatic AEs from prior trials as being highly important to patients (mean rating of ≥7 on a scale from 0 to 10): hand-foot syndrome, diarrhea, fatigue, decreased appetite, rash, vomiting, and weight loss. PRO-CTCAE symptomatic AEs rated by medical oncologists as being highly important to patients included diarrhea, vomiting, shivering or shaking chills, hand-foot syndrome, rash, fatigue, difficulty swallowing, decreased appetite, and loss of control of bowel movements. Patients rated the following PRO-CTCAE symptomatic AEs as being highly important: loss of appetite/lack of interest in food, pain/tenderness at injection/insertion site, fatigue/lack of energy/tiredness, nausea, and hair loss.ConclusionsThis study identified a preliminary list of clinically relevant symptomatic AEs from interviews with both medical oncologists and patients that can be used to support assessments of treatment safety and tolerability in HCC clinical trials.

Project description:BackgroundClinician reporting of symptomatic adverse events (AEs) in phase I trials uses the Common Terminology Criteria for Adverse Events (CTCAE). The utility of the patient-reported outcomes (PROs) version of the CTCAE (PRO-CTCAE) in this setting is unknown. This prospective, observational study compared patient- and clinician-reported symptomatic AEs in phase I patients.MethodsPhase I study-eligible patients at Princess Margaret were surveyed with the PRO-CTCAE full-item library (78 symptomatic AEs) at baseline (BL), mid-cycle 1, and mid-cycle 2 (C2). Patient and trial characteristics, best response, and survival data were collected. Presence or absence of patient- (PRO-CTCAE) or clinician-reported symptomatic AEs were compared (kappa) at defined timepoints and overall (BL+ mid-cycle 1 + C2).ResultsOf 292 patients approached from May 2017 to January 2019, a total of 265 (90.8%) were consented, with 243 (91.7%) evaluable and 552 PRO-CTCAE surveys (completion rate = 98.7%) included in analyses. Evaluation of overall patient-reported symptomatic AEs identified 50 PRO-CTCAE and 11 CTCAE items with 10% or greater reporting frequency. Nineteen CTCAE items were reported as 1% or less despite matched PRO-CTCAE items reporting as 10% or greater. Underreported categories included sexual health, bodily emissions, and cognition. Clinician- relative to patient-reporting frequency (ratio) demonstrated 9 symptomatic AEs with a 50-fold or more lower clinician reporting rate. Overall patient-clinician agreement for individual symptomatic AEs ranged from poor (κ = 0.00-0.19) to moderate (κ = 0.40-0.59), with discordance driven by lack of clinician reporting. Dyspnea (κ = 0.54) and peripheral neuropathy (κ = 0.63) at BL and limb edema (κ = 0.55) at C2 demonstrated the highest patient-clinician agreement.ConclusionsPoor to moderate patient-clinician agreement for symptomatic AEs suggests clinician underreporting in phase I trials. Analyses of severity and interference PRO categories are ongoing.

Project description:ImportanceIn cancer clinical trials, symptomatic adverse events (AEs), such as nausea, are reported by investigators rather than by patients. There is increasing interest to collect symptomatic AE data via patient-reported outcome (PRO) questionnaires, but it is unclear whether it is feasible to implement this approach in multicenter trials.ObjectiveTo examine whether patients are willing and able to report their symptomatic AEs in multicenter trials.Design, setting, and participantsA total of 361 consecutive patients enrolled in any 1 of 9 US multicenter cancer treatment trials were invited to self-report 13 common symptomatic AEs using a PRO adaptation of the National Cancer Institute's Common Terminology Criteria for Adverse Events (CTCAE) via tablet computers at 5 successive clinic visits. Patient adherence was tracked with reasons for missed self-reports. Agreement with clinician AE reports was analyzed with weighted κ statistics. Patient and investigator perspectives were elicited by survey. The study was conducted from March 15, 2007, to August 11, 2011. Data analysis was performed from August 9, 2013, to March 21, 2014.ResultsOf the 361 patients invited to participate, 285 individuals enrolled, with a median age of 57 years (range, 24-88), 202 (74.3%) female, 241 (85.5%) white, 73 (26.8%) with a high school education or less, and 176 (64.7%) who reported regular internet use (denominators varied owing to missing data). Across all patients and trials, there were 1280 visits during which patients had an opportunity to self-report (ie, patients were alive and enrolled in a treatment trial at the time of the visit). Self-reports were completed at 1202 visits (93.9% overall adherence). Adherence was highest at baseline and declined over time (visit 1, 100%; visit 2, 96%; visit 3, 95%; visit 4, 91%; and visit 5, 85%). Reasons for missing PROs included institutional errors in 27 of 48 (56.3%) of the cases (eg, staff forgetting to bring computers to patients at visits), patients feeling "too ill" in 8 (16.7%), patient refusal in 8 (16.7%), and internet connectivity problems in 5 (10.4%). Patient-investigator CTCAE agreement was moderate or worse for most symptoms (most κ < 0.05), with investigators reporting fewer AEs than patients across symptoms. Most patients believed that the system was easy to use (234 [93.2%]) and useful (230 [93.1%]), and investigators thought that the patient-reported AEs were useful (133 [94.3%]) and accurate (119 [83.2%]).Conclusions and relevanceParticipants in multicenter cancer trials are willing and able to report their own symptomatic AEs at most clinic visits and report more AEs than investigators. This approach may improve the precision of AE reporting in cancer trials.

Project description:Background/aimsRegulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.MethodsIn this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.ResultsOur simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.ConclusionIn the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.

Project description:Background:The study is based on a national cluster randomized trial investigating the effect of electronic patient-reported outcomes (ePRO) on treatment outcomes in breast cancer patients receiving adjuvant chemotherapy. All 13 oncology departments (11 clusters) treating breast cancer patients in Denmark were randomized to use electronic patient-reported outcomes with real-time clinician feedback (ePRO arm) to track symptoms or usual care for eliciting symptoms using a short paper tracking list (usual care arm). The impact of ePRO on clinical outcomes were examined, which is reported elsewhere. The purpose of the present study was to examine patient-reported experience measure (PREM) regarding communication and handling of side effects/symptoms. Methods:For this sub-study, patient representatives were involved in the development of a PREM questionnaire. Patients enrolled in the cluster randomized trial completed the PREM questionnaire at their last treatment visit. Semi-structured telephone-interviews were performed with a subgroup of patients. The interviews were based on an interview guide comprised of the questions from the PREM questionnaire and aimed to elaborate on the PREM questionnaire data. Results:A 12 item PREM questionnaire was developed in partnership with patient representatives. In total, 439 out of 682 patients (64.4%) included patients completed the PREM questionnaire. Telephone semi-structured interviews were performed with 22 patients. In total, 52% (ePRO arm) and 65% (usual care arm) reported having talked with the oncologist/nurse about their responses in the tracking systems before each chemotherapy cycle. Fewer patients in the ePRO arm compared to the usual care arm experienced side effects/symptoms not included in the side effect questionnaire. Patients experienced high satisfaction with oncologists' and nurses' handling of side effects/symptoms. Conclusions:Patients experienced high satisfaction with oncologists' and nurses' handling of chemotherapy adverse events. The study indicates a need for a more comprehensive side effect questionnaire as tracking system covering more symptoms than the one used in usual care today. Trial registration:Clinicaltrials.gov identifier NCT02996201. Registered 19 December 2016, retrospectively registered.

Project description:OBJECTIVE:The study sought to describe patient-entered supplemental information on symptomatic adverse events (AEs) in cancer clinical research reported via a National Cancer Institute software system and examine the feasibility of mapping these entries to established terminologies. MATERIALS AND METHODS:Patients in 3 multicenter trials electronically completed surveys during cancer treatment. Each survey included a prespecified subset of items from the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Upon completion of the survey items, patients could add supplemental symptomatic AE information in a free text box. As patients typed into the box, structured dropdown terms could be selected from the PRO-CTCAE item library or Medical Dictionary for Regulatory Activities (MedDRA), or patients could type unstructured free text for submission. RESULTS:Data were pooled from 1760 participants (48% women; 78% White) who completed 8892 surveys, of which 2387 (26.8%) included supplemental symptomatic AE information. Overall, 1024 (58%) patients entered supplemental information at least once, with an average of 2.3 per patient per study. This encompassed 1474 of 8892 (16.6%) dropdowns and 913 of 8892 (10.3%) unstructured free text entries. One-third of the unstructured free text entries (32%) could be mapped post hoc to a PRO-CTCAE term and 68% to a MedDRA term. DISCUSSION:Participants frequently added supplemental information beyond study-specific survey items. Almost half selected a structured dropdown term, although many opted to submit unstructured free text entries. Most free text entries could be mapped post hoc to PRO-CTCAE or MedDRA terms, suggesting opportunities to enhance the system to perform real-time mapping for AE reporting. CONCLUSIONS:Patient reporting of symptomatic AEs using a text box functionality with mapping to existing terminologies is both feasible and informative.

Dataset Information

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Publications

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure