Dataset Information

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

ABSTRACT: BACKGROUND:Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT) -based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events (AEs), and explore the utility of this method in patient-reported outcome (PRO) and health-related quality of life (HRQOL) research. METHODS:Data were derived from a National Cancer Institute-sponsored study examining the validity of a measurement system (PRO-CTCAE) for patient self-reporting of AEs in cancer patients receiving treatment (N?=?940). AEs included 13 multiple attribute patient-reported symptoms that had corresponding unidimensional clinician AE grades. A Bayesian IRT Model was fitted to calculate the latent grading thresholds between raters. The posterior mean values of the model-fitted item responses were calculated to represent model-based AE grades obtained from patients and clinicians. RESULTS:Model-based AE grades showed a general pattern of clinician underestimation relative to patient-graded AEs. However, the magnitude of clinician underestimation was associated with AE severity, such that clinicians' underestimation was more pronounced for moderate/very severe model-estimated AEs, and less so with mild AEs. CONCLUSIONS:The Bayesian IRT approach reconciles multiple symptom attributes and elaborates on the patterns of clinician-patient non-concordance beyond that provided by traditional metrics. This IRT-based technique may be used as a supplemental tool to detect and characterize nuanced differences in patient-, clinician-, and proxy-based ratings of HRQOL and patient-centered outcomes. TRIAL REGISTRATION:ClinicalTrials.gov NCT01031641 . Registered 1 December 2009.

SUBMITTER: Atkinson TM

PROVIDER: S-EPMC6279753 | biostudies-literature | 2018 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Atkinson Thomas M TM Reeve Bryce B BB Dueck Amylou C AC Bennett Antonia V AV Mendoza Tito R TR Rogak Lauren J LJ Basch Ethan E Li Yuelin Y

Journal of patient-reported outcomes 20181204 1

<h4>Background</h4>Traditional concordance metrics have shortcomings based on dataset characteristics (e.g., multiple attributes rated, missing data); therefore it is necessary to explore supplemental approaches to quantifying agreement between independent assessments. The purpose of this methodological paper is to apply an Item Response Theory (IRT) -based framework to an existing dataset that included unidimensional clinician and multiple attribute patient ratings of symptomatic adverse events ...[more]

PMID: 30515599

Similar Datasets

Project description:BackgroundIncorporating patient reporting of symptomatic adverse events (AEs) is important in evaluating safety and tolerability in cancer clinical trials. The Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) was developed to assess the frequency, severity, and/or interference of patient-reported symptomatic AEs. The objective of this study was to identify, based on oncologist and patient interviews, a relevant subset of symptomatic AEs from the PRO-CTCAE that can be used to optimize patient reporting of symptomatic AEs in hepatocellular carcinoma (HCC) clinical trials.MethodsQualitative and quantitative data on HCC diagnosis, treatment, symptoms, and side effects were collected from patients. Using a numerical rating scale, medical oncologists specializing in HCC rated the importance of (1) 34 symptomatic AEs (grade ≥ 2) identified in past sponsor HCC clinical trials, (2) each PRO-CTCAE symptomatic AE item to the patient, and (3) each PRO-CTCAE symptomatic AE item with respect to patient safety or tolerability. Patients completed the PRO-CTCAE items and were debriefed on the importance of each PRO-CTCAE symptomatic AE to them.ResultsFive medical oncologists from the United States, Spain, Taiwan, Korea, and Hong Kong with 14 to 30 years of experience, and 17 patients with HCC and Child-Pugh class A or B cirrhosis status completed interviews. Medical oncologists rated the following symptomatic AEs from prior trials as being highly important to patients (mean rating of ≥7 on a scale from 0 to 10): hand-foot syndrome, diarrhea, fatigue, decreased appetite, rash, vomiting, and weight loss. PRO-CTCAE symptomatic AEs rated by medical oncologists as being highly important to patients included diarrhea, vomiting, shivering or shaking chills, hand-foot syndrome, rash, fatigue, difficulty swallowing, decreased appetite, and loss of control of bowel movements. Patients rated the following PRO-CTCAE symptomatic AEs as being highly important: loss of appetite/lack of interest in food, pain/tenderness at injection/insertion site, fatigue/lack of energy/tiredness, nausea, and hair loss.ConclusionsThis study identified a preliminary list of clinically relevant symptomatic AEs from interviews with both medical oncologists and patients that can be used to support assessments of treatment safety and tolerability in HCC clinical trials.

Project description:BackgroundClinician reporting of symptomatic adverse events (AEs) in phase I trials uses the Common Terminology Criteria for Adverse Events (CTCAE). The utility of the patient-reported outcomes (PROs) version of the CTCAE (PRO-CTCAE) in this setting is unknown. This prospective, observational study compared patient- and clinician-reported symptomatic AEs in phase I patients.MethodsPhase I study-eligible patients at Princess Margaret were surveyed with the PRO-CTCAE full-item library (78 symptomatic AEs) at baseline (BL), mid-cycle 1, and mid-cycle 2 (C2). Patient and trial characteristics, best response, and survival data were collected. Presence or absence of patient- (PRO-CTCAE) or clinician-reported symptomatic AEs were compared (kappa) at defined timepoints and overall (BL+ mid-cycle 1 + C2).ResultsOf 292 patients approached from May 2017 to January 2019, a total of 265 (90.8%) were consented, with 243 (91.7%) evaluable and 552 PRO-CTCAE surveys (completion rate = 98.7%) included in analyses. Evaluation of overall patient-reported symptomatic AEs identified 50 PRO-CTCAE and 11 CTCAE items with 10% or greater reporting frequency. Nineteen CTCAE items were reported as 1% or less despite matched PRO-CTCAE items reporting as 10% or greater. Underreported categories included sexual health, bodily emissions, and cognition. Clinician- relative to patient-reporting frequency (ratio) demonstrated 9 symptomatic AEs with a 50-fold or more lower clinician reporting rate. Overall patient-clinician agreement for individual symptomatic AEs ranged from poor (κ = 0.00-0.19) to moderate (κ = 0.40-0.59), with discordance driven by lack of clinician reporting. Dyspnea (κ = 0.54) and peripheral neuropathy (κ = 0.63) at BL and limb edema (κ = 0.55) at C2 demonstrated the highest patient-clinician agreement.ConclusionsPoor to moderate patient-clinician agreement for symptomatic AEs suggests clinician underreporting in phase I trials. Analyses of severity and interference PRO categories are ongoing.

Project description:BackgroundResearch on risk factors for neuropsychiatric adverse events (NAEs) in smoking cessation with pharmacotherapy is scarce. We aimed to identify predictors and develop a prediction model for risk of NAEs in smoking cessation with medications using Bayesian regularization.MethodsBayesian regularization was implemented by applying two shrinkage priors, Horseshoe and Laplace, to generalized linear mixed models on data from 1203 patients treated with nicotine patch, varenicline or placebo. Two predictor models were considered to separate summary scores and item scores in the psychosocial instruments. The summary score model had 19 predictors or 26 dummy variables and the item score model 51 predictors or 58 dummy variables. A total of 18 models were investigated.ResultsAn item score model with Horseshoe prior and 7 degrees of freedom was selected as the final model upon model comparison and assessment. At baseline, smokers reporting more abnormal dreams or nightmares had 16% greater odds of experiencing NAEs during treatment (regularized odds ratio (rOR) = 1.16, 95% credible interval (CrI) = 0.95 - 1.56, posterior probability P(rOR > 1) = 0.90) while those with more severe sleep problems had 9% greater odds (rOR = 1.09, 95% CrI = 0.95 - 1.37, P(rOR > 1) = 0.85). The prouder a person felt one week before baseline resulted in 13% smaller odds of having NAEs (rOR = 0.87, 95% CrI = 0.71 - 1.02, P(rOR < 1) = 0.94). Odds of NAEs were comparable across treatment groups. The final model did not perform well in the test set.ConclusionsWorse sleep-related symptoms reported at baseline resulted in 85%-90% probability of being more likely to experience NAEs during smoking cessation with pharmacotherapy. Treatment for sleep disturbance should be incorporated in smoking cessation program for smokers with sleep disturbance at baseline. Bayesian regularization with Horseshoe prior permits including more predictors in a regression model when there is a low number of events per variable.

Project description:ImportanceIn cancer clinical trials, symptomatic adverse events (AEs), such as nausea, are reported by investigators rather than by patients. There is increasing interest to collect symptomatic AE data via patient-reported outcome (PRO) questionnaires, but it is unclear whether it is feasible to implement this approach in multicenter trials.ObjectiveTo examine whether patients are willing and able to report their symptomatic AEs in multicenter trials.Design, setting, and participantsA total of 361 consecutive patients enrolled in any 1 of 9 US multicenter cancer treatment trials were invited to self-report 13 common symptomatic AEs using a PRO adaptation of the National Cancer Institute's Common Terminology Criteria for Adverse Events (CTCAE) via tablet computers at 5 successive clinic visits. Patient adherence was tracked with reasons for missed self-reports. Agreement with clinician AE reports was analyzed with weighted κ statistics. Patient and investigator perspectives were elicited by survey. The study was conducted from March 15, 2007, to August 11, 2011. Data analysis was performed from August 9, 2013, to March 21, 2014.ResultsOf the 361 patients invited to participate, 285 individuals enrolled, with a median age of 57 years (range, 24-88), 202 (74.3%) female, 241 (85.5%) white, 73 (26.8%) with a high school education or less, and 176 (64.7%) who reported regular internet use (denominators varied owing to missing data). Across all patients and trials, there were 1280 visits during which patients had an opportunity to self-report (ie, patients were alive and enrolled in a treatment trial at the time of the visit). Self-reports were completed at 1202 visits (93.9% overall adherence). Adherence was highest at baseline and declined over time (visit 1, 100%; visit 2, 96%; visit 3, 95%; visit 4, 91%; and visit 5, 85%). Reasons for missing PROs included institutional errors in 27 of 48 (56.3%) of the cases (eg, staff forgetting to bring computers to patients at visits), patients feeling "too ill" in 8 (16.7%), patient refusal in 8 (16.7%), and internet connectivity problems in 5 (10.4%). Patient-investigator CTCAE agreement was moderate or worse for most symptoms (most κ < 0.05), with investigators reporting fewer AEs than patients across symptoms. Most patients believed that the system was easy to use (234 [93.2%]) and useful (230 [93.1%]), and investigators thought that the patient-reported AEs were useful (133 [94.3%]) and accurate (119 [83.2%]).Conclusions and relevanceParticipants in multicenter cancer trials are willing and able to report their own symptomatic AEs at most clinic visits and report more AEs than investigators. This approach may improve the precision of AE reporting in cancer trials.

Project description:Background/aimsRegulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.MethodsIn this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.ResultsOur simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.ConclusionIn the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.

Dataset Information

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Publications

Application of a Bayesian graded response model to characterize areas of disagreement between clinician and patient grading of symptomatic adverse events.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets