Dataset Information

Software for Administering the National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: Usability Study.

ABSTRACT: BACKGROUND:The US National Cancer Institute (NCI) developed software to gather symptomatic adverse events directly from patients participating in clinical trials. The software administers surveys to patients using items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) through Web-based or automated telephone interfaces and facilitates the management of survey administration and the resultant data by professionals (clinicians and research associates). OBJECTIVE:The purpose of this study was to iteratively evaluate and improve the usability of the PRO-CTCAE software. METHODS:Heuristic evaluation of the software functionality was followed by semiscripted, think-aloud protocols in two consecutive rounds of usability testing among patients with cancer, clinicians, and research associates at 3 cancer centers. We conducted testing with patients both in clinics and at home (remotely) for both Web-based and telephone interfaces. Furthermore, we refined the software between rounds and retested. RESULTS:Heuristic evaluation identified deviations from the best practices across 10 standardized categories, which informed initial software improvement. Subsequently, we conducted user-based testing among 169 patients and 47 professionals. Software modifications between rounds addressed identified issues, including difficulty using radio buttons, absence of survey progress indicators, and login problems (for patients) as well as scheduling of patient surveys (for professionals). The initial System Usability Scale (SUS) score for the patient Web-based interface was 86 and 82 (P=.22) before and after modifications, respectively, whereas the task completion score was 4.47, which improved to 4.58 (P=.39) after modifications. Following modifications for professional users, the SUS scores improved from 71 to 75 (P=.47), and the mean task performance improved significantly (4.40 vs 4.02; P=.001). CONCLUSIONS:Software modifications, informed by rigorous assessment, rendered a usable system, which is currently used in multiple NCI-sponsored multicenter cancer clinical trials. TRIAL REGISTRATION:ClinicalTrials.gov NCT01031641; https://clinicaltrials.gov/ct2/show/NCT01031641 (Archived by WebCite at http://www.webcitation.org/708hTjlTl).

SUBMITTER: Schoen MW

PROVIDER: S-EPMC6066634 | biostudies-literature | 2018 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Software for Administering the National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: Usability Study.

Schoen Martin W MW Basch Ethan E Hudson Lori L LL Chung Arlene E AE Mendoza Tito R TR Mitchell Sandra A SA St Germain Diane D Baumgartner Paul P Sit Laura L Rogak Lauren J LJ Shouery Marwan M Shalley Eve E Reeve Bryce B BB Fawzy Maria R MR Bhavsar Nrupen A NA Cleeland Charles C Schrag Deborah D Dueck Amylou C AC Abernethy Amy P AP

JMIR human factors 20180716 3

<h4>Background</h4>The US National Cancer Institute (NCI) developed software to gather symptomatic adverse events directly from patients participating in clinical trials. The software administers surveys to patients using items from the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE) through Web-based or automated telephone interfaces and facilitates the management of survey administration and the resultant data by professionals (clinicians and ...[more]

PMID: 30012546

Similar Datasets

Project description:BackgroundThe Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events is an item library designed for eliciting patient-reported adverse events in oncology. For each adverse event, up to three individual items are scored for frequency, severity, and interference with daily activities. To align the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events with other standardized tools for adverse event assessment including the Common Terminology Criteria for Adverse Events, an algorithm for mapping individual items for any given adverse event to a single composite numerical grade was developed and tested.MethodsA five-step process was used: (1) All 179 possible Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events score combinations were presented to 20 clinical investigators to subjectively map combinations to single numerical grades ranging from 0 to 3. (2) Combinations with <75% agreement were presented to investigator committees at a National Clinical Trials Network cooperative group meeting to gain majority consensus via anonymous voting. (3) The resulting algorithm was refined via graphical and tabular approaches to assure directional consistency. (4) Validity, reliability, and sensitivity were assessed in a national study dataset. (5) Accuracy for delineating adverse events between study arms was measured in two Phase III clinical trials (NCT02066181 and NCT01522443).ResultsIn Step 1, 12/179 score combinations had <75% initial agreement. In Step 2, majority consensus was reached for all combinations. In Step 3, five grades were adjusted to assure directional consistency. In Steps 4 and 5, composite grades performed well and comparably to individual item scores on validity, reliability, sensitivity, and between-arm delineation.ConclusionA composite grading algorithm has been developed and yields single numerical grades for adverse events assessed via the Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events, and can be useful in analyses and reporting.

Project description:The National Cancer Institute (NCI) is developing a patient-reported version of its Common Terminology Criteria for Adverse Events, called the "PRO-CTCAE." The PRO-CTCAE consists of a library of patient-reported items which can be administered in clinical trials to directly capture the patient experience of adverse events during cancer treatment, as well as a software platform for administering these items via computer or telephone. In order to better understand the impressions of stakeholders involved in cancer clinical research about the potential value of the PRO-CTCAE approach to capturing adverse event information in clinical research, as well as their perspectives about barriers and strategies for implementing the PRO-CTCAE in NCI-sponsored cancer trials, a survey was conducted. A survey including structured and open-ended questions was developed to elicit perceptions about the use of patient-reported outcomes (PROs) for adverse event reporting, and to explore logistical considerations for implementing the PRO-CTCAE in cancer trials. The survey was distributed electronically and by paper to a convenience sample of leadership and committee members in the NCI's cooperative group network, including principal investigators, clinical investigators, research nurses, data managers, patient advocates, and representatives of the NCI and Food and Drug Administration. Between October, 2008 through February, 2009, 727 surveys were collected. Most respondents (93%) agreed that patient reporting of adverse symptoms would be useful for improving understanding of the patient experience with treatment in cancer trials, and 88%, 80%, and 76%, respectively, endorsed that administration of PRO-CTCAE items in clinical trials would improve the completeness, accuracy, and efficiency of symptom data collection. More than three fourths believed that patient reports would be useful for informing treatment dose modifications and towards FDA regulatory evaluation of drugs. Eighty-eight percent felt that patients in clinical trials would be willing to self-report adverse symptoms at clinic visits via computer, and 68% felt patients would self-report weekly from home via the internet or an automated telephone system. Lack of computers and limited space and personnel were seen as potential barriers to in-clinic self-reporting, but these were judged to be surmountable with adequate funding. The PRO-CTCAE items and software are viewed by a majority of survey respondents as a means to improve adverse event data quality and comprehensiveness, enhance clinical decision-making, and foster patient-clinician communication. Research is ongoing to assess the measurement properties and feasibility of implementing this measure in cancer clinical trials.

Project description:ImportanceTo integrate the patient perspective into adverse event reporting, the National Cancer Institute developed a patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).ObjectiveTo assess the construct validity, test-retest reliability, and responsiveness of PRO-CTCAE items.Design, setting, and participantsA total of 975 adults with cancer undergoing outpatient chemotherapy and/or radiation therapy enrolled in this questionnaire-based study between January 2011 and February 2012. Eligible participants could read English and had no clinically significant cognitive impairment. They completed PRO-CTCAE items on tablet computers in clinic waiting rooms at 9 US cancer centers and community oncology practices at 2 visits 1 to 6 weeks apart. A subset completed PRO-CTCAE items during an additional visit 1 business day after the first visit.Main outcomes and measuresPrimary comparators were clinician-reported Eastern Cooperative Oncology Group Performance Status (ECOG PS) and the European Organisation for Research and Treatment of Cancer Core Quality of Life Questionnaire (QLQ-C30).ResultsA total of 940 of 975 (96.4%) and 852 of 940 (90.6%) participants completed PRO-CTCAE items at visits 1 and 2, respectively. At least 1 symptom was reported by 938 of 940 (99.8%) participants. Participants' median age was 59 years; 57.3% were female, 32.4% had a high school education or less, and 17.1% had an ECOG PS of 2 to 4. All PRO-CTCAE items had at least 1 correlation in the expected direction with a QLQ-C30 scale (111 of 124, P<.05 for all). Stronger correlations were seen between PRO-CTCAE items and conceptually related QLQ-C30 domains. Scores for 94 of 124 PRO-CTCAE items were higher in the ECOG PS 2 to 4 vs 0 to 1 group (58 of 124, P<.05 for all). Overall, 119 of 124 items met at least 1 construct validity criterion. Test-retest reliability was 0.7 or greater for 36 of 49 prespecified items (median [range] intraclass correlation coefficient, 0.76 [0.53-.96]). Correlations between PRO-CTCAE item changes and corresponding QLQ-C30 scale changes were statistically significant for 27 prespecified items (median [range] r=0.43 [0.10-.56]; all P≤.006).Conclusions and relevanceEvidence demonstrates favorable validity, reliability, and responsiveness of PRO-CTCAE in a large, heterogeneous US sample of patients undergoing cancer treatment. Studies evaluating other measurement properties of PRO-CTCAE are under way to inform further development of PRO-CTCAE and its inclusion in cancer trials.

Project description:OBJECTIVE:The study sought to describe patient-entered supplemental information on symptomatic adverse events (AEs) in cancer clinical research reported via a National Cancer Institute software system and examine the feasibility of mapping these entries to established terminologies. MATERIALS AND METHODS:Patients in 3 multicenter trials electronically completed surveys during cancer treatment. Each survey included a prespecified subset of items from the National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE). Upon completion of the survey items, patients could add supplemental symptomatic AE information in a free text box. As patients typed into the box, structured dropdown terms could be selected from the PRO-CTCAE item library or Medical Dictionary for Regulatory Activities (MedDRA), or patients could type unstructured free text for submission. RESULTS:Data were pooled from 1760 participants (48% women; 78% White) who completed 8892 surveys, of which 2387 (26.8%) included supplemental symptomatic AE information. Overall, 1024 (58%) patients entered supplemental information at least once, with an average of 2.3 per patient per study. This encompassed 1474 of 8892 (16.6%) dropdowns and 913 of 8892 (10.3%) unstructured free text entries. One-third of the unstructured free text entries (32%) could be mapped post hoc to a PRO-CTCAE term and 68% to a MedDRA term. DISCUSSION:Participants frequently added supplemental information beyond study-specific survey items. Almost half selected a structured dropdown term, although many opted to submit unstructured free text entries. Most free text entries could be mapped post hoc to PRO-CTCAE or MedDRA terms, suggesting opportunities to enhance the system to perform real-time mapping for AE reporting. CONCLUSIONS:Patient reporting of symptomatic AEs using a text box functionality with mapping to existing terminologies is both feasible and informative.

Project description:BackgroundThe U.S. National Cancer Institute's Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™) is a library of items for assessing symptomatic adverse events by patient self-report in oncology trials. The aim of this multi-site study was to generate and linguistically validate a Dutch language version of the U.S. PRO-CTCAE for use in the Netherlands and Dutch-speaking Belgium.MethodsAll 124 items in the PRO-CTCAE item library were translated into Dutch using established translation procedures, including dual forward translations, reconciliation, back-translation, reconciliation of the source with the back-translation, and expert reviews. Harmonization of the translation for use in both the Netherlands and Belgium was achieved via an iterative review process in which the translations were discussed and reconciled by consensus of PRO experts, clinicians and bilingual Dutch translators. The translated PRO-CTCAE™ items were completed by a geographically-diverse sample of Dutch speaking patients from the Netherlands (n = 40) and Belgium (n = 60), and who were currently receiving or who had recently completed cancer-directed therapy. Patients were diverse with respect to age, sex, educational attainment, and cancer diagnosis. Cognitive debriefing, using a semi-structured interview guide, probed for comprehension and clarity of PRO-CTCAE symptom terms, attributes (e.g. frequency, severity, interference), response choices, and understanding of 'at its worst' and 'in the last 7 days'. Items for which the patient data indicated possible difficulties were considered for revision.ResultsThree items underwent minor phrasing revision and retesting was not deemed necessary. The symptom term for stretch marks was poorly understood by 12.5% of participants, and this item was revised to include parenthetical phrasing. It was retested with 10 participants from Belgium (n = 5) and the Netherlands (n = 5) and demonstrated acceptable comprehension.ConclusionsThe Dutch language version of PRO-CTCAE has been successfully developed and linguistically validated for use in oncology studies in the Netherlands and Dutch-speaking Belgium. Extending the availability of NCI PRO-CTCAE in languages beyond English increases international consistency in the capture of Patient-Reported outcomes in patients participating in cancer clinical trials.

Project description:BackgroundPatient-reported outcome (PRO) measurements linked to Common Terminology Criteria for Adverse Events (CTCAE) grading may improve symptom adverse event (AE) reporting in pediatric oncology trials. We evaluated construct validity, responsiveness, and test-retest reliability of the Ped-PRO-CTCAE measurement system for children and adolescents undergoing cancer care.MethodsA total of 482 children and adolescents (7-18 years, 41.5% not non-Hispanic white) newly diagnosed with cancer and their caregivers participated from nine pediatric oncology hospitals. Surveys were completed at 72 hours preceding treatment initiation (T1) and at follow-up (T2) approximately 7-17 days later for chemotherapy, and 4+ weeks for radiation. Psychometric analyses examined the relationship of Ped-PRO-CTCAE items (assessing 62 symptom AEs) with Patient-Reported Outcomes Measurement Information System, Memorial Symptom Assessment Scale, Lansky Play-Performance Scale, and medication use. A separate test-retest study included 46 children.ResultsPed-PRO-CTCAE and Memorial Symptom Assessment Scale were strongly correlated across age groups at T2: 7-12 years (r = 0.62-0.80), 13-15 years (r = 0.44-0.94), and 16-18 years (r = 0.65-0.98); and over time. The Ped-PRO-CTCAE was strongly correlated with Patient-Reported Outcomes Measurement Information System Pediatric measures at T2; for example, pain interference (r = 0.70, 95% confidence interval [CI] = 0.64 to 0.76), fatigue severity (r = 0.63, 95% CI = 0.56 to 0.69), and depression severity (r = 0.76, 95% CI = 0.71 to 0.81). Ped-PRO-CTCAE items differentiated children by Lansky Play-Performance Scale and by medication use. Test-retest agreement ranged from 54.3% to 93.5%.ConclusionsThis longitudinal study provided evidence for the construct validity and reliability of the core Ped-PRO-CTCAE symptom AE items relative to several established measures. Additional responsiveness data with clinical anchors are recommended. Incorporation of Ped-PRO-CTCAE in trials may lead to a better understanding of the cancer treatment experience.

Project description:BackgroundIntegrating the patient's perspective has become an increasingly important component of adverse event reporting. The National Cancer Institute has developed a Patient-Reported Outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE™). This instrument has been translated into German and linguistically validated; however, its quantitative measurement properties have not been evaluated.Patients and methodsA German language survey that included 31 PRO-CTCAE items, as well as the EORTC QLQ-C30 and the Oral Mucositis Daily Questionnaire (OMDQ), was distributed at 10 cancer treatment settings in Germany and Austria. Item quality was assessed by analysis of acceptability and comprehensibility. Reliability was evaluated by using Cronbach's' alpha and validity by principal components analysis (PCA), multitrait-multimethod matrix (MTMM) and known groups validity techniques.ResultsOf 660 surveys distributed to the study centres, 271 were returned (return rate 41%), and data from 262 were available for analysis. Participants' median age was 59.7 years, and 69.5% of the patients were female. Analysis of item quality supported the comprehensibility of the 31 PRO-CTCAE items. Reliability was very good; Cronbach's' alpha correlation coefficients were >0.9 for almost all item clusters. Construct validity of the PRO-CTCAE core item set was shown by identifying 10 conceptually meaningful item clusters via PCA. Moreover, construct validity was confirmed by the MTMM: monotrait-heteromethod comparison showed 100% high correlation, whereas heterotrait-monomethod comparison indicated 0% high correlation. Known groups validity was supported; PRO-CTCAE scores were significantly lower for those with impaired versus preserved health-related quality of life.ConclusionA set of 31 items drawn from the German PRO-CTCAE item library demonstrated favourable measurement properties. These findings add to the body of evidence that PRO-CTCAE provides a rigorous method to capture patient self-reports of symptomatic toxicity for use in cancer clinical trials.

Dataset Information

Software for Administering the National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: Usability Study.

Publications

Software for Administering the National Cancer Institute's Patient-Reported Outcomes Version of the Common Terminology Criteria for Adverse Events: Usability Study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets