Dataset Information

The promise of computer adaptive testing in collection of orthopaedic outcomes: an evaluation of PROMIS utilization.

ABSTRACT:

Background

A crucial component to improving patient care is better clinician understanding of patients' health-related quality of life (HRQoL). In orthopaedic surgery, HRQoL assessment instruments such as the NIH developed Patient Reported Outcomes Measurement Information System (PROMIS), provide surgeons with a framework to assess how a treatment or medical condition is affecting each patient's HRQoL. PROMIS has been demonstrated as a valuable instrument in many diseases; however, the extent to which orthopaedic surgery subspecialties have used and validated PROMIS measures in peer-reviewed research is unclear.

Methods

Systematic scoping methodology was used to investigate the characteristics of studies using PROMIS to assess HRQoL measures as orthopaedic surgical outcomes as well as studies validating computerized adaptive test (CAT) PROMIS physical health (PH) domains including: Physical Function (PF), Upper Extremity (UE), Lower Extremity (LE).

Results

A systematic search of PubMed identified 391 publications utilizing PROMIS in orthopaedics; 153 (39%) were PROMIS PH CAT validation publications. One-hundred publications were in Hand and Upper Extremity, 69 in Spine, 44 in Adult Reconstruction, 43 in Foot and Ankle, 43 in Sports, 37 in Trauma, 31 in General orthopaedics, and 24 in Tumor. From 2011 through 2020 there was an upward trend in orthopaedic PROMIS publications each year (range, 1-153) and an increase in studies investigating or utilizing PROMIS PH CAT domains (range, 1-105). Eighty-five percent (n = 130) of orthopaedic surgery PROMIS PH CAT validation publications (n = 153) analyzed PF; 30% (n = 46) analyzed UE; 3% (n = 4) analyzed LE.

Conclusions

PROMIS utilization within orthopaedics as a whole has significantly increased within the past decade, particularly within PROMIS CAT domains. The existing literature reviewed in this scoping study demonstrates that PROMIS PH CAT domains (PF, UE, and LE) are reliable, responsive, and interpretable in most contexts of patient care throughout all orthopaedic surgery subspecialties. The expanded use of PROMIS CATs in orthopaedic surgery highlights the potential for improved quality of patient care. While challenges of integrating PROMIS into electronic medical records exist, expanded use of PROMIS CAT measurement instruments throughout orthopaedic surgery should be performed. Plain english summary In orthopaedic surgery, health-related quality of life tools such as the NIH developed Patient Reported Outcomes Measurement Information System (PROMIS), offer patients an opportunity to better understand their medical condition and be involved in their own care. Additionally, PROMIS provides surgeons with a framework to assess how a treatment or medical condition is affecting each patient's functional status and quality of life. The efficacy of PROMIS has been demonstrated in many diseases; however, its application throughout orthopaedic care has yet to be depicted. This study sought to identify the extent to which all orthopaedic surgery subspecialties have used and validated PROMIS measures in peer-reviewed research in order to identify its potential as an applicable and valuable tool across specialties. We determined that PROMIS utilization has significantly increased within the past decade. The existing literature reviewed in this scoping study demonstrates that the PROMIS computerized adaptive test domains evaluating physical function status are reliable, responsive, and interpretable in most contexts of patient care throughout all orthopaedic surgery subspecialties. Based on these results, this study recommends the expanded and more uniform use of PROMIS computerized adaptive test measurement instruments in the clinical care of orthopaedic patients.

SUBMITTER: Wong LH

PROVIDER: S-EPMC8727662 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:To build an item response theory-based computer adaptive test (CAT) for balance from 3 traditional, fixed-form balance measures: Berg Balance Scale (BBS), Performance-Oriented Mobility Assessment (POMA), and dynamic gait index (DGI); and to examine whether the CAT's psychometric performance exceeded that of individual measures.Secondary analysis combining 2 existing datasets.Community based.Community-dwelling older adults (N=187) who were aged ?65 years (mean age, 75.2±6.8y, 69% women).Not applicable.The BBS, POMA, and DGI items were compiled into an initial 38-item bank. The Rasch partial credit model was used for final item bank calibration. CAT simulations were conducted to identify the ideal CAT. CAT score accuracy, reliability, floor and ceiling effects, and validity were examined. Floor and ceiling effects and validity of the CAT and individual measures were compared.A 23-item bank met model expectations. A 10-item CAT was selected, showing a very strong association with full item bank scores (r=.97) and good overall reliability (.78). Reliability was better in low- to midbalance ranges as a result of better item targeting to balance ability when compared with the highest balance ranges. No floor effect was noted. The CAT ceiling effect (11.2%) was significantly lower than the POMA (40.1%) and DGI (40.3%) ceiling effects (P<.001 per comparison). The CAT outperformed individual measures, being the only test to discriminate between fallers and nonfallers (P=.007), and being the strongest predictor of self-reported function.The balance CAT showed excellent accuracy, good overall reliability, and excellent validity compared with individual measures, being the only measure to discriminate between fallers and nonfallers. Prospective examination, particularly in low-functioning older adults and clinical populations with balance deficits, is recommended. Development of an improved CAT based on an expanded item bank containing higher difficulty items is also recommended.

Project description:BackgroundMinimal clinically important difference (MCID) estimates are useful for gauging clinical relevance when interpreting changes or differences in patient-reported outcomes scores. These values are lacking in the setting of elbow trauma. Our primary purpose was to estimate the MCID of the Patient-Reported Outcomes Measurement Information System (PROMIS) physical function (PF) computer adaptive test (CAT), the PROMIS upper extremity (UE) CAT, and the QuickDASH using an anchor-based approach for patients recovering from elbow trauma and related surgeries. Secondarily, we aimed to estimate the MCID using the 1/2 standard deviation method.Materials & methodsAdult patients undergoing treatment for isolated elbow injuries between July 2014 and April 2020 were identified at a single tertiary academic medical center. Outcomes, including the PROMIS PF CAT v1.2/2.0, PROMIS UE CAT v1.2, and QuickDASH, were collected via a tablet computer. For inclusion, baseline (6 months before injury up to 11 days postoperatively or after injury) and follow-up (11 to 150 days postoperative or after injury) PF or UE CAT scores were required, as well as a response to an anchor question querying improvement in physical function. The MCID was calculated using (1) an anchor-based approach using the difference in mean score change between anchor groups reporting "No change" and "Slightly Improved/Improved" and (2) the 1/2 standard deviation method.ResultsOf the 146 included patients, the mean age was 46 ± 18 years and 67 (46%) were women. Most patients (129 of 146 or 88%) were recovering from surgery, and the remaining 12% were recovering from nonoperatively managed fractures and/or dislocations. The mean follow-up was 157 ± 192 days. Scores for each instrument improved significantly between baseline and follow-up. Anchor-based MCID values were calculated as follows: 5.7, 4.6, and 5.3 for the PROMIS PF CAT, PROMIS UE CAT, and QuickDASH, respectively. MCID values estimated using the 1/2 standard deviation method were 4.3, 4.8, and 11.7 for the PROMIS PF CAT, PROMIS UE CAT, and QuickDASH, respectively.ConclusionsIn the setting of elbow trauma, we propose MCID ranges of 4.3 to 5.7 for the PROMIS PF CAT, 4.6 to 4.8 for the PROMIS UE CAT, and 5.3 to 11.7 for the QuickDASH. These values will provide a framework for clinical relevance when interpreting clinical outcomes studies, or powering clinical trials, for populations recovering from trauma.

Project description:BACKGROUND:There is a need for shorter-length assessments that capture patient questionnaire data while attaining high data quality without an undue response burden on patients. Computerized adaptive testing (CAT) and classification and regression tree (CART) methods have the potential to meet these needs and can offer attractive options to shorten questionnaire lengths. OBJECTIVE:The objective of this study was to test whether CAT or CART was best suited to reduce the number of questionnaire items in multiple domains (eg, anxiety, depression, quality of life, and social support) used for a needs assessment procedure (NAP) within the field of cardiac rehabilitation (CR) without the loss of data quality. METHODS:NAP data of 2837 CR patients from a multicenter Cardiac Rehabilitation Decision Support System (CARDSS) Web-based program was used. Patients used a Web-based portal, MyCARDSS, to provide their data. CAT and CART were assessed based on their performances in shortening the NAP procedure and in terms of sensitivity and specificity. RESULTS:With CAT and CART, an overall reduction of 36% and 72% of NAP questionnaire length, respectively, was achieved, with a mean sensitivity and specificity of 0.765 and 0.817 for CAT, 0.777 and 0.877 for classification trees, and 0.743 and 0.40 for regression trees, respectively. CONCLUSIONS:Both CAT and CART can be used to shorten the questionnaires of the NAP used within the field of CR. CART, however, showed the best performance, with a twice as large overall decrease in the number of questionnaire items of the NAP compared to CAT and the highest sensitivity and specificity. To our knowledge, our study is the first to assess the differences in performance between CAT and CART for shortening questionnaire lengths. Future research should consider administering varied assessments of patients over time to monitor their progress in multiple domains. For CR professionals, CART integrated with MyCARDSS would provide a feedback loop that informs the rehabilitation progress of their patients by providing real-time patient measurements.