Rasch Analysis for Instrument Development: Why, When, and How?
Ontology highlight
ABSTRACT:
SUBMITTER: Boone WJ
PROVIDER: S-EPMC5132390 | biostudies-literature | Winter 2016
REPOSITORIES: biostudies-literature
Publications
Rasch Analysis for Instrument Development: Why, When, and How?
CBE life sciences education 20160101 4
This essay describes Rasch analysis psychometric techniques and how such techniques can be used by life sciences education researchers to guide the development and use of surveys and tests. Specifically, Rasch techniques can be used to document and evaluate the measurement functioning of such instruments. Rasch techniques also allow researchers to construct "Wright maps" to explain the meaning of a test score or survey score and develop alternative forms of tests and surveys. Rasch techniques pr ...[more]
Similar Datasets
Project description:BackgroundDiabetic retinopathy (DR) is a leading cause of vision loss in working age individuals worldwide. While screening is effective and cost effective, it remains underutilized, and novel methods are needed to increase detection of DR. This clinical validation study compared diagnostic gradings of retinal fundus photographs provided by volunteers on the Amazon Mechanical Turk (AMT) crowdsourcing marketplace with expert-provided gold-standard grading and explored whether determination of the consensus of crowdsourced classifications could be improved beyond a simple majority vote (MV) using regression methods.ObjectiveThe aim of our study was to determine whether regression methods could be used to improve the consensus grading of data collected by crowdsourcing.MethodsA total of 1200 retinal images of individuals with diabetes mellitus from the Messidor public dataset were posted to AMT. Eligible crowdsourcing workers had at least 500 previously approved tasks with an approval rating of 99% across their prior submitted work. A total of 10 workers were recruited to classify each image as normal or abnormal. If half or more workers judged the image to be abnormal, the MV consensus grade was recorded as abnormal. Rasch analysis was then used to calculate worker ability scores in a random 50% training set, which were then used as weights in a regression model in the remaining 50% test set to determine if a more accurate consensus could be devised. Outcomes of interest were the percent correctly classified images, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) for the consensus grade as compared with the expert grading provided with the dataset.ResultsUsing MV grading, the consensus was correct in 75.5% of images (906/1200), with 75.5% sensitivity, 75.5% specificity, and an AUROC of 0.75 (95% CI 0.73-0.78). A logistic regression model using Rasch-weighted individual scores generated an AUROC of 0.91 (95% CI 0.88-0.93) compared with 0.89 (95% CI 0.86-92) for a model using unweighted scores (chi-square P value<.001). Setting a diagnostic cut-point to optimize sensitivity at 90%, 77.5% (465/600) were graded correctly, with 90.3% sensitivity, 68.5% specificity, and an AUROC of 0.79 (95% CI 0.76-0.83).ConclusionsCrowdsourced interpretations of retinal images provide rapid and accurate results as compared with a gold-standard grading. Creating a logistic regression model using Rasch analysis to weight crowdsourced classifications by worker ability improves accuracy of aggregated grades as compared with simple majority vote.
| S-EPMC5497070 | biostudies-literature
Project description:BackgroundDespite the availability of a wide range of critical thinking instruments, there was no original design for nurses that has been translated into Chinese. However, only instruments designed specifically for the nursing discipline would be reliable. This study aimed to translate, culturally adapt, and validate the Yoon Critical Thinking Disposition Instrument in the Chinese context.MethodsA four-step translation process was implemented according to Word Health Organization guidelines, which included forward translation, expert panel review, backward translation, and pre-testing. Experts and nursing students participated in testing the validity and reliability of the Chinese version.ResultsThe translation of the instrument went smoothly. According to a confirmatory factor analysis, there was an acceptable fit for the seven-factor model. Content validity indices ranged from 0.6 to 1 at item level, and 0.94 at scale level. In addition, there was extremely high internal consistency and test-retest reliability in the translated instrument. There was a good fit for the items with both person and item reliabilities greater than 0.6 and a separation index of 2.19, respectively. The item location was identified from the wright map as not covering person ability, but the scale did not have a gender-related differential item functioning.ConclusionsIn this study, a critical thinking disposition instrument for nursing students was translated into Chinese for the first time. This translated instrument is a reliable tool with satisfactory validity and reliability. It could provide opportunities for building a cross-cultural understanding of critical thinking disposition.
| S-EPMC10559418 | biostudies-literature
Project description:The 2014-2019 Prevention Research Centers (PRC) Program Funding Opportunity Announcement stated that "all applicants will be expected to collaborate with CDC to collect data to be able to perform cost analysis." For the first time in the 30-year history of the PRC Program, a cost indicator was included in the PRC Program Evaluation and a cost analysis (CA) instrument developed. The PRC-CA instrument systematically collects data on the cost of the PRC core research project to eventually answer the CDC PRC Program Evaluation question: "To what extent do investments in PRCs support the scalability, sustainability, and effectiveness of the outcomes resulting from community-engaged efforts to improve public health?" The objective of this article is to briefly describe the development of the PRC-CA instrument. Data obtained from the PRC-CA instrument can be used to generate cost summaries to inform decision making within the PRC Program and each individual PRC.
| S-EPMC6373870 | biostudies-literature
Project description:This study aimed to assess Rasch-based psychometric properties of the Trypophobia Questionnaire measuring proneness to trypophobia, which refers to disgust and unpleasantness induced by the observation of clusters of objects (e.g., lotus seed pods).Rasch analysis was performed on data from 582 healthy Japanese adults. The results suggested that Trypophobia Questionnaire has a unidimensional structure with ordered response categories and sufficient person and item reliabilities, and that it does not have differential item functioning across sexes and age groups, whereas the targeting of the scale leaves room for improvements. When items that did not fit the Rasch model were removed, the shortened version showed slightly improved psychometric properties. However, results were not conclusive in determining whether the full or shortened version is better for practical use. Further assessment and validation are needed.
| S-EPMC5813320 | biostudies-literature
Project description:ObjectiveTo describe the development and validation of a novel patient reported outcome measure for symptom burden from long covid, the symptom burden questionnaire for long covid (SBQ-LC).DesignMultiphase, prospective mixed methods study.SettingRemote data collection and social media channels in the United Kingdom, 14 April to 1 August 2021.Participants13 adults (aged ≥18 years) with self-reported long covid and 10 clinicians evaluated content validity. 274 adults with long covid field tested the draft questionnaire.Main outcome measuresPublished systematic reviews informed development of SBQ-LC's conceptual framework and initial item pool. Thematic analysis of transcripts from cognitive debriefing interviews and online clinician surveys established content validity. Consensus discussions with the patient and public involvement group of the Therapies for Long COVID in non-hospitalised individuals: From symptoms, patient reported outcomes and immunology to targeted therapies (TLC Study) confirmed face validity. Rasch analysis of field test data guided item and scale refinement and provided initial evidence of the SBQ-LC's measurement properties.ResultsSBQ-LC (version 1.0) is a modular instrument measuring patient reported outcomes and is composed of 17 independent scales with promising psychometric properties. Respondents rate their symptom burden during the past seven days using a dichotomous response or 4 point rating scale. Each scale provides coverage of a different symptom domain and returns a summed raw score that can be transformed to a linear (0-100) score. Higher scores represent higher symptom burden. After rating scale refinement and item reduction, all scales satisfied the Rasch model requirements for unidimensionality (principal component analysis of residuals: first residual contrast values <2.00 eigenvalue units) and item fit (outfit mean square values within 0.5 -1.5 logits). Rating scale categories were ordered with acceptable category fit statistics (outfit mean square values <2.0 logits). 14 item pairs had evidence of local dependency (residual correlation values >0.4). Across the 17 scales, person reliability ranged from 0.34 to 0.87, person separation ranged from 0.71 to 2.56, item separation ranged from 1.34 to 13.86, and internal consistency reliability (Cronbach's alpha) ranged from 0.56 to 0.91.ConclusionsSBQ-LC (version 1.0) is a comprehensive patient reported outcome instrument developed using modern psychometric methods. It measures symptoms of long covid important to people with lived experience of the condition and may be used to evaluate the impact of interventions and inform best practice in clinical management.
| S-EPMC9043395 | biostudies-literature
Project description:BackgroundAntepartum activity restriction (AAR) is a common recommendation given to women at risk for preterm delivery. However, such treatment has been shown to cause heavy emotional burdens on the women receiving it since it requires them to face many challenges derived from the intervention. Nevertheless, current existing scales lack effective items that can reflect the distress of these women. The aim of this study was to develop a reliable instrument to assess the distress of women experiencing AAR.MethodThe Prenatal Activity Restriction Stress Questionnaire (PARSQ) was developed according to comprehensive literature review, women's interviews, and existing pregnancy-special stress scales from August 2016 to July 2017 in southern Taiwan. Six experts evaluated its content validity; the Rasch rating scale model (RSM) was used to examine its item-fit, dimensionality, and reliability with 200 women with AAR experience. Furthermore, the concurrent validity was assessed through computing the correlation of AAR women's scores on the PARSQ and Perceived Stress Scale (PSS), and discriminant validity of the PARSQ was assessed to compare the scores' differences between the AAR women and the healthy pregnant women.ResultsThe PARSQ was constructed with 23 items in the 4-dimensional scale: Role function changes (8 items), Fetal safety and health (5 items), Physical and psychological care issues (5 items), and Socioeconomic and medical issues (5 items). It was confirmed to have satisfactory content vitality (CVI = 0.78 to 1.0), reasonable item-fit (0.77 to 1.51), and good reliability in RSM model, as well as adequate concurrent validity (p = 0.005) and discriminant validity (p < 0.001).ConclusionsUnderstanding the distress of women undergoing AAR is necessary for developing appropriate prenatal care to assist women in coping with their situation to alleviate their emotional burdens. The developed PARSQ with satisfied psychometric properties can be an informative instrument for clinicians/researchers to assess the specific stress of pregnant women with AAR.
| S-EPMC7603674 | biostudies-literature
Project description:BackgroundAccurate measurement of health literacy is essential to improve accessibility and effectiveness of health care and prevention. One measure frequently applied in international research is the Short Assessment of Health Literacy (SAHL). While the Dutch SAHL (SAHL-D) has proven to be valid and reliable, its administration is time consuming and burdensome for participants. Our aim was to further validate, strengthen and shorten the SAHL-D using Rasch analysis.MethodsAvailable cross-sectional SAHL-D data was used from adult samples (N = 1231) to assess unidimensionality, local independence, item fit, person fit, item hierarchy, scale targeting, precision (person reliability and person separation), and presence of differential item functioning (DIF) depending on age, gender, education and study sample.ResultsThirteen items for a short form were selected based on item fit and DIF, and scale properties were compared between the two forms. The long form had several items with DIF for age, gender, educational level and study sample. Both forms showed lower measurement precision at higher health literacy levels.ConclusionsThe findings support the validity and reliability of the SAHL-D for the long form and the short form, which can be used for a rapid assessment of health literacy in research and clinical practice.
| S-EPMC6567391 | biostudies-literature
Project description:Acquiring gait parameters from usual walking is important to predict clinical outcomes including life expectancy, risk of fall, and neurocognitive performance in older people. We developed a novel gait analysis tool that is small, less-intrusive and is based on two-dimensional light detection and ranging (2D-LiDAR) technology. Using an object-tracking algorithm, we conducted a validation study of the spatiotemporal tracking of ankle locations of young, healthy participants (n = 4) by comparing our tool and a stereo camera with the motion capture system as a gold standard modality. We also assessed parameters including step length, step width, cadence, and gait speed. The 2D-LiDAR system showed a much better accuracy than that of a stereo camera system, where mean absolute errors were 46.2 ± 17.8 mm and 116.3 ± 69.6 mm, respectively. Gait parameters from the 2D-LiDAR system were in good agreement with those from the motion capture system (r = 0.955 for step length, r = 0.911 for cadence). Simultaneous tracking of multiple targets by the 2D-LiDAR system was also demonstrated. The novel system might be useful in space and resource constrained clinical practice for older adults.
| S-EPMC7826665 | biostudies-literature
Project description:BACKGROUND:People with multiple sclerosis (PwMS) frequently have impaired balance from an early stage of the disease. Balance difficulties can be divided into categories; although, to date, these lack scientific foundation. Impaired balance in PwMS can be addressed using specific and challenging exercises. Such exercises should provide an optimal challenge point; however, the difficulty of balance exercises is often unknown, making it difficult to target the exercises to an individual's abilities. The aims of this study were: to develop an exercise programme for PwMS relating the exercises to the balance problem categories; to establish the order of difficulty of exercises in each category and; to evaluate the content and structural validity of the exercise programme. METHODS:A "construct map" approach was used to design and develop an exercise programme for PwMS. Potentially relevant balance exercises were identified, then a framework was set up, comprising four dimensions (subsequently reduced to three dimensions) of balance exercises. The relevance, comprehensibility, and comprehensiveness of the exercise programme were rated by 13 physiotherapists, who also linked 19 key exercises to balance categories. A total of 65 PwMS performed the 19 balance exercises, rated their difficulty and commented on the relevance and comprehensibility of each exercise. A Rasch model was used to evaluate the relative difficulty of the exercises. To assess fit of the data to the Rasch model a rating scale model was used, which is a unidimensional latent trait model for polytomous item responses. RESULTS:Evaluation by the physiotherapists and PwMS indicated that the content validity of the exercise programme was adequate. Rasch analysis showed that the latent trait "balance exercises in PwMS" comprised three subdimensions ("stable BOS", "sway" and "step and walk"). The 19 balance exercises showed adequate fit to the respective dimensions. The difficulties of the balance exercises were adequate to cover the ability spectrum of the PwMS. CONCLUSION:A balance exercise programme for PwMS comprising three dimensions of balance exercises was developed. Difficulty estimates have been established for each of the exercises, which can be used for targeted balance training. Content and structural validity of the programme was adequate.
| S-EPMC8672542 | biostudies-literature
Project description:BackgroundThe aim of the current study was to evaluate the structural validity of the 20-item long-term conditions questionnaire (LTCQ) and to explore a potential short-form version of the scale using Rasch analysis.MethodsData were collected through postal surveys (February 2016-January 2017) from a sample of 1,211 participants diagnosed with at least one long-term condition (LTC). Identified participants were invited through either local authorities for a social care cohort (n?=?294) or primary care practices for a health care cohort (n?=?917). Participants were mailed a survey, including the LTCQ, demographic questions, a comorbidities measure, and other validated outcome measures. Respondents were invited to complete a follow-up survey including the LTCQ for assessment of reproducibility.ResultsThe main assumptions of the Rasch model from the LTCQ were fulfilled, although infit and outfit indices indicated some items showed misfit. Misfitted items, items that did not have a preceding set or showed some local dependence were removed one at a time, with the remaining candidate items to form an 8-item short version, the LTCQ-8. The Rasch model for the LTCQ-8 explained 64% variance and had a reliability estimate greater than 0.80. Several items in the LTCQ showed uniform differential item function (DIF) in relation to the number of reported LTCs, age, cohort and type of LTCs, but fewer items exhibited DIF in the LTCQ-8. Spearman's rho correlations between the LTCQ and the LTCQ-8 were strong across the total sample and various subgroups. Correlations between the LTCQ-8 and all reference measures were moderate to strong, and comparable to correlations found between the LTCQ and these measures.ConclusionsThe LTCQ measures a unidimensional construct, and it is therefore acceptable to use a summed total score. The LTCQ-8 also met the assumption of unidimensionality and had comparable construct validity with the LTCQ. Additional validation is required in an independent sample.
| S-EPMC7706038 | biostudies-literature