Rasch Analysis for Instrument Development: Why, When, and How?
Ontology highlight
ABSTRACT:
SUBMITTER: Boone WJ
PROVIDER: S-EPMC5132390 | biostudies-literature | Winter 2016
REPOSITORIES: biostudies-literature
Publications
Rasch Analysis for Instrument Development: Why, When, and How?
CBE life sciences education 20160101 4
This essay describes Rasch analysis psychometric techniques and how such techniques can be used by life sciences education researchers to guide the development and use of surveys and tests. Specifically, Rasch techniques can be used to document and evaluate the measurement functioning of such instruments. Rasch techniques also allow researchers to construct "Wright maps" to explain the meaning of a test score or survey score and develop alternative forms of tests and surveys. Rasch techniques pr ...[more]
Similar Datasets
Project description:BackgroundDiabetic retinopathy (DR) is a leading cause of vision loss in working age individuals worldwide. While screening is effective and cost effective, it remains underutilized, and novel methods are needed to increase detection of DR. This clinical validation study compared diagnostic gradings of retinal fundus photographs provided by volunteers on the Amazon Mechanical Turk (AMT) crowdsourcing marketplace with expert-provided gold-standard grading and explored whether determination of the consensus of crowdsourced classifications could be improved beyond a simple majority vote (MV) using regression methods.ObjectiveThe aim of our study was to determine whether regression methods could be used to improve the consensus grading of data collected by crowdsourcing.MethodsA total of 1200 retinal images of individuals with diabetes mellitus from the Messidor public dataset were posted to AMT. Eligible crowdsourcing workers had at least 500 previously approved tasks with an approval rating of 99% across their prior submitted work. A total of 10 workers were recruited to classify each image as normal or abnormal. If half or more workers judged the image to be abnormal, the MV consensus grade was recorded as abnormal. Rasch analysis was then used to calculate worker ability scores in a random 50% training set, which were then used as weights in a regression model in the remaining 50% test set to determine if a more accurate consensus could be devised. Outcomes of interest were the percent correctly classified images, sensitivity, specificity, and area under the receiver operating characteristic (AUROC) for the consensus grade as compared with the expert grading provided with the dataset.ResultsUsing MV grading, the consensus was correct in 75.5% of images (906/1200), with 75.5% sensitivity, 75.5% specificity, and an AUROC of 0.75 (95% CI 0.73-0.78). A logistic regression model using Rasch-weighted individual scores generated an AUROC of 0.91 (95% CI 0.88-0.93) compared with 0.89 (95% CI 0.86-92) for a model using unweighted scores (chi-square P value<.001). Setting a diagnostic cut-point to optimize sensitivity at 90%, 77.5% (465/600) were graded correctly, with 90.3% sensitivity, 68.5% specificity, and an AUROC of 0.79 (95% CI 0.76-0.83).ConclusionsCrowdsourced interpretations of retinal images provide rapid and accurate results as compared with a gold-standard grading. Creating a logistic regression model using Rasch analysis to weight crowdsourced classifications by worker ability improves accuracy of aggregated grades as compared with simple majority vote.
| S-EPMC5497070 | biostudies-literature
Project description:The 2014-2019 Prevention Research Centers (PRC) Program Funding Opportunity Announcement stated that "all applicants will be expected to collaborate with CDC to collect data to be able to perform cost analysis." For the first time in the 30-year history of the PRC Program, a cost indicator was included in the PRC Program Evaluation and a cost analysis (CA) instrument developed. The PRC-CA instrument systematically collects data on the cost of the PRC core research project to eventually answer the CDC PRC Program Evaluation question: "To what extent do investments in PRCs support the scalability, sustainability, and effectiveness of the outcomes resulting from community-engaged efforts to improve public health?" The objective of this article is to briefly describe the development of the PRC-CA instrument. Data obtained from the PRC-CA instrument can be used to generate cost summaries to inform decision making within the PRC Program and each individual PRC.
| S-EPMC6373870 | biostudies-literature
Project description:This study aimed to assess Rasch-based psychometric properties of the Trypophobia Questionnaire measuring proneness to trypophobia, which refers to disgust and unpleasantness induced by the observation of clusters of objects (e.g., lotus seed pods).Rasch analysis was performed on data from 582 healthy Japanese adults. The results suggested that Trypophobia Questionnaire has a unidimensional structure with ordered response categories and sufficient person and item reliabilities, and that it does not have differential item functioning across sexes and age groups, whereas the targeting of the scale leaves room for improvements. When items that did not fit the Rasch model were removed, the shortened version showed slightly improved psychometric properties. However, results were not conclusive in determining whether the full or shortened version is better for practical use. Further assessment and validation are needed.
| S-EPMC5813320 | biostudies-literature
Project description:BACKGROUND:Accurate measurement of health literacy is essential to improve accessibility and effectiveness of health care and prevention. One measure frequently applied in international research is the Short Assessment of Health Literacy (SAHL). While the Dutch SAHL (SAHL-D) has proven to be valid and reliable, its administration is time consuming and burdensome for participants. Our aim was to further validate, strengthen and shorten the SAHL-D using Rasch analysis. METHODS:Available cross-sectional SAHL-D data was used from adult samples (N = 1231) to assess unidimensionality, local independence, item fit, person fit, item hierarchy, scale targeting, precision (person reliability and person separation), and presence of differential item functioning (DIF) depending on age, gender, education and study sample. RESULTS:Thirteen items for a short form were selected based on item fit and DIF, and scale properties were compared between the two forms. The long form had several items with DIF for age, gender, educational level and study sample. Both forms showed lower measurement precision at higher health literacy levels. CONCLUSIONS:The findings support the validity and reliability of the SAHL-D for the long form and the short form, which can be used for a rapid assessment of health literacy in research and clinical practice.
| S-EPMC6567391 | biostudies-other
Project description:BACKGROUND:Antepartum activity restriction (AAR) is a common recommendation given to women at risk for preterm delivery. However, such treatment has been shown to cause heavy emotional burdens on the women receiving it since it requires them to face many challenges derived from the intervention. Nevertheless, current existing scales lack effective items that can reflect the distress of these women. The aim of this study was to develop a reliable instrument to assess the distress of women experiencing AAR. METHOD:The Prenatal Activity Restriction Stress Questionnaire (PARSQ) was developed according to comprehensive literature review, women's interviews, and existing pregnancy-special stress scales from August 2016 to July 2017 in southern Taiwan. Six experts evaluated its content validity; the Rasch rating scale model (RSM) was used to examine its item-fit, dimensionality, and reliability with 200 women with AAR experience. Furthermore, the concurrent validity was assessed through computing the correlation of AAR women's scores on the PARSQ and Perceived Stress Scale (PSS), and discriminant validity of the PARSQ was assessed to compare the scores' differences between the AAR women and the healthy pregnant women. RESULTS:The PARSQ was constructed with 23 items in the 4-dimensional scale: Role function changes (8 items), Fetal safety and health (5 items), Physical and psychological care issues (5 items), and Socioeconomic and medical issues (5 items). It was confirmed to have satisfactory content vitality (CVI?=?0.78 to 1.0), reasonable item-fit (0.77 to 1.51), and good reliability in RSM model, as well as adequate concurrent validity (p?=?0.005) and discriminant validity (p?<?0.001). CONCLUSIONS:Understanding the distress of women undergoing AAR is necessary for developing appropriate prenatal care to assist women in coping with their situation to alleviate their emotional burdens. The developed PARSQ with satisfied psychometric properties can be an informative instrument for clinicians/researchers to assess the specific stress of pregnant women with AAR.
| S-EPMC7603674 | biostudies-literature
Project description:Acquiring gait parameters from usual walking is important to predict clinical outcomes including life expectancy, risk of fall, and neurocognitive performance in older people. We developed a novel gait analysis tool that is small, less-intrusive and is based on two-dimensional light detection and ranging (2D-LiDAR) technology. Using an object-tracking algorithm, we conducted a validation study of the spatiotemporal tracking of ankle locations of young, healthy participants (n = 4) by comparing our tool and a stereo camera with the motion capture system as a gold standard modality. We also assessed parameters including step length, step width, cadence, and gait speed. The 2D-LiDAR system showed a much better accuracy than that of a stereo camera system, where mean absolute errors were 46.2 ± 17.8 mm and 116.3 ± 69.6 mm, respectively. Gait parameters from the 2D-LiDAR system were in good agreement with those from the motion capture system (r = 0.955 for step length, r = 0.911 for cadence). Simultaneous tracking of multiple targets by the 2D-LiDAR system was also demonstrated. The novel system might be useful in space and resource constrained clinical practice for older adults.
| S-EPMC7826665 | biostudies-literature
Project description:BACKGROUND:People with multiple sclerosis (PwMS) frequently have impaired balance from an early stage of the disease. Balance difficulties can be divided into categories; although, to date, these lack scientific foundation. Impaired balance in PwMS can be addressed using specific and challenging exercises. Such exercises should provide an optimal challenge point; however, the difficulty of balance exercises is often unknown, making it difficult to target the exercises to an individual's abilities. The aims of this study were: to develop an exercise programme for PwMS relating the exercises to the balance problem categories; to establish the order of difficulty of exercises in each category and; to evaluate the content and structural validity of the exercise programme. METHODS:A "construct map" approach was used to design and develop an exercise programme for PwMS. Potentially relevant balance exercises were identified, then a framework was set up, comprising four dimensions (subsequently reduced to three dimensions) of balance exercises. The relevance, comprehensibility, and comprehensiveness of the exercise programme were rated by 13 physiotherapists, who also linked 19 key exercises to balance categories. A total of 65 PwMS performed the 19 balance exercises, rated their difficulty and commented on the relevance and comprehensibility of each exercise. A Rasch model was used to evaluate the relative difficulty of the exercises. To assess fit of the data to the Rasch model a rating scale model was used, which is a unidimensional latent trait model for polytomous item responses. RESULTS:Evaluation by the physiotherapists and PwMS indicated that the content validity of the exercise programme was adequate. Rasch analysis showed that the latent trait "balance exercises in PwMS" comprised three subdimensions ("stable BOS", "sway" and "step and walk"). The 19 balance exercises showed adequate fit to the respective dimensions. The difficulties of the balance exercises were adequate to cover the ability spectrum of the PwMS. CONCLUSION:A balance exercise programme for PwMS comprising three dimensions of balance exercises was developed. Difficulty estimates have been established for each of the exercises, which can be used for targeted balance training. Content and structural validity of the programme was adequate.
| S-EPMC8672542 | biostudies-literature
Project description:BackgroundThe aim of the current study was to evaluate the structural validity of the 20-item long-term conditions questionnaire (LTCQ) and to explore a potential short-form version of the scale using Rasch analysis.MethodsData were collected through postal surveys (February 2016-January 2017) from a sample of 1,211 participants diagnosed with at least one long-term condition (LTC). Identified participants were invited through either local authorities for a social care cohort (n?=?294) or primary care practices for a health care cohort (n?=?917). Participants were mailed a survey, including the LTCQ, demographic questions, a comorbidities measure, and other validated outcome measures. Respondents were invited to complete a follow-up survey including the LTCQ for assessment of reproducibility.ResultsThe main assumptions of the Rasch model from the LTCQ were fulfilled, although infit and outfit indices indicated some items showed misfit. Misfitted items, items that did not have a preceding set or showed some local dependence were removed one at a time, with the remaining candidate items to form an 8-item short version, the LTCQ-8. The Rasch model for the LTCQ-8 explained 64% variance and had a reliability estimate greater than 0.80. Several items in the LTCQ showed uniform differential item function (DIF) in relation to the number of reported LTCs, age, cohort and type of LTCs, but fewer items exhibited DIF in the LTCQ-8. Spearman's rho correlations between the LTCQ and the LTCQ-8 were strong across the total sample and various subgroups. Correlations between the LTCQ-8 and all reference measures were moderate to strong, and comparable to correlations found between the LTCQ and these measures.ConclusionsThe LTCQ measures a unidimensional construct, and it is therefore acceptable to use a summed total score. The LTCQ-8 also met the assumption of unidimensionality and had comparable construct validity with the LTCQ. Additional validation is required in an independent sample.
| S-EPMC7706038 | biostudies-literature
Project description:BackgroundAppropriate dietary intake can have a significant influence on athletic performance. There is a growing consensus on sports nutrition and professionals working with athletes often provide dietary education. However, due to the limitations of existing sports nutrition knowledge questionnaires, previous reports of athletes' nutrition knowledge may be inaccurate.MethodsAn updated questionnaire has been developed based on a recent review of sports nutrition guidelines. The tool has been validated using a robust methodology that incorporates relevant techniques from classical test theory (CTT) and Item response theory (IRT), namely, Rasch analysis.ResultsThe final questionnaire has 89 questions and six sub-sections (weight management, macronutrients, micronutrients, sports nutrition, supplements, and alcohol). The content and face validity of the tool have been confirmed based on feedback from expert sports dietitians and university sports students, respectively. The internal reliability of the questionnaire as a whole is high (KR = 0.88), and most sub-sections achieved an acceptable internal reliability. Construct validity has been confirmed, with an independent T-test revealing a significant (p < 0.001) difference in knowledge scores of nutrition (64 ± 16%) and non-nutrition students (51 ± 19%). Test-retest reliability has been assured, with a strong correlation (r = 0.92, p < 0.001) between individuals' scores on two attempts of the test, 10 days to 2 weeks apart. Three of the sub-sections fit the Rasch Unidimensional Model.ConclusionsThe final version of the questionnaire represents a significant improvement over previous tools. Each nutrition sub-section is unidimensional, and therefore researchers and practitioners can use these individually, as required. Use of the questionnaire will allow researchers to draw conclusions about the effectiveness of nutrition education programs, and differences in knowledge across athletes of varying ages, genders, and athletic calibres.
| S-EPMC5543556 | biostudies-literature
Project description:The Intermittent Exotropia Questionnaire (IXTQ) is a patient, proxy, and parental report of quality of life specific to children with intermittent exotropia. We refine the IXTQ using Rasch analysis to improve reliability and validity.Rasch analysis was performed on responses of 575 patients with intermittent exotropia enrolled from May 15, 2008, through July 24, 2013, and their parents from each of the 4 IXTQ health-related quality-of-life questionnaires (child 5 through 7 years of age and child 8 through 17 years of age, proxy, and parent questionnaires). Questionnaire performance and structure were confirmed in a separate cohort of 379 patients with intermittent exotropia. One item was removed from the 12-item child and proxy questionnaires, and response options in the 8- to 17-year-old child IXTQ and proxy IXTQ were combined into 3 response options for both questionnaires. Targeting was relatively poor for the child and proxy questionnaires. For the parent questionnaire, 3 subscales (psychosocial, function, and surgery) were evident. One item was removed from the psychosocial subscale. Resulting subscales had appropriate targeting.The Rasch-revised IXTQ may be a useful instrument for determining how intermittent exotropia affects health-related quality of life of children with intermittent exotropia and their parents, particularly for cohort studies.
| S-EPMC4393341 | biostudies-literature