OpenMx: A Modular Research Environment for Item Response Theory Method Development.
Ontology highlight
ABSTRACT:
SUBMITTER: Pritikin JN
PROVIDER: S-EPMC7495793 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:BACKGROUND:Brief screening instruments focusing on autism spectrum disorder (ASD) that can be administered in primary care are scarce; there is a need for shorter and more precise instruments. The Autism-Tics, AD/HD and other Comorbidities inventory (A-TAC) has previously been validated for ASD reporting excellent validity. This study aims to determine the psychometric properties of each item in the ASD domain (17 items) in the A-TAC using item response theory (IRT), and thereby construct and validate a short form that could be used as a screening instrument in the general population. METHODS:Since 2004, parents of all 9-year-old Swedish twins have been invited to participate in a telephone interview in the Child and Adolescent Twin Study in Sweden (CATSS). The CATSS is linked to the National Patient Register (NPR), which includes data from in- and outpatient care. Data on ASD (A-TAC) collected in CATSS were compared with diagnoses from the NPR. Diagnoses that had been made both before (previous validity) and after (predictive validity) the interviews were included. The sample was divided into a developmental sample and a validation sample. An IRT model was fitted to the developmental sample and item parameters were used to select a subset of items for the short form. The performance of the proposed short form was examined in the validation sample by the use of receiver operation characteristic curves. RESULTS:Four items which were able to discriminate among individuals with more autism traits were deemed sufficient for use in the short form. The values of the area under the receiver operating characteristic curve for a clinical diagnosis of ASD was .95 (previous validity) and .72 (predictive validity). CONCLUSIONS:The proposed short form with 4 out of the original 17 items from A-TAC, showed excellent previous validity while the predictive validity was fair. The validity of the short form was in agreement with previous validations of the full ASD domain. The short form can be a valuable screening instrument in primary care settings in order to identify individuals in need for further assessment and for use in epidemiological studies.
| S-EPMC6829932 | biostudies-literature
Project description:Testing item-level fit is important in scale development to guide item revision/deletion. Many item-level fit indices have been proposed in literature, yet none of them were directly applicable to an important family of models, namely, the higher order item response theory (HO-IRT) models. In this study, chi-square-based fit indices (i.e., Yen's Q 1, McKinley and Mill's G 2, Orlando and Thissen's S-X 2, and S-G 2) were extended to HO-IRT models. Their performances are evaluated via simulation studies in terms of false positive rates and correct detection rates. The manipulated factors include test structure (i.e., test length and number of dimensions), sample size, level of correlations among dimensions, and the proportion of misfitting items. For misfitting items, the sources of misfit, including the misfitting item response functions, and misspecifying factor structures were also manipulated. The results from simulation studies demonstrate that the S-G 2 is promising for higher order items.
| S-EPMC6291895 | biostudies-literature
Project description:Composite assessments aim to combine different aspects of a disease in a single score and are utilized in a variety of therapeutic areas. The data arising from these evaluations are inherently discrete with distinct statistical properties. This tutorial presents the framework of the item response theory (IRT) for the analysis of this data type in a pharmacometric context. The article considers both conceptual (terms and assumptions) and practical questions (modeling software, data requirements, and model building).
| S-EPMC5915608 | biostudies-literature
Project description:Advancements in item response theory (IRT) have led to models for dual dependence, which control for cluster and method effects during a psychometric analysis. Currently, however, this class of models does not include one that controls for when the method effects stem from two method sources in which one source functions differently across the aspects of another source (i.e., a nested method-source interaction). For this study, then, a Bayesian IRT model is proposed, one that accounts for such interaction among method sources while controlling for the clustering of individuals within the sample. The proposed model accomplishes these tasks by specifying a multilevel trifactor structure for the latent trait space. Details of simulations are also reported. These simulations demonstrate that this model can identify when item response data represent a multilevel trifactor structure, and it does so in data from samples as small as 250 cases nested within 50 clusters. Additionally, the simulations show that misleading estimates for the item discriminations could arise when the trifactor structure reflected in the data is not correctly accounted for. The utility of the model is also illustrated through the analysis of empirical data.
| S-EPMC6506989 | biostudies-literature
Project description:BackgroundModern psychometric methods based on item response theory (IRT) can be used to develop adaptive measures of health-related quality of life (HRQL). Adaptive assessment requires an item bank for each domain of HRQL. The purpose of this study was to develop item banks for five domains of HRQL relevant to arthritis.MethodsAbout 1,400 items were drawn from published questionnaires or developed from focus groups and individual interviews and classified into 19 domains of HRQL. We selected the following 5 domains relevant to arthritis and related conditions: Daily Activities, Walking, Handling Objects, Pain or Discomfort, and Feelings. Based on conceptual criteria and pilot testing, 219 items were selected for further testing. A questionnaire was mailed to patients from two hospital-based clinics and a stratified random community sample. Dimensionality of the domains was assessed through factor analysis. Items were analyzed with the Generalized Partial Credit Model as implemented in Parscale. We used graphical methods and a chi-square test to assess item fit. Differential item functioning was investigated using logistic regression.ResultsData were obtained from 888 individuals with arthritis. The five domains were sufficiently unidimensional for an IRT-based analysis. Thirty-one items were deleted due to lack of fit or differential item functioning. Daily Activities had the narrowest range for the item location parameter (-2.24 to 0.55) and Handling Objects had the widest range (-1.70 to 2.27). The mean (median) slope parameter for the items ranged from 1.15 (1.07) in Feelings to 1.73 (1.75) in Walking. The final item banks are comprised of 31-45 items each.ConclusionWe have developed IRT-based item banks to measure HRQL in 5 domains relevant to arthritis. The items in the final item banks provide adequate psychometric information for a wide range of functional levels in each domain.
| S-EPMC1550394 | biostudies-literature
Project description:OBJECTIVE:External shame reflects a person's anxiety that he or she might be rejected by others. The Other as Shamer Scale (OAS) is a scale for assessing external shame. The Japanese version of the OAS was developed, and its reliability and validity were examined using Item Response Theory (IRT). RESULTS:A survey was conducted with university students (N?=?199). Exploratory factor analysis of the results indicated a significantly high factor loading on the first factor, which was identical to the original version of the scale as well as high internal consistency. Moreover, the results confirmed that each item had adequate discrimination and information levels, suggesting that external shame could be discriminated against with high accuracy for a wide range of relatively low and relatively high external shame groups. These results suggest that the OAS could be used to screen external shame as a stress factor and to assess intervention effects.
| S-EPMC7133010 | biostudies-literature
Project description:BackgroundThe Multiple Sclerosis Quality of Life-54 (MSQOL-54, 52 items grouped in 12 subscales plus two single items) is the most used MS specific health related quality of life inventory.ObjectiveTo develop a shortened version of the MSQOL-54.MethodsMSQOL-54 dimensionality and metric properties were investigated by confirmatory factor analysis (CFA) and Rasch modelling (Partial Credit Model, PCM) on MSQOL-54s completed by 473 MS patients. Their mean age was 41 years, 65% were women, and median Expanded Disability Status Scale (EDSS) score was 2.0 (range 0-9.5). Differential item functioning (DIF) was evaluated for gender, age and EDSS. Dimensionality of the resulting short version was assessed by exploratory factor analysis (EFA) and CFA. Cognitive debriefing of the short instrument (vs. the original) was then performed on 12 MS patients.ResultsCFA of MSQOL-54 subscales showed that the data fitted the overall model well. Two subscales (Role Limitations--Physical, Role Limitations--Emotional) did not fit the PCM, and were removed; two other subscales (Health Perceptions, Social Function) did not fit the model, but were retained as single items. Sexual Satisfaction (single-item subscale) was also removed. The resulting MSQOL-29 consisted of 25 items grouped in 7 subscales, plus 4 single items. PCM fit statistics were within the acceptability range for all MSQOL-29 items except one which had significant DIF by age. EFA and CFA indicated adequate fit to the original two-factor (Physical and Mental Health Composites) hypothesis. Cognitive debriefing confirmed that MSQOL-29 was acceptable and had lost no key items.ConclusionsThe proposed MSQOL-29 is 50% shorter than MSQOL-54, yet preserves key quality of life dimensions. Prospective validation on a large, independent MS patient sample is ongoing.
| S-EPMC4831784 | biostudies-literature
Project description:Item response theory (IRT) is a psychometric method that provides probabilistic model-based measurements. Its use is relatively recent in the assessment of food consumption, especially through dietary assessment tools. This study aims (1) to develop a food-based diet quality scale for Brazilian schoolchildren using IRT, and (2) to apply the scale to a representative sample of schoolchildren from a Southern Brazilian city. The scale was developed with daily consumption frequency of foods from 835 students who completed the Food Intake and Physical Activity of Schoolchildren questionnaire. Questionnaire foods were grouped into 10 items according to their nutritional similarities and were evaluated by full-information factor analysis that indicated a dominant factor explaining 28% of the variance. Psychometric item analysis was performed using Samejima's model. The scale covered all levels of diet quality, from "very poor" (scores < 95) to "very good" (scores ≥ 130). Children who had higher diet quality scores consumed beans, meat, fish, eggs, fruits, vegetables, dairy products, and water more frequently, while reducing the consumption of ultraprocessed sugary foods, ultraprocessed savoury snacks and sausages, and sugary drinks. Of 6323 children, an average of less than 10% consumed the highest diet quality scores (good or very good diet quality) and about 60% of children consumed low diet quality scores. The scale can be applied to other schoolchildren with the same measure precision.
| S-EPMC8469312 | biostudies-literature
Project description:Responses to survey data are determined not only by item characteristics and respondents' trait standings but also by response styles. Recently, methods for modeling response style with personality and attitudinal data have turned toward the use of anchoring vignettes, which provide fixed rating targets. Although existing research is promising, a few outstanding questions remain. First, it is not known how many vignettes and vignette ratings are necessary to identify response style parameters. Second, the comparative accuracy of these models is largely unexplored. Third, it remains unclear whether correcting for response style improves criterion validity. Both simulated data and data observed from a population-representative sample responding to a measure of personality pathology (the Personality Inventory for DSM-5 [PID-5]) were modeled using an array of response style models. In simulations, most models estimating response styles outperformed the graded response model (GRM), and in observed data, all response style models were superior to the GRM. Correcting for response style had a small, but in some cases significant, effect on the prediction of self-reported social dysfunction.
| S-EPMC6297915 | biostudies-literature
Project description:Automated essay scoring (AES) is the task of automatically assigning scores to essays as an alternative to human grading. Conventional AES methods typically rely on manually tuned features, which are laborious to effectively develop. To obviate the need for feature engineering, many deep neural network (DNN)-based AES models have been proposed and have achieved state-of-the-art accuracy. DNN-AES models require training on a large dataset of graded essays. However, assigned grades in such datasets are known to be strongly biased due to effects of rater bias when grading is conducted by assigning a few raters in a rater set to each essay. Performance of DNN models rapidly drops when such biased data are used for model training. In the fields of educational and psychological measurement, item response theory (IRT) models that can estimate essay scores while considering effects of rater characteristics have recently been proposed. This study therefore proposes a new DNN-AES framework that integrates IRT models to deal with rater bias within training data. To our knowledge, this is a first attempt at addressing rating bias effects in training data, which is a crucial but overlooked problem.
| S-EPMC7334153 | biostudies-literature