On the Relationship Between Item Stem Formulation and Criterion Validity of Multiple-Component Measuring Instruments.
Ontology highlight
ABSTRACT:
SUBMITTER: Menold N
PROVIDER: S-EPMC8850765 | biostudies-literature | 2022 Apr
REPOSITORIES: biostudies-literature
Publications
On the Relationship Between Item Stem Formulation and Criterion Validity of Multiple-Component Measuring Instruments.
Menold Natalja N Raykov Tenko T
Educational and psychological measurement 20210208 2
The possible dependency of criterion validity on item formulation in a multicomponent measuring instrument is examined. The discussion is concerned with evaluation of the differences in criterion validity between two or more groups (populations/subpopulations) that have been administered instruments with items having differently formulated item stems. The case of complex item stems involving two stimuli description sentences (double-barreled questions) is thereby compared with the setting where ...[more]
Similar Datasets
Project description:Validity coefficients for multicomponent measuring instruments are known to be affected by measurement error that attenuates them, affects associated standard errors, and influences results of statistical tests with respect to population parameter values. To account for measurement error, a latent variable modeling approach is discussed that allows point and interval estimation of the relationship of an underlying latent factor to a criterion variable in a setting that is more general than the commonly considered homogeneous psychometric test case. The method is particularly helpful in validity studies for scales with a second-order factorial structure, by allowing evaluation of the relationship between the second-order factor and a criterion variable. The procedure is similarly useful in studies of discriminant, convergent, concurrent, and predictive validity of measuring instruments with complex latent structure, and is readily applicable when measuring interrelated traits that share a common variance source. The outlined approach is illustrated using data from an authoritarianism study.
| S-EPMC7328229 | biostudies-literature
Project description:Maladaptive coping such as fear avoidance behavior can prolong recovery from mild traumatic brain injury (mTBI). Routine assessment of fear avoidance may improve management of mTBI. This study aimed to validate a single-item measure of fear avoidance to make its assessment more pragmatic. The present study is a secondary analysis of a clinical trial that involved adults with persistent post-concussion symptoms (N = 90, 63% female). Participants completed the single-item fear avoidance rating, a validated legacy measure of fear avoidance (Fear Avoidance Behavior after Traumatic Brain Injury; FAB-TBI), and measures of anxiety (General Anxiety Disorder-7), depression (Patient Health Questionnaire-9), post-concussion symptoms (Rivermead Postconcussion Symptoms Questionnaire), and disability (World Health Organization Disability Assessment Schedule 12.0). Questionnaires were completed twice, at baseline (mean 18.1 weeks post injury) and again 12-16 weeks later following study-delivered rehabilitation in addition to usual care. We analyzed the associations (Spearman's correlations) and agreement (weighted Kappa) between the single-item and FAB-TBI at baseline, posttreatment, individual FAB-TBI item scores, and the change in scores between baseline and posttreatment. In addition, we examined correlations between the single-item fear avoidance measure and related constructs, including anxiety, depression, post-concussion symptoms, and disability. The single-item fear avoidance measure correlated strongly with the FAB-TBI both at baseline and following treatment (ρ = 0.63 - 0.67, p < .001), and moderately with FAB-TBI item scores (ρ = 0.4 - 0.6). The correlation between the change in these scores from baseline to posttreatment was moderate (ρ = 0.45, p < .001). Agreement between the single-item fear avoidance measure and discretized FAB-TBI scores was moderate (κ = 0.45 - 0.51). Before and after treatment, the single-item fear avoidance measure correlated moderately with anxiety (ρ = 0.34), depression (ρ = 0.43), post-concussion symptoms (ρ = 0.50), and disability (ρ = 0.43). The FAB-TBI was more strongly correlated with these measures (ρ = 0.53 - 0.73). In summary, the present study supports the criterion validity of the single-item fear avoidance measure. This measure may be a useful screening and monitoring tool for patients with mTBI but is not a substitute for the FAB-TBI questionnaire.
| S-EPMC11437642 | biostudies-literature
Project description:The validity of a local positioning system (LPS) to measure inter-unit distance was investigated during a team sport movement circuit. Eight recreationally active, female indoor team-sport players completed a circuit, comprising seven types of movements (walk, jog, jump, sprint, 45° change of direction and shuffle), on an indoor court. Participants wore a receiver tag (ClearSky T6, Catapult Sports) and seven reflective markers, to allow for a comparison with the reference system (©Vicon Motion Systems, Oxford Metrics, UK). Inter-unit distance was collected for each combination of participants. Validity was assessed via root mean square error, mean bias and percentage of variance accounted for, both as an overall dataset and split into distance bands. The results presented a mean root mean square error of 0.20 ± 0.05 m, and mean bias detected an overestimation for all distance bands. The LPS shows acceptable accuracy for measuring inter-unit distance, opening up opportunities to utilise player tracking for tactical variables indoors.
| S-EPMC7374308 | biostudies-literature
Project description:IntroductionDegenerative joint disease and associated pain are common in cats, particularly in older cats. There is a need for treatment options, however evaluation of putative therapies is limited by a lack of suitable, validated outcome measures that can be used in the target population of client owned cats. The objectives of this study were to evaluate low-dose daily meloxicam for the treatment of pain associated with degenerative joint disease in cats, and further validate two clinical metrology instruments, the Feline Musculoskeletal Pain Index (FMPI) and the Client Specific Outcome Measures (CSOM).MethodsSixty-six client owned cats with degenerative joint disease and owner-reported impairments in mobility were screened and enrolled into a double-masked, placebo-controlled, randomized clinical trial. Following a run-in baseline period, cats were given either placebo or meloxicam for 21 days, then in a masked washout, cats were all given placebo for 21 days. Subsequently, cats were given the opposite treatment, placebo or meloxicam, for 21 days. Cats wore activity monitors throughout the study, owners completed clinical metrology instruments following each period.ResultsActivity counts were increased in cats during treatment with daily meloxicam (p<0.0001) compared to baseline. The FMPI results and activity count data offer concurrent validation for the FMPI, though the relationship between baseline activity counts and FMPI scores at baseline was poor (R2=0.034). The CSOM did not show responsiveness for improvement in this study, and the relationship between baseline activity counts and CSOM scores at baseline was similarly poor (R2=0.042).ConclusionsRefinements to the FMPI, including abbreviation of the instrument and scoring as percent of possible score are recommended. This study offered further validation of the FMPI as a clinical metrology instrument for use in detecting therapeutic efficacy in cats with degenerative joint disease.
| S-EPMC4498683 | biostudies-literature
Project description:ObjectiveTo analyse the Nutritional Knowledge Test (NKT) using Item Response Theory (ITR) analysis and to assess the construct validity of the Nutritional Knowledge Scale (NKTS) and its associations with adolescent food group consumption and nutritional biomarkers.DesignCross-sectional study.SettingMulticentre investigation conducted in ten European cities.ParticipantsAdolescents aged 12·5-17·5 years (n 3215) who completed over 75 % of the NKT.ResultsFactor analysis indicated that the NKT can be analysed with a one-dimensional model. Eleven out of twenty-three items from the NKT presented adequate parameters and were selected to be included in the NKTS. Nutrition knowledge was positively associated with consumption of fruits, cereals, dairy products, pulses, meat and eggs, and fish, as well as with blood concentrations of vitamin C, β-carotene, n-3 fatty acids, holo-transcobalamin, cobalamin and folate; nutrition knowledge was negatively associated with intake of olives and avocado, alcohol and savoury snacks.ConclusionsThe NKTS assessed nutritional knowledge adequately and it is proposed as a new tool to investigate this subject in future studies.
| S-EPMC10260635 | biostudies-literature
Project description:BackgroundThe psychometric properties of self-administered instruments for measuring patient satisfaction with removable dentures should be tested before inviting patients to express their opinions. This study aimed to evaluate the validity and reliability of new instruments in the Arabic language that measure patient satisfaction with all types of removable dentures.MethodsA three-step methodology was used to translate and test the instruments. In step one, the instruments were translated from tested German instruments to develop the pilot questionnaires. In step two, the face validity of the pilot questionnaires was tested through three rounds of interviews. There were 15, 13, and 15 participants per round, respectively. At the end of every round, the results of the interviews were discussed with an expert panel. The expert panel confirmed the form and the type of questionnaires' adjustments before a new round of interviews began. At the end of step two, the final form of the questionnaires was reached. In step three, 235 questionnaires were distributed to 133 participants to estimate the construct validity of the upper jaw and the lower jaw questionnaires. After one week, the participants were asked to complete the questionnaires again. A total of 102 questionnaires were returned and used to assess the instruments' reliability. Factor analysis was used to assess the construct validity. The intraclass correlation coefficient and Cronbach's alpha were used to estimate the reliability and suitability of the items in the indexes.ResultsThe result of step one was two pilot questionnaires. The pilot questionnaires were adjusted in step two. At the end of step two, the questionnaires proved to have good face validity. Factor analyses in step three revealed that only one factor could be retained. The one-factor model explained 60.95% and 63.06 of the total variance of the upper jaw and lower jaw questionnaires, respectively. The items in every questionnaire shared the same cluster and could be summed to form an upper jaw index and lower jaw index that reflected patient satisfaction with removable dentures. Cronbach's alpha values indicated excellent internal consistency and reliability for the upper jaw questionnaire (α = 0.91) and the lower jaw questionnaire (α = 0.92). Intraclass correlation coefficient values ranged from 0.72 to 0.95, which can be considered "moderate" to "excellent".ConclusionsThe Arabic version of questionnaires and indexes assessing patient satisfaction with upper and lower removable dentures are reliable and valid self-administered instruments.
| S-EPMC8442818 | biostudies-literature
Project description:BackgroundThe Roland-Morris Disability Questionnaire (RMDQ) is one of the most used instruments to measure self-reported disability in patients with low back pain, however, the uncertainty on which version to use may lead to inadequate disability measurement and consequently, improper management of patients with chronic low back pain.ObjectiveTo propose a short version of the RMDQ, compare it with the other short versions presented by the specialized literature, and identify the best internal structure of the RMDQ for the Brazilian population.MethodsThis is a cross-sectional study in which we used confirmatory factor analysis to identify the best structure of the RMDQ. We assessed 545 participants, most of which were women, aged ≥ 30 years old, single, with mean low back pain intensity ~ 5 points, and mean pain chronicity ~ 72 months. We used lavaan and semPlot packages, with implementation of a tetrachoric matrix and the robust diagonally weighted least squares extraction method. We also used fit indices chi-square/degree of freedom, comparative fit index, Tucker-Lewis index, root mean square error of approximation, and standardized root mean squared residual. For the comparison between models, we considered the structure with the lowest values of the Akaike information criterion and Bayesian information criterion. In addition, we assessed criterion validity via Spearman's correlation coefficient to correlate the long and short versions. In this study, the 15-item structure was created through the use of modification indices to identify redundant items (9 items were excluded).ResultsRMDQ structure with one domain and 15 items and the structure with two domains and 16 items showed all fit indices with adequate values, but the one-dimensional version showed the lowest Akaike information criterion and Bayesian information criterion values. Regarding criterion validity, correlation between the RMDQ with 24 items and 15 items is adequate (rho = 0.954, p < 0.001).ConclusionThe RMDQ-15 is a short version of the RMDQ instrument with the most adequate internal structure and satisfactorily correlated with the long version of the instrument.
| S-EPMC9652881 | biostudies-literature
Project description:BackgroundEcological momentary assessment (EMA) is an in situ method of gathering self-report on behaviors using mobile devices. In typical phone-based EMAs, participants are prompted repeatedly with multiple-choice questions, often causing participation burden. Alternatively, microinteraction EMA (micro-EMA or μEMA) is a type of EMA where all the self-report prompts are single-question surveys that can be answered using a 1-tap glanceable microinteraction conveniently on a smartwatch. Prior work suggests that μEMA may permit a substantially higher prompting rate than EMA, yielding higher response rates and lower participation burden. This is achieved by ensuring μEMA prompt questions are quick and cognitively simple to answer. However, the validity of participant responses from μEMA self-report has not yet been formally assessed.ObjectiveIn this pilot study, we explored the criterion validity of μEMA self-report on a smartwatch, using physical activity (PA) assessment as an example behavior of interest.MethodsA total of 17 participants answered 72 μEMA prompts each day for 1 week using a custom-built μEMA smartwatch app. At each prompt, they self-reported whether they were doing sedentary, light/standing, moderate/walking, or vigorous activities by tapping on the smartwatch screen. Responses were compared with a research-grade activity monitor worn on the dominant ankle simultaneously (and continuously) measuring PA.ResultsParticipants had an 87.01% (5226/6006) μEMA completion rate and a 74.00% (5226/7062) compliance rate taking an average of only 5.4 (SD 1.5) seconds to answer a prompt. When comparing μEMA responses with the activity monitor, we observed significantly higher (P<.001) momentary PA levels on the activity monitor when participants self-reported engaging in moderate+vigorous activities compared with sedentary or light/standing activities. The same comparison did not yield any significant differences in momentary PA levels as recorded by the activity monitor when the μEMA responses were randomly generated (ie, simulating careless taps on the smartwatch).ConclusionsFor PA measurement, high-frequency μEMA self-report could be used to capture information that appears consistent with that of a research-grade continuous sensor for sedentary, light, and moderate+vigorous activity, suggesting criterion validity. The preliminary results show that participants were not carelessly answering μEMA prompts by randomly tapping on the smartwatch but were reporting their true behavior at that moment. However, more research is needed to examine the criterion validity of μEMA when measuring vigorous activities.
| S-EPMC7991987 | biostudies-literature
Project description:BackgroundMultiple sclerosis (MS) is a chronic demyelinating/neurodegenerative disease associated with change in cognitive function (CF) over time. This systematic review aims to describe the instruments used to measure change in CF over time in people with MS (PwMS).MethodsPubMed, OVID, Web of Science, and Scopus databases were searched in English until May 2021. Articles were included if they had at least 100 participants and at least a 1-year interval between baseline and last follow-up measurement of CF. Results were quantitatively synthesized, presented in tables and risk of bias was assessed with the Newcastle-Ottawa Scale.ResultsFifty-seven articles met the inclusion criteria (41,623 PwMS and 1105 controls). An intervention (drug/rehabilitation) was assessed in 22 articles. In the studies that used a test battery, Visual and verbal learning and memory were the most frequently measured domains, but when studies that used test battery or a single test are combined, Information processing speed was the most measured. The Symbol Digit Modalities Test (SDMT) was the most frequently used test as a single test and in a test battery combined. Most studied assessed "change in CF" as cognitive decline defined as 1 or more tests measured as ≥ 1.5 SD from the study control or normative mean in a test battery at baseline and follow-up. Meta-analysis of change in SDMT scores with seven articles indicated a nonstatistically significant -0.03 (95% CI -0.14, 0.09) decrease in mean SDMT score per year.ConclusionThis study highlights the slow rate of measured change in cognition in PwMS and emphasizes the lack of a gold standard test and consistency in measuring cognitive change at the population level. More sensitive testing utilizing multiple domains and longer follow-up may define subgroups where CF change follows different trajectories thus allowing targeted interventions to directly support those where CF is at greatest risk of becoming a clinically meaningful issue.
| S-EPMC10275522 | biostudies-literature
Project description:Study objectivesThis review aimed to summarize instruments that measure one or more domains of sleep health (i.e. duration, quality, efficiency, timing, daytime sleepiness and sleep-related behaviors) in a general population of 4-12-year old children, and to assess these instruments' content validity. Other measurement properties were evaluated for instruments with indications of sufficient content validity.MethodsA systematic literature search was performed in PubMed, PsycINFO, Web of Science, and EmBase. Methodological quality, content validity, and other measurement properties were assessed via the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) methodology. Instruments with indications of sufficient content validity (i.e. relevance, comprehensiveness and comprehensibility) were further evaluated on other measurement properties (i.e. other aspects of validity, reliability, responsiveness). A modified GRADE approach was applied to determine the quality of evidence.ResultsTwenty instruments, containing 36 subscales, were included. None of the instruments measured all sleep health domains. For five (subscales of) instruments sufficient relevance and comprehensibility was found. The quality of evidence ranged from very low to moderate. For these five instruments all additional measurement properties were assessed. Sufficient results were found for structural validity (n = 1), internal consistency (n = 1), and construct validity (n = 1), with quality of evidence ranging from very low to high.ConclusionsSeveral (subscales of) instruments measuring domains of child sleep health showed good promise, demonstrating sufficient relevance, comprehensibility, and some also sufficient results on other measurement properties. However, more high quality studies on instrument development and the evaluation of measurement properties are required.PROSPERO registration number: CRD42021224109.
| S-EPMC9644118 | biostudies-literature