Transitivity Violations Undermine Rating Scales in Motivation Research
Ontology highlight
ABSTRACT:
SUBMITTER: Tyumeneva Y
PROVIDER: S-EPMC8514613 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Inattention, impulsivity and hyperactivity are the primary behaviors associated with Attention Deficit / Hyperactivity Disorder (ADHD). Previous studies proved that peripheral blood gene expression signature could mirror central nervous system disease. This study determined if gene expression in blood correlated with inattention, hyperactivity/impulsivity rating scales and/or both in subjects with Tourette syndrome (TS).
2014-06-30 | GSE30470 | GEO
Project description:Many rating scales have been applied to the evaluation of dystonia, but only few have been assessed for clinimetric properties. The Movement Disorders Society commissioned this task force to critique existing dystonia rating scales and place them in the clinical and clinimetric context. A systematic literature review was conducted to identify rating scales that have either been validated or used in dystonia. Thirty-six potential scales were identified. Eight were excluded because they did not meet review criteria, leaving 28 scales that were critiqued and rated by the task force. Seven scales were found to meet criteria to be "recommended": the Blepharospasm Disability Index is recommended for rating blepharospasm; the Cervical Dystonia Impact Scale and the Toronto Western Spasmodic Torticollis Rating Scale for rating cervical dystonia; the Craniocervical Dystonia Questionnaire for blepharospasm and cervical dystonia; the Voice Handicap Index (VHI) and the Vocal Performance Questionnaire (VPQ) for laryngeal dystonia; and the Fahn-Marsden Dystonia Rating Scale for rating generalized dystonia. Two "recommended" scales (VHI and VPQ) are generic scales validated on few patients with laryngeal dystonia, whereas the others are disease-specific scales. Twelve scales met criteria for "suggested" and 7 scales met criteria for "listed." All the scales are individually reviewed in the online information. The task force recommends 5 specific dystonia scales and suggests to further validate 2 recommended generic voice-disorder scales in dystonia. Existing scales for oromandibular, arm, and task-specific dystonia should be refined and fully assessed. Scales should be developed for body regions for which no scales are available, such as lower limbs and trunk.
| S-EPMC4207366 | biostudies-literature
Project description:Alcohol intoxication is well known to impair a number of cognitive abilities required for sound decision making. We tested whether an intoxicating dose of alcohol altered whether individuals satisfied a basic property of rational decision making, transitivity of preference. Our study was within-subjects in design and our analysis teased apart stable, yet error-prone, preferences from variable, error-free preferences. We find that alcohol intoxication does not appear to play a major role in determining whether subjects violate transitivity. For a minority of individuals, we find that alcohol intoxication does impact how they select among and/or perceive lotteries with similar attribute values. This, in turn, can cause them to alter various aspects of their preference structure.
| S-EPMC8294651 | biostudies-literature
Project description:BackgroundResearchers setting out to conduct research employing questionnaires in non-English speaking populations need instruments that have been validated in the indigenous languages. In this study we have tried to review the literature on the status of cross-cultural and/or criterion validity of all the questionnaires measuring psychiatric symptoms available in Urdu language.MethodsA search of Medline, Embase, PsycINFO and http://www.pakmedinet.com was conducted using the search terms; Urdu psychiatric rating scale, and Urdu and Psychiatry. References of retrieved articles were searched. Only studies describing either cross-cultural or criterion validation of a questionnaire in Urdu measuring psychiatric symptoms were included.ResultsThirty two studies describing validation of 19 questionnaires were identified. Six of these questionnaires were developed indigenously in Urdu while thirteen had been translated from English. Of the six indigenous questionnaires five had had their criterion validity examined. Of the thirteen translated questionnaires only four had had both their cross-cultural and criterion validity assessed.ConclusionThere is a paucity of validated questionnaires assessing psychiatric symptoms in Urdu. The BSI, SRQ and AKUADS are the questionnaires that have been most thoroughly evaluated in Urdu.
| S-EPMC2186305 | biostudies-literature
Project description:ObjectiveExecutive functions are commonly measured using rating scales and performance tests. However, replicated evidence indicates weak/nonsignificant cross-method associations that suggest divergent rather than convergent validity. The current study is the first to investigate the relative concurrent and predictive validities of executive function tests and ratings using (a) multiple gold-standard performance tests, (b) multiple standardized rating scales completed by multiple informants, and (c) both performance-based and ratings-based assessment of academic achievement-a key functional outcome with strong theoretical links to executive function.MethodA well-characterized sample of 136 children oversampled for ADHD and other forms of child psychopathology associated with executive dysfunction (ages 8-13; 68% Caucasian/non-Hispanic) completed a counterbalanced series of executive function and academic tests. Parents/teachers completed executive function ratings; teachers also rated children's academic performance.ResultsThe executive function tests/ratings association was modest (r = .30) and significantly lower than the academic tests/ratings association (r = .63). Relative to ratings, executive function tests showed significantly higher cross-method predictive validity and significantly better within-method prediction; executive function ratings failed to demonstrate improved within-method prediction. Both methods uniquely predicted academic tests and ratings.ConclusionThese findings replicate prior evidence that executive function tests and ratings cannot be used interchangeably as executive function measures in research and clinical applications, while suggesting that executive function tests may have superior validity for predicting academic behavior/achievement. (PsycInfo Database Record (c) 2020 APA, all rights reserved).
| S-EPMC7483691 | biostudies-literature
Project description:Rating scales are popular methods for generating quantitative data directly by persons rather than automated technologies. But scholars increasingly challenge their foundations. This article contributes epistemological and methodological analyses of the processes involved in person-generated quantification. They are crucial for measurement because data analyses can reveal information about study phenomena only if relevant properties were encoded systematically in the data. The Transdisciplinary Philosophy-of-Science Paradigm for Research on Individuals (TPS-Paradigm) is applied to explore psychological and social-science concepts of measurement and quantification, including representational measurement theory, psychometric theories and their precursors in psychophysics. These are compared to theories from metrology specifying object-dependence of measurement processes and subject-independence of outcomes as key criteria, which allow tracing data to the instances measured and the ways they were quantified. Separate histories notwithstanding, the article's basic premise is that general principles of scientific measurement and quantification should apply to all sciences. It elaborates principles by which these metrological criteria can be implemented also in psychology and social sciences, while considering their research objects' peculiarities. Application of these principles is illustrated by quantifications of individual-specific behaviors ('personality'). The demands rating methods impose on data-generating persons are deconstructed and compared with the demands involved in other quantitative methods (e.g., ethological observations). These analyses highlight problematic requirements for raters. Rating methods sufficiently specify neither the empirical study phenomena nor the symbolic systems used as data nor rules of assignment between them. Instead, pronounced individual differences in raters' interpretation and use of items and scales indicate considerable subjectivity in data generation. Together with recoding scale categories into numbers, this introduces a twofold break in the traceability of rating data, compromising interpretability of findings. These insights question common reliability and validity concepts for ratings and provide novel explanations for replicability problems. Specifically, rating methods standardize only data formats but not the actual data generation. Measurement requires data generation processes to be adapted to the study phenomena's properties and the measurement-executing persons' abilities and interpretations, rather than to numerical outcome formats facilitating statistical analyses. Researchers must finally investigate how people actually generate ratings to specify the representational systems underlying rating data.
| S-EPMC6308206 | biostudies-other
Project description:Upper and lower gastrointestinal dysautonomia symptoms (GIDS)--sialorrhea, dysphagia, and constipation are common in Parkinson's disease (PD) and often socially as well as physically disabling for patients. Available invasive quantitative measures for assessing these symptoms and their response to therapy are time-consuming, require specialized equipment, can cause patient discomfort and present patients with risk. The Movement Disorders Society commissioned a task force to assess available clinical rating scales, critique their clinimetric properties, and make recommendations regarding their clinical utility. Six clinical researchers and a biostatistician systematically searched the literature for scales of sialorrhea, dysphagia, and constipation, evaluated the scales' previous use, performance parameters, and quality of validation data (if available). A scale was designated "Recommended" if the scale was used in clinical studies beyond the group that developed it, has been specifically used in PD reports, and clinimetric studies have established that it is a valid, reliable, and sensitive. "Suggested" scales met at least part of the above criteria, but fell short of meeting all. Based on the systematic review, scales for individual symptoms of sialorrhea, dysphagia, and constipation were identified along with three global scales that include these symptoms in the context of assessing dysautonomia or nonmotor symptoms. Three sialorrhea scales met criteria for Suggested: Drooling Severity and Frequency Scale (DSFS), Drooling Rating Scale, and Sialorrhea Clinical Scale for PD (SCS-PD). Two dysphagia scales, the Swallowing Disturbance Questionnaire (SDQ) and Dysphagia-Specific Quality of Life (SWAL-QOL), met criteria for Suggested. Although Rome III constipation module is widely accepted in the gastroenterology community, and the earlier version from the Rome II criteria has been used in a single study of PD patients, neither met criteria for Suggested or Recommended. Among the global scales, the Scales for Outcomes in PD-Autonomic (SCOPA-AUT) and Nonmotor Symptoms Questionnaire for PD (NMSQuest) both met criteria for Recommended, and the Nonmotor Symptoms Scale (NMSS) met criteria for Suggested; however, none specifically focuses on the target gastrointestinal symptoms (sialorrhea, dysphagia, and constipation) of this report. A very small number of rating scales have been applied to studies of gastrointestinal-related dysautonomia in PD. Only two scales met "Recommended" criteria and neither focuses specifically on the symptoms of sialorrhea, dysphagia, and constipation. Further scale testing in PD among the scales that focus on these symptoms is warranted, and no new scales are needed until the available scales are fully tested clinimetrically.
| S-EPMC4404514 | biostudies-literature
Project description:BACKGROUND:Cash and in-kind incentives can improve health outcomes in various settings; however, there is concern that incentives may 'crowd out' intrinsic motivation to engage in beneficial behaviors. We examined this hypothesis in a randomized trial of food and cash incentives for people living with HIV infection in Tanzania. METHODS:We analyzed data from 469 individuals randomized to one of three study arms: standard of care, short-term cash transfers, or short-term food assistance. Eligible participants were: 1) ?18 years old; 2) HIV-infected; 3) food insecure; and 4) initiated antiretroviral therapy (ART) ?90 days before the study. Food or cash transfers, valued at ~$11 per month and conditional on attending clinic visits, were provided for ?6 months. Intrinsic motivation was measured at baseline, 6, and 12 months using the autonomous motivation section of the Treatment Self-Regulation Questionnaire (TSRQ). We compared the change in TSRQ score from baseline to 6 and 12 months and the change within study arms. RESULTS:The mean intrinsic motivation score was 2.79 at baseline (range: 1-3), 2.91 at 6 months (range: 1-3), and 2.95 at 12 months (range: 2-3), which was 6 months after the incentives had ended. Among all patients, the intrinsic motivation score increased by 0.13 points at 6 months (95% CI (0.09, 0.17), Cohen's d = 0.29) and 0.19 points at 12 months (95% CI (0.14, 0.24), Cohen's d = 0.49). Intrinsic motivation also increased within each study group at 6 months: 0.15 points in the food arm (95% CI (0.09, 0.21), Cohen's d = 0.37), 0.11 points in the cash arm (95% CI (0.05, 0.18), Cohen's d = 0.25), and 0.08 points in the comparison arm (95% CI (-0.03, 0.19), Cohen's d = 0.21); findings were similar at 12 months. Increases in motivation were statistically similar between arms at 6 and 12 months. CONCLUSION:Intrinsic motivation for ART adherence increased significantly both overall and within the food and cash incentive arms, even after the incentive period was over. Increases in motivation did not differ by study group. These results suggest that incentive interventions for treatment adherence should not be withheld due to concerns of crowding out intrinsic motivation.
| S-EPMC6002082 | biostudies-literature
Project description:BackgroundA critical component that influences the measurement properties of a patient-reported outcome (PRO) instrument is the rating scale. Yet, there is a lack of general consensus regarding optimal rating scale format, including aspects of question structure, the number and the labels of response categories. This study aims to explore the characteristics of rating scales that function well and those that do not, and thereby develop guidelines for formulating rating scales.MethodsSeventeen existing PROs designed to measure vision-related quality of life dimensions were mailed for self-administration, in sets of 10, to patients who were on a waiting list for cataract extraction. These PROs included questions with ratings of difficulty, frequency, severity, and global ratings. Using Rasch analysis, performance of rating scales were assessed by examining hierarchical ordering (indicating categories are distinct from each other and follow a logical transition from lower to higher value), evenness (indicating relative utilization of categories), and range (indicating coverage of the attribute by the rating scale).ResultsThe rating scales with complicated question format, a large number of response categories, or unlabelled categories, tended to be dysfunctional. Rating scales with five or fewer response categories tended to be functional. Most of the rating scales measuring difficulty performed well. The rating scales measuring frequency and severity demonstrated hierarchical ordering but the categories lacked even utilization.ConclusionDevelopers of PRO instruments should use a simple question format, fewer (four to five) and labelled response categories.
| S-EPMC3503574 | biostudies-literature
Project description:The selection of an appropriate outcome measure is crucial to the success of a clinical trial, in order to obtain accurate results, which, in turn, influence patient care and future research. Outcomes that can be directly measured are mortality/survival. More frequently, neurology clinical trials evaluate outcomes that cannot be directly measured, such as disability, cognitive function, or change in symptoms of the condition under study. These complex outcomes are abstract ideas or latent constructs and are measured using rating scales. Functional rating scales typically assess the ability of patients to perform tasks and roles for everyday life. Rating scales should be valid (measure what they are supposed to measure), reliable (provide similar results if administered under the same conditions), and responsive (able to detect clinically important changes over time). The clinical relevance of rating scales depends on their ability to detect a minimal clinically important difference, and should be distinguished from statistical significance. Most rating scales are ordinal scales and have limitations. Modern psychometric methods of Rasch analysis and item response theory, termed latent trait theory, are increasingly being utilized to convert ordinal data to interval measurements, both to validate existing scales and to develop new scales. Patient-reported outcomes are being increasingly used in clinical trials and have a role in clinical quality assessment. The PROMIS and NeuroQoL databases are excellent resources for rigorously developed and validated patient-reported outcomes.
| S-EPMC5233630 | biostudies-literature