Dataset Information

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

ABSTRACT:

Background

In three recent studies, Maul demonstrated that sets of nonsense items can acquire excellent psychometric properties. Our aim was to find out why responses to nonsense items acquire a well-defined structure and high internal consistency.

Method

We designed two studies. In the first study, 610 participants responded to eight items where the central term (intelligence) was replaced by the term "gavagai". In the second study, 548 participants responded to seven items whose content was totally invented. We asked the participants if they gave any meaning to "gavagai", and conducted analyses aimed at uncovering the most suitable structure for modeling responses to meaningless items.

Results

In the first study, 81.3% of the sample gave "gavagai" meaning, while 18.7% showed they had given it no interpretation. The factorial structures of the two groups were very different from each other. In the second study, the factorial model fitted almost perfectly. However, further analysis revealed that the structure of the data was not continuous but categorical with three unordered classes very similar to midpoint, disacquiescent, and random response styles.

Discussion

Apparently good psychometric properties on meaningless scales may be due to (a) respondents actually giving an interpretation to the item and responding according to that interpretation, or (b) a false positive because the statistical fit of the factorial model is not sensitive to cases where the actual structure of the data does not come from a common factor. In conclusion, the problem is not in factor analysis, but in the ability of the researcher to elaborate substantive hypotheses about the structure of the data, to employ analytical procedures congruent with those hypotheses, and to understand that a good fit in factor analysis does not have a univocal interpretation and is not sufficient evidence of either validity nor good psychometric properties.

SUBMITTER: Arias VB

PROVIDER: S-EPMC7678495 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

Arias Victor B VB Ponce Fernando P FP Bruggeman Martin M Flores Noelia N Jenaro Cristina C

PeerJ 20201117

<h4>Background</h4>In three recent studies, Maul demonstrated that sets of nonsense items can acquire excellent psychometric properties. Our aim was to find out why responses to nonsense items acquire a well-defined structure and high internal consistency.<h4>Method</h4>We designed two studies. In the first study, 610 participants responded to eight items where the central term (intelligence) was replaced by the term "gavagai". In the second study, 548 participants responded to seven items whose ...[more]

PMID: 33240604

Similar Datasets

Project description:Our previous review of compassion measures in healthcare between 1985 and 2016 concluded that no available measure assessed compassion in healthcare in a comprehensive or methodologically rigorous fashion. The present study provided a comparative review of the design and psychometric properties of recently updated or newly published compassion measures. The search strategy of our previous review was replicated. PubMed, MEDLINE, CINAHL, and PsycINFO databases and grey literature were searched to identify studies that reported information on instruments that measure compassion or compassionate care in clinicians, physicians, nurses, healthcare students, and patients. Textual qualitative descriptions of included studies were prepared. Instruments were evaluated using the Evaluating Measures of Patient-Reported Outcomes (EMPRO) tool. Measures that underwent additional testing since our last review included the Compassion Competence Scale (CCS), the Compassionate Care Assessment Tool (CCAT)©, and the Schwartz Center Compassionate Care Scale (SCCCS)™. New compassion measures included the Sussex-Oxford Compassion for Others Scale (SOCS-O), a self-report measure of compassion for others; the Bolton Compassion Strengths Indicators (BSCI), a self-report measure of the characteristics (strengths) associated with a compassionate nurse; a five-item Tool to Measure Patient Assessment of Clinician Compassion (TMPACC); and the Sinclair Compassion Questionnaire (SCQ). The SCQ was the only measure that adhered to measure development guidelines, established initial construct validity by first defining the concept of interest, and included the patient perspective across all stages of development. The SCQ had the highest EMPRO overall score at 58.1, almost 9 points higher than any other compassion measure, and achieved perfect EMPRO subscale scores for internal consistency, reliability, validity, and respondent burden, which were up to 43 points higher than any other compassion measure. These findings establish the SCQ as the 'gold standard' compassion measure, providing an empirical basis for evaluations of compassion in routine care.

Dataset Information

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

Background

Method

Results

Discussion

Publications

A valid and reliable measure of nothing: disentangling the "Gavagai effect" in survey data.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets