Measuring the reliability of picture story exercises like the TAT.
Ontology highlight
ABSTRACT:
SUBMITTER: Gruber N
PROVIDER: S-EPMC3865338 | biostudies-literature | 2013
REPOSITORIES: biostudies-literature
Publications
Measuring the reliability of picture story exercises like the TAT.
Gruber Nicole N Kreuzpointner Ludwig L
PloS one 20131105 11
As frequently reported, psychometric assessments on Picture Story Exercises, especially variations of the Thematic Apperception Test, mostly reveal inadequate scores for internal consistency. We demonstrate that the reason for this apparent shortcoming is not caused by the coding system itself but from the incorrect use of internal consistency coefficients, especially Cronbach's α. This problem could be eliminated by using the category-scores as items instead of the picture-scores. In addition t ...[more]
Similar Datasets
Project description:BackgroundRecently, there has been growing interest in using smartphone applications to assess gait speed and quantify isometric core stability exercise intensity. The purpose of this study was to investigate the between-session reliability and minimal detectable change of a smartphone app for two dynamic exercise tests of the lumbopelvic complex.MethodsThirty-three healthy young and active students (age: 22.3 ± 5.9 years, body weight: 66.9 ± 11.3 kg, height: 167.8 ± 10.3 cm) participated in this study. Intraclass correlation coefficient (ICC), coefficient of variation (%CV), and Bland-Altman plots were used to verify the reliability of the test. The standard error of measurement (SEM) and the minimum detectable difference (MDD) were calculated for clinical applicability.ResultsThe ICCs ranged from 0.73 to 0.96, with low variation (0.9% to 4.8%) between days of assessments. The Bland-Altman plots and one-sample t-tests (p > 0.05) indicated that no dynamic exercise tests changed systematically. Our analyses showed that SEM 0.6 to 1.5 mm/s-2) and MDD (2.1 to 3.5 mm/s-2).ConclusionThe OCTOcore app is a reliable tool to assess core stability for two dynamic exercises. A minimal change of 3.5 mm/s-2 is needed to be confident that the change is not a measurement error between two sessions.
| S-EPMC6690332 | biostudies-literature
Project description:The aim of this study was to investigate the reliability and validity of an affordable wireless force sensor in measuring mean and peak forces during resistance training.A Suiff Pro wireless force sensor (Suiff, Spain) and a MuscleLab force platform (Ergotest, Norway) were used concurrently to assess tensile load and the ground reaction force resulting from an upright row exercise. Thirteen participants (28.2 ± 5.7 years, 76.2 ± 9.6 kg, 178.2 ± 9.2 cm) performed the exercise under three velocity conditions and isometrically. Each condition involved three sets of exercise. Mean (Fmean) and peak (Fpeak) force values from both sensors were collected and compared.Suiff Pro exhibited excellent reliability for Fmean and Fpeak (ICCs = 0.99). When compared to the criterion measures, Suiff Pro showed trivial standardized bias for Fmean (Mean = 0.00 [CI 95% = 0.00 to 0.01]) and Fpeak (-0.02 [-0.04 to 0.00]). The standardized typical error was also trivial for Fmean (0.03 [0.02 to 0.03]) and Fpeak (0.07 [0.05 to 0.09]). Correlations with the MuscleLab force platform were nearly perfect: Fmean (0.97 [0.94 to 0.98]; p<0.001); Fpeak (0.96 [0.92 to 0.97]; p<0.001).The findings demonstrate that the Suiff Pro sensor is reliable and valid device for measuring force during isometric and dynamic resistance training exercises. Therefore, practitioners can confidently use this device to monitor kinematic variables of resistance training exercises and to obtain real-time augmented feedback during a training session.
| S-EPMC10956869 | biostudies-literature
Project description:PurposeMeaningful changes in picture naming responses may be obscured when measuring accuracy instead of quality. A statistic that incorporates information about the severity and nature of impairments may be more sensitive to the effects of treatment.MethodWe analyzed data from repeated administrations of a naming test to 72 participants with stroke aphasia in a clinical trial for anomia therapy. Participants were divided into two groups for analysis to demonstrate replicability. We assessed reliability among response type scores from five raters. We then derived four summary statistics of naming ability and their changes over time for each participant: (a) the standard accuracy measure, (b) an accuracy measure adjusted for item difficulty, (c) an accuracy measure adjusted for item difficulty for specific response types, and (d) a distance measure adjusted for item difficulty for specific response types. While accuracy measures address the likelihood of a correct response, the distance measure reflects that different response types range in their similarity to the target. Model fit was assessed. The frequency of significant improvements and the average magnitude of improvements for each summary statistic were compared between treatment groups and a control group. Effect sizes for each model-based statistic were compared with the effect size for the standard accuracy measure.ResultsInterrater and intrarater reliability were near perfect, on average, though compromised somewhat by phonological-level errors. The effects of treatment were more evident, in terms of both frequency and magnitude, when using the distance measure versus the other accuracy statistics.ConclusionsConsideration of item difficulty and response types revealed additional effects of treatment on naming scores beyond those observed for the standard accuracy measure. The results support theories that assume naming ability is decomposable into subabilities rather than being monolithic, suggesting new opportunities for measuring treatment outcomes.Supplemental materialhttps://doi.org/10.23641/asha.17019515.
| S-EPMC9154021 | biostudies-literature
Project description:BackgroundFunctional MRI (fMRI) tasks are increasingly being used to advance knowledge of the etiology and maintenance of obesity and eating disorders. Thus, understanding the test-retest reliability of BOLD signal contrasts from these tasks is important.ObjectivesTo evaluate test-retest reliability of responses in reward-related brain regions to food receipt paradigms (palatable tastes, anticipated palatable tastes), food picture paradigms (high-calorie food pictures), a monetary reward paradigm (winning money and anticipating winning money), and a thin female model picture paradigm (thin female model pictures).MethodWe conducted secondary univariate contrast-based analyses in data drawn from 4 repeated-measures fMRI studies. Participants (Study 1: N = 60, mean [M] age = 15.2 ± 1.1 y; Study 2: N = 109, M age = 15.1 ± 0.9 y; Study 3: N = 39, M age = 21.2 ± 3.7 y; Study 4: N = 62, M age = 29.7 ± 6.2 y) completed the same tasks over 3-wk to 3-y test-retest intervals. Studies 3 and 4 included participants with eating disorders and obesity, respectively.ResultsTest-retest reliability of the food receipt and food picture paradigms was poor, with average ICC values ranging from 0.07 to 0.20. The monetary reward paradigm and the thin female model picture paradigm also showed poor test-retest reliability: average ICC values 0.21 and 0.12, respectively. Although several regions demonstrated moderate to good test-retest reliability, these results did not replicate across studies using similar paradigms. In Studies 3 and 4, but not Study 1, test-retest reliability in visual processing regions was moderate to good when contrasting single conditions with a low-level baseline.ConclusionsResults underscore the importance of examining the temporal reliability of fMRI tasks and call for the development and use of well-validated standardized fMRI tasks in eating- and obesity-related studies that can provide more reliable measures of neural activation. The trials were registered at clinicaltrials.gov as NCT02084836, NCT01949636, NCT03261050, and NCT03375853.
| S-EPMC8326039 | biostudies-literature
Project description:Multiple sequence alignment, which is of fundamental importance for comparative genomics, is a difficult problem and error-prone. Therefore, it is essential to measure the reliability of the alignments and incorporate it into downstream analyses. We propose a new probabilistic sampling-based alignment reliability (PSAR) score. Instead of relying on heuristic assumptions, such as the correlation between alignment quality and guide tree uncertainty in progressive alignment methods, we directly generate suboptimal alignments from an input multiple sequence alignment by a probabilistic sampling method, and compute the agreement of the input alignment with the suboptimal alignments as the alignment reliability score. We construct the suboptimal alignments by an approximate method that is based on pairwise comparisons between each single sequence and the sub-alignment of the input alignment where the chosen sequence is left out. By using simulation-based benchmarks, we find that our approach is superior to existing ones, supporting that the suboptimal alignments are highly informative source for assessing alignment reliability. We apply the PSAR method to the alignments in the UCSC Genome Browser to measure the reliability of alignments in different types of regions, such as coding exons and conserved non-coding regions, and use it to guide cross-species conservation study.
| S-EPMC3159474 | biostudies-literature
Project description:The Corpus Callosum (CC) is an important structure connecting the two brain hemispheres. As several neurodegenerative diseases are known to alter its shape, it is an interesting structure to assess as biomarker. Yet, currently, the CC-segmentation is often performed manually and is consequently an error prone and time-demanding procedure. In this paper, we present an accurate and automated method for corpus callosum segmentation based on T1-weighted MRI images. After the initial construction of a CC atlas based on healthy controls, a new image is subjected to a mid-sagittal plane (MSP) detection algorithm and a 3D affine registration in order to initialise the CC within the extracted MSP. Next, an active shape model is run to extract the CC. We calculated the reliability of most popular CC features (area, circularity, corpus callosum index and thickness profile) in healthy controls, Alzheimer's Disease patients and Multiple Sclerosis patients. Importantly, we also provide inter-scanner reliability estimates. We obtained an intra-class correlation coefficient (ICC) of over 0.95 for most features and most datasets. The inter-scanner reliability assessed on the MS patients was remarkably well and ranged from 0.77 to 0.97. In summary, we have constructed an algorithm that reliably detects the CC in 3D T1 images in a fully automated way in healthy controls and different neurodegenerative diseases. Although the CC area and the circularity are the most reliable features (ICC > 0.97); the reliability of the thickness profile (ICC > 0.90; excluding the tip) is sufficient to warrant its inclusion in future clinical studies.
| S-EPMC5094205 | biostudies-literature
Project description:BackgroundResistance and balance training are important exercise interventions for older populations living with chronic diseases. Accurately measuring if an individual is adhering to exercises as prescribed is important to determine if lack of improvement in health outcomes is because of issues with adherence. Measuring adherence to resistance and balance exercises is limited by current methods that depend heavily on self-report and are often better at and tailored towards capturing aerobic training parameters (e.g., step count, minutes of moderate to vigorous physical activity). Adherence measures must meet users' needs to be useful.MethodsUsing a Dillman tailored study design, we surveyed researchers who conduct exercise trials, clinicians who prescribe exercise for older adults, and older adults to determine: (1) how they are currently measuring adherence; (2) barriers and facilitators they have experienced to measurement; and (3) the information they would like collected about adherence (e.g., repetitions, sets, intensity, duration, frequency, quality). Surveys were disseminated internationally through professional networks, professional organizations, and social media. Participants completed an online survey between August 2021 and April 2022.ResultsEighty-eight older adults, 149 clinicians, and 41 researchers responded to the surveys. Most clinicians and researchers were between the ages of 30 and 39 years, and 70.0% were female. Most older adults were aged 70-79 years, and 46.6% were female. Diaries and calendars (either analog or digital) were the most common current methods of collecting adherence data. Users would like information about the intensity and quality of exercises completed that are presented in clear, easy to use formats that are meaningful for older adults where all data can be tracked in one place. Most older adults did not measure adherence because they did not want to, while clinicians most frequently reported not having measurement tools for adherence. Time, resources, motivation, and health were also identified as barriers to recording adherence.ConclusionsOur work provides information about current methods of measuring exercise adherence and suggestions to inform the design of future adherence measures. Future measures should comprehensively track adherence data in one place, including the intensity and quality of exercises.
| S-EPMC10470185 | biostudies-literature
Project description:Background: Belief in the paranormal is widespread worldwide. Recent surveys suggest that subjective experiences of the paranormal are common. A concise instrument that adequately evaluates beliefs as distinct from experiences does not currently exist. To address this gap, we created the Noetic Experiences and Beliefs Scale (NEBS) which evaluates belief and experience as separate constructs. Methods: The NEBS is a 20-item survey with 10 belief and 10 experience items rated on a visual analog scale from 0-100. In an observational study, the survey was administered to 361 general population adults in the United States and a subsample of 96 one month later. Validity, reliability and internal consistency were evaluated. A confirmatory factor analysis was conducted to confirm the latent variables of belief and experience. The survey was then administered to a sample of 646 IONS Discovery Lab participants to evaluate divergent validity and confirm belief and experience as latent variables of the model in a different population. Results: The NEBS demonstrated convergent validity, reliability and internal consistency (Cronbach's alpha Belief 0.90; Experience 0.93) and test-retest reliability (Belief: r = 0.83 ; Experience: r = 0.77). A confirmatory factor analysis model with belief and experience as latent variables demonstrated a good fit. The factor model was confirmed as having a good fit and divergent validity was established in the sample of 646 IONS Discovery Lab participants. Conclusions: The NEBS is a short, valid, and reliable instrument for evaluating paranormal belief and experience.
| S-EPMC7256471 | biostudies-literature
Project description:BackgroundImprovements in the quality of health care services are often measured using data present in medical records. Electronic Medical Records (EMRs) contain potentially valuable new sources of health data. However, data quality in EMRs may not be optimal and should be assessed. Data reliability (are the same data elements being measured over time?) is a prerequisite for data validity (are the data accurate?). Our objective was to measure the reliability of data for preventive services in primary care EMRs during the transition to EMR.MethodsOur data sources were randomly selected eligible patients' medical records and data obtained from provincial administrative datasets. Eighteen community-based family physicians in Toronto, Ontario that implemented EMRs starting in 2006 participated in this study. We measured the proportion of patients eligible for a service (Pap smear, screening mammogram or influenza vaccination) that received the service. We compared the change in rates of selected preventive services calculated from the medical record audits with the change in administrative datasets.ResultsIn the first year of EMR use (2006) services decreased by 8.7% more (95% CI -11.0%- - 6.4%, p < 0.0001) when measured through medical record audits as compared with administrative datasets. Services increased by 2.4% more (95% CI 0%-4.9%, p = 0.05) in the medical record audits during the second year of EMR use (2007).ConclusionThere were differences between the change measured through medical record audits and administrative datasets. Problems could include difficulties with organizing new data entry processes as well as continued use of both paper and EMRs. Data extracted from EMRs had limited reliability during the initial phase of EMR implementation. Unreliable data interferes with the ability to measure and improve health care quality.
| S-EPMC3442990 | biostudies-literature
Project description:This study describes an automatic technique to accurately determine the maximum head circumference (MHC) measurement from MRI studies within the Picture Archiving and Communications System, and can automatically add this measurement to the final radiology report. Participants were selected through a retrospective chart review of patients referred to the neurosurgery clinic. Forty-nine pediatric patients with ages ranging from 5 months to 11 years were included in the study. We created 14 printed ring structures to mirror the head circumference values at various ages along the x-axis of the Nellhaus chart. The 3D-printed structures were used to create MRI phantoms. Analytical obtainment of circumference values from the 3D objects and phantom images allowed for a fair estimation and correction of errors on the image-based-measuring instrument. Then, standard manual MHC measurements were performed and compared to values obtained from the patients' MRI T1 images using the tuned instrument proposed in this document. A T-test revealed no statistical difference between the manual assessments and the ones obtained by the automation p = 0.357, α = 0.05. This automatic application augments the more error-prone manual MHC measurement, and can add a numerical value to the final radiology report as a standard application.
| S-EPMC8326831 | biostudies-literature