Impact of survey length and compensation on validity, reliability, and sample characteristics for Ultrashort-, Short-, and Long-Research Participant Perception Surveys.
Ontology highlight
ABSTRACT: Introduction
The validated Research Participant Perception Survey (RPPS-Long) elicits valuable data at modest response rates.Methods
To address this limitation, we developed shorter RPPS-Ultrashort and RPPS-Short versions, fielded them with the RPPS-Long to a random sample of a national research volunteer registry, and assessed response and completion rates, test/retest reliability, and demographics.Results
2228 eligible registry members received survey links. Response rates were 64% (RPPS-Ultrashort), 63% (RPPS-Short), and 51% (RPPS-Long), respectively (P<0.001). Completion rates were 63%, 54%, and 37%, respectively (p<0.001). All surveys were reliable with Cronbach's alpha = 0.81, 0.84, and 0.87, respectively. Retest reliability was highest for RPPS-Short (Kappa=0.85). Provision of compensation increased RPPS-Short completion rate from 54% to 71% (p<0.001). Compensated respondents were younger (p<0.001), with greater minority representation (p=0.03).Conclusions
Shorter surveys were reliable and produced higher response and completion rates then long surveys. Compensation further increased completion rates and shifted sample age and race profiles.
SUBMITTER: Kost RG
PROVIDER: S-EPMC6208327 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description: Not available
| S-EPMC6068938 | biostudies-literature
Project description:BackgroundThe Subjective Well-Being Under Neuroleptic Treatment Scale short form (SWN-K) is a self-rating scale developed to measure mentally ill patients' well-being under the antipsychotic drug treatment. This paper reports on adaptation and psychometric properties of the instrument in an Estonian psychiatric sample.MethodsIn a naturalistic study design, 124 inpatients or outpatients suffering from the first psychotic episode or chronic psychotic illness completed the translated SWN-K instrument. Item content analysis, internal consistency analysis, exploratory principal components analysis, and confirmatory factor analysis were used to construct the Estonian version of the SWN-K (SWN-K-E). Additionally, socio-demographic and clinical data, observer-rated psychopathology, medication side effects, daily antipsychotic drug dosages, and general functioning were assessed at two time points, at baseline and after a 29-week period; the associations of the SWN-K-E scores with these variables were explored.ResultsAfter having selected 20 items for the Estonian adaptation, the internal consistency of the total SWN-K-E was 0.93 and the subscale consistencies ranged from 0.70 to 0.80. Good test-retest reliabilities were observed for the adapted scale scores, with the correlation of the total score over about 6 months being r = 0.70. Confirmatory factor analysis replicated the presence of a higher-order factor (general well-being) and five first-order factors (mental functioning, physical functioning, social integration, emotional regulation, and self-control); the model fitted the data well. The results indicated a moderate-high correlations r = 0.54 between the SWN-K-E total score and the evaluation how satisfied patients were with their lives in generally. No significant correlations were found between the overall subjective well-being score and age, severity of the psychopathology, drug adverse effects, or prescribed drug dosage.ConclusionTaken together, the results demonstrated that the Estonian version of the SWN-K is a reliable and valid instrument with psychometric properties similar to the original English version. The potential uses of the scale in both research and clinical settings are considered.
| S-EPMC3847444 | biostudies-literature
Project description:BackgroundPreoperative expectations of outcome of total hip and knee arthroplasty are important determinants of patients' satisfaction and functional outcome. Aims of the study were (1) to translate the Hospital for Special Surgery Hip Replacement Expectations Survey and Knee Replacement Expectations Survey into Dutch and (2) to study test-retest reliability and concurrent validity.MethodsPatients scheduled for total hip (N = 112) or knee replacement (N = 101) were sent the Dutch Expectations Surveys twice with a 2 week interval to determine test-retest reliability. To determine concurrent validity, the Expectation WOMAC was sent.ResultsThe results for the Dutch Hip Replacement Expectations Survey revealed good test-retest reliability (ICC 0.87), no bias and good internal consistency (alpha 0.86) (N = 72). The correlation between the Hip Expectations Score and the Expectation WOMAC score was 0.59 (N = 86). The results for the Dutch Knee Replacement Expectations Survey revealed good test-retest reliability (ICC 0.79), no bias and good internal consistency (alpha 0.91) (N = 46). The correlation with the Expectation WOMAC score was 0.52 (N = 57).ConclusionsBoth Dutch Expectations Surveys are reliable instruments to determine patients' expectations before total hip or knee arthroplasty. As for concurrent validity, the correlation between both surveys and the Expectation WOMAC was moderate confirming that the same construct was determined. However, patients scored systematically lower on the Expectation WOMAC compared to the Dutch Expectation Surveys. Research on patients' expectations before total hip and knee replacement has only been performed in a limited amount of countries. With the Dutch Expectations Surveys it is now possible to determine patients' expectations in another culture and healthcare setting.
| S-EPMC2973929 | biostudies-literature
Project description:Moral expansiveness refers to the range of entities (human and non-human) deemed worthy of moral concern and treatment. Previous research has established that the Moral Expansiveness Scale (MES) is a powerful predictor of altruistic moral decision-making and captures a unique dimension of moral cognition. However, the length of the full MES may be restrictive for some researchers. Here we establish the reliability and validity of a reduced moral expansiveness scale, the MESx. Consistent with the full version, the MESx is strongly associated with (but not reducible to) theoretically related constructs, such as endorsement of universalism values, identification with all humanity, and connectedness to nature. The MESx also predicted measures of altruistic moral decision-making to the same degree as the full MES. Further, the MESx passed tests of discriminant validity, was unrelated to political conservatism (unlike the full MES), only mildly associated with the tendency to provide socially desirable responses, and produced moderate reliability over time. We conclude that the MESx is a psychometrically valid alternative for researchers requiring a short measure of moral expansiveness.
| S-EPMC6193647 | biostudies-literature
Project description:OBJECTIVE:The study aim was to test the intra-assessor and interassessor reliability of the Healthcare Complaints Analysis Tool (HCAT) for categorising the information in the claim letters in a sample of Danish patient compensation claims. DESIGN, SETTING AND PARTICIPANTS:We used a random sample of 140 compensation cases completed by the Danish Patient Compensation Association that were filed in the field of acute medicine at Danish hospitals from 2007 to 2018. Four assessors were trained in using the HCAT manual before assessing the claim letters independently. MAIN OUTCOME MEASURES:Intra-assessor and interassessor reliability was tested at domain, problem category and subcategory levels of the HCAT. We also investigated the reliability of ratings on the level of harm and of the descriptive details contained in the claim letters. RESULTS:The HCAT was reliable for identifying problem categories, with reliability scores ranging from 0.55 to 0.99. Reliability was lower when coding the 'severity' of the problem. Interassessor reliability was generally lower than intra-assessor reliability. The categories of 'quality' and 'safety' were the least reliable of the seven HCAT problem categories. Reliability at the subcategory level was generally satisfactory, with only a few subcategories having poor reliability. Reliability was at least moderate when coding the stage of care, the complainant and the staff group involved. However, the coding of 'level of harm' was found to be unreliable (intrareliability 0.06; inter-reliability 0.29). CONCLUSION:Overall, HCAT was found to be a reliable tool for categorising problem types in patient compensation claims.
| S-EPMC6887041 | biostudies-literature
Project description:Fully automated vehicles (AVs) hold promise toward providing numerous societal benefits including reducing road fatalities. However, we are uncertain about how individuals' perceptions will influence their ability to accept and adopt AVs. The 28-item Automated Vehicle User Perception Survey (AVUPS) is a visual analog scale that was previously constructed, with established face and content validity, to assess individuals' perceptions of AVs. In this study, we examined construct validity, via exploratory factor analysis and subsequent Mokken scale analyses. Next, internal consistency was assessed via Cronbach's alpha (α) and 2-week test-retest reliability was assessed via Spearman's rho (ρ) and intraclass correlation coefficient (ICC). The Mokken scale analyses resulted in a refined 20-item AVUPS and three Mokken subscales assessing specific domains of adults' perceptions of AVs: (a) Intention to use; (b) perceived barriers; and (c) well-being. The Mokken scale analysis showed that all item-coefficients of homogeneity (H) exceeded 0.3, indicating that the items reflect a single latent variable. The AVUPS indicated a strong Mokken scale (H scale = 0.51) with excellent internal consistency (α = 0.95) and test-retest reliability (ρ = 0.76, ICC = 0.95). Similarly, the three Mokken subscales ranged from moderate to strong (range H scale = 0.47-0.66) and had excellent internal consistency (range α = 0.84-0.94) and test-retest reliability (range ICC = 0.84-0.93). The AVUPS and three Mokken subscales of AV acceptance were validated in a moderate sample size (N = 312) of adults living in the United States. Two-week test-retest reliability was established using a subset of Amazon Mechanical Turk participants (N = 84). The AVUPS, or any combination of the three subscales, can be used to validly and reliably assess adults' perceptions before and after being exposed to AVs. The AVUPS can be used to quantify adults' acceptance of fully AVs.
| S-EPMC7868437 | biostudies-literature
Project description:Pectoralis minor muscle length is believed to play an important role in shoulder pain and dysfunction. Current clinical procedures for assessing pectoralis minor muscle length may not provide the most useful information for clinical decision making.To establish the reliability and construct validity of a novel technique to measure pectoralis minor muscle length under actively and passively lengthened conditions.Cross-sectional repeated measures.Thirty-four healthy adults (age: 23.9, SD=1.6 years; 18 females) participated in this study. Pectoralis minor muscle length was measured on the dominant arm in three length conditions: resting, actively lengthened, and passively lengthened. Based upon availability, two raters, out of a pool of five, used a caliper to measure the distance between the coracoid process and the 4th rib. The average of two pectoralis minor muscle length measures was used for all muscle length conditions and analyses. Intraclass correlation coefficients determined intra-and inter-rater reliability, and measurement error was determined via standard error of measurement and minimal detectable change. Construct validity was assessed by ANOVA to determine differences in muscle length across the three conditions.Our intra- and inter-rater reliability values across all three conditions ranged from 0.84 to 0.92 and from 0.80 to 0.90, respectively. Significant differences (p<0.001) in muscle length were found among all three conditions: rest-active (3.66; SD=1.36cm), rest-passive (4.72, SD=1.41cm), and active-passive (1.06, SD=0.47cm).The techniques described in this study for measuring pectoralis minor muscle length under resting and actively and passively lengthened conditions have acceptable reliability for clinical decision making.
| S-EPMC5537459 | biostudies-other
Project description:BackgroundThe overuse of antibiotics is becoming an increasing concern. Antibiotic resistance, which increases both the burden of disease, and the cost of health services, is perhaps the most profound impact of antibiotics overuse. Attempts have been made to develop instruments to measure the psychosocial constructs underlying antibiotics use, however, none of these instruments have undergone thorough psychometric validation. This study evaluates the psychometric properties of the Parental Perceptions on Antibiotics (PAPA) scales. The PAPA scales attempt to measure the factors influencing parental use of antibiotics in children.Methods1111 parents of children younger than 12 years old were recruited from primary schools' parental meetings in the Eastern Province of Saudi Arabia from September 2012 to January 2013. The structure of the PAPA instrument was validated using Confirmatory Factor Analysis (CFA) with measurement model fit evaluated using the raw and scaled ?2, Goodness of Fit Index, and Root Mean Square Error of Approximation.ResultsA five-factor model was confirmed with the model showing good fit. Constructs in the model include: Knowledge and Beliefs, Behaviors, Sources of information, Adherence, and Awareness about antibiotics resistance. The instrument was shown to have good internal consistency, and good discriminant and convergent validity.ConclusionThe availability of an instrument able to measure the psychosocial factors underlying antibiotics usage allows the risk factors underlying antibiotic use and overuse to now be investigated.
| S-EPMC3909352 | biostudies-literature
Project description:Families strongly influence the health of communities and individuals across the life course, but no validated measure of family health exists. The absence of such a measure has limited the examination of family health trends and the intersection of family health with individual and community health. The purpose of this study was to examine the reliability and validity of the Family Health Scale (FHS), creating a multi-factor long-form and a uniform short-form. The primary sample included 1,050 adults recruited from a national quota sample Qualtrics panel. Mplus version 7 was used to analyze the data using a structural equation modeling framework. Exploratory factor analysis (EFA) and confirmatory factor analysis (CFA) confirmed a 32-item, 4-factor long-form scale. The four factors included (1) family social and emotional health processes; (2) family healthy lifestyle; (3) family health resources; and (4) family external social supports. A 10-item short-form of the FHS was also validated in the initial sample and a second sample of 401 adults. Both the long-form and short-form FHS correlated in the expected direction with validated measures of family functioning and healthy lifestyle. A preliminary assessment of clinical cutoffs in the short-form were correlated with depression risk. The FHS offers the potential to assess family health trends and to develop accessible, de-identified databases on the well-being of families. Important next steps include validating the scale among multiple family members and collecting longitudinal data.
| S-EPMC7717993 | biostudies-literature
Project description:BackgroundNumerous studies have documented high rates of functional impairment among bipolar disorder (BD) patients, even during phases of remission. However, the majority of the available instruments used to assess functioning have focused on global measures of functional recovery rather than specific domains of psychosocial functioning. In this context, the Functioning Assessment Short Test (FAST) is a brief instrument designed to assess the main functioning problems experienced by psychiatric patients, particularly bipolar patients. It comprises 24 items that assess impairment or disability in six specific areas of functioning: autonomy, occupational functioning, cognitive functioning, financial issues, interpersonal relationships and leisure time.Methods101 patients with DSM-IV TR bipolar disorder and 61 healthy controls were assessed in the Bipolar Disorder Program, Hospital Clinic of Barcelona. The psychometric properties of FAST (feasibility, internal consistency, concurrent validity, discriminant validity (euthymic vs acute patients), factorial analyses, and test-retest reliability) were analysed.ResultsThe internal consistency obtained was very high with a Cronbach's alpha of 0.909. A highly significant negative correlation with GAF was obtained (r = -0.903; p < 0.001) pointing to a reasonable degree of concurrent validity. Test-retest reliability analysis showed a strong correlation between the two measures carried out one week apart (ICC = 0.98; p < 0.001). The total FAST scores were lower in euthymic (18.55 +/- 13.19; F = 35.43; p < 0.001) patients, as compared with manic (40.44 +/- 9.15) and depressive patients (43.21 +/- 13.34).ConclusionThe FAST showed strong psychometrics properties and was able to detect differences between euthymic and acute BD patients. In addition, it is a short (6 minutes) simple interview-administered instrument, which is easy to apply and requires only a short period of time for its application.
| S-EPMC1904447 | biostudies-literature