Dataset Information

Reliability and validity of a nutrition and physical activity environmental self-assessment for child care.

ABSTRACT:

Background

Few assessment instruments have examined the nutrition and physical activity environments in child care, and none are self-administered. Given the emerging focus on child care settings as a target for intervention, a valid and reliable measure of the nutrition and physical activity environment is needed.

Methods

To measure inter-rater reliability, 59 child care center directors and 109 staff completed the self-assessment concurrently, but independently. Three weeks later, a repeat self-assessment was completed by a sub-sample of 38 directors to assess test-retest reliability. To assess criterion validity, a researcher-administered environmental assessment was conducted at 69 centers and was compared to a self-assessment completed by the director. A weighted kappa test statistic and percent agreement were calculated to assess agreement for each question on the self-assessment.

Results

For inter-rater reliability, kappa statistics ranged from 0.20 to 1.00 across all questions. Test-retest reliability of the self-assessment yielded kappa statistics that ranged from 0.07 to 1.00. The inter-quartile kappa statistic ranges for inter-rater and test-retest reliability were 0.45 to 0.63 and 0.27 to 0.45, respectively. When percent agreement was calculated, questions ranged from 52.6% to 100% for inter-rater reliability and 34.3% to 100% for test-retest reliability. Kappa statistics for validity ranged from -0.01 to 0.79, with an inter-quartile range of 0.08 to 0.34. Percent agreement for validity ranged from 12.9% to 93.7%.

Conclusion

This study provides estimates of criterion validity, inter-rater reliability and test-retest reliability for an environmental nutrition and physical activity self-assessment instrument for child care. Results indicate that the self-assessment is a stable and reasonably accurate instrument for use with child care interventions. We therefore recommend the Nutrition and Physical Activity Self-Assessment for Child Care (NAP SACC) instrument to researchers and practitioners interested in conducting healthy weight intervention in child care. However, a more robust, less subjective measure would be more appropriate for researchers seeking an outcome measure to assess intervention impact.

SUBMITTER: Benjamin SE

PROVIDER: S-EPMC1934916 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND:The purpose of this study was to examine the internal consistency, test-retest reliability, construct validity and predictive validity of a new German self-report instrument to assess the influence of social support and the physical environment on physical activity in adolescents. METHODS:Based on theoretical consideration, the short scales on social support and physical environment were developed and cross-validated in two independent study samples of 9 to 17?year-old girls and boys. The longitudinal sample of Study I (n?=?196) was recruited from a German comprehensive school, and subjects in this study completed the questionnaire twice with a between-test interval of seven days. Cronbach's alphas were computed to determine the internal consistency of the factors. Test-retest reliability of the latent factors was assessed using intra-class coefficients. Factorial validity of the scales was assessed using principle components analysis. Construct validity was determined using a cross-validation technique by performing confirmatory factor analysis with the independent nationwide cross-sectional sample of Study II (n?=?430). Correlations between factors and three measures of physical activity (objectively measured moderate-to-vigorous physical activity (MVPA), self-reported habitual MVPA and self-reported recent MVPA) were calculated to determine the predictive validity of the instrument. RESULTS:Construct validity of the social support scale (two factors: parental support and peer support) and the physical environment scale (four factors: convenience, public recreation facilities, safety and private sport providers) was shown. Both scales had moderate test-retest reliability. The factors of the social support scale also had good internal consistency and predictive validity. Internal consistency and predictive validity of the physical environment scale were low to acceptable. CONCLUSIONS:The results of this study indicate moderate to good reliability and construct validity of the social support scale and physical environment scale. Predictive validity was only confirmed for the social support scale but not for the physical environment scale. Hence, it remains unclear if a person's physical environment has a direct or an indirect effect on physical activity behavior or a moderation function.

Project description:BackgroundA questionnaire to assess physical activity related environmental factors in the European population (a 49-item and an 11-item version) was created as part of the framework of the EU-funded project "Instruments for Assessing Levels of PHysical Activity and fitness (ALPHA)". This paper reports on the development and assessment of the questionnaire's test-retest stability, predictive validity, and applicability to European adults.MethodsThe first pilot test was conducted in Belgium, France and the UK. In total 190 adults completed both forms of the ALPHA questionnaire twice with a one-week interval. Physical activity was concurrently measured (i) by administration of the long version of the International Physical Activity Questionnaire (IPAQ) by interview and (ii) by accelerometry (Actigraph device). After adaptations, the second field test took place in Belgium, the UK and Austria; 166 adults completed the adapted questionnaire at two time points, with minimum one-week interval. In both field studies intraclass correlation coefficients (ICC) and proportion of agreement were computed to assess the stability of the two test scores. Predictive validity was examined in the first field test by correlating the results of the questionnaires with physical activity data from accelerometry and long IPAQ-last 7 days.ResultsThe reliability scores of the ALPHA questionnaire were moderate-to good in the first field testing (ICC range 0.66 - 0.86) and good in the second field testing (ICC range 0.71 - 0.87). The proportion of agreement for the ALPHA short increased significantly from the first (range 50 - 83%) to the second field testing (range 85 - 95%). Environmental scales from both versions of the ALPHA questionnaire were significantly associated with self-reported minutes of transport-related walking, and objectively measured low intensity physical activity levels, particularly in women. Both versions were easily administered with an average completion time of six minutes for the 49-item version and less than two minutes for the short version.ConclusionThe ALPHA questionnaire is an instrument to measure environmental perceptions in relation to physical activity. It appears to have good reliability and predictive validity. The questionnaire is now available to other researchers to investigate its usefulness and applicability across Europe.

Project description:BACKGROUND: Evidence suggests that differences exist in physical activity (PA) participation among Culturally and Linguistically Diverse (CALD) children and adolescents. It is possible that these differences could be influenced by variations in measurement technique and instrument reliability. However, culturally sensitive instruments for examining PA behaviour among CALD populations are lacking. This study tested the reliability of the Child and Adolescent Physical Activity and Nutrition Survey (CAPANS-PA) recall questionnaire among a sample of Chinese-Australian youth. METHODS: The psychometric property of the CAPANS-PA questionnaire was examined among a sample of 77 Chinese-Australian youth (aged 11-14 y) who completed the questionnaire twice within 7 days. Test-retest reliability of individual items and scales within the CAPANS-PA questionnaire was determined using Kappa statistics for categorical variables and intraclass correlation coefficients (ICC) for continuous variables. RESULTS: The CAPANS-PA questionnaire demonstrated acceptable test-retest reliability for frequency and duration of time spent in weekly Moderate to Vigorous Physical Activity (MVPA) (ICC ? 0.70) for all participants. Test-retest reliability for time spent in weekly sedentary activities was acceptable for females (ICC = 0.82) and males (ICC = 0.72). CONCLUSIONS: The results suggest the CAPANS-PA questionnaire provides reliable estimates for type, frequency and duration of MVPA participation among Chinese-Australian youth. Further investigation into the reliability of the sedentary items within the CAPANS-PA is required before these items can be used with confidence. This study is novel in that the reliability of instruments among CALD groups nationally and internationally remains sparse and this study contributes to the wider body of available psychometrically tested instruments. In addition, this study is the first to our knowledge to successfully engage and investigate the basic health enhancing behaviours of Chinese-Australian adolescents.

Project description:Early care and education (ECE) settings play an important role in shaping the nutrition and physical activity habits of young children. Increasing research attention is being directed toward family child care homes (FCCHs) specifically. However, existing measures of child care nutrition and physical activity environments are limited in that they have been created for use with center-based programs and require modification for studies involving FCCHs. This paper describes the modification of the Environment and Policy Assessment and Observation (EPAO) for use in FCCHs.The EPAO underwent a through modification process that incorporated an updated format for the data collection instrument, assessment of emerging best practices, tailoring to the FCCH environment, and creation of a new scoring rubric. The new instrument was implemented as part of a larger randomized control trial. To assess inter-rater reliability, observations on 61 different days were performed independently by two data collectors. To assess construct validity, associations between EPAO scores and measures of children's dietary intake (Healthy Eating Index (HEI) score) and physical activity (accelerometer-measured minutes per hour of moderate to vigorous physical activity, MVPA) were examined.The modified EPAO assesses 38 nutrition and 27 physical activity best practices, which can be summarized into 7 nutrition-related and 10 physical activity-related environmental sub- scores as well as overall nutrition and overall physical activity scores. There was generally good agreement between data collectors (ICC > 0.60). Reliability was slightly lower for feeding practices and physical activity education and professional development (ICC = 0.56 and 0.22, respectively). Child HEI was significantly correlated with the overall nutrition score (r = 0.23), foods provided (r = 0.28), beverages provided (r = 0.15), nutrition education and professional development (r = 0.21), and nutrition policy (r = 0.18). Child MVPA was significantly associated with overall time provided for activity (r = 0.18) and outdoor playtime (r = 0.20). There was also an unexpected negative association between child MVPA and screen time (-0.16) and screen time practices (r = -0.21).The EPAO for the FCCH instrument is a useful tool for researchers working with this unique type of ECE setting. It has undergone rigorous development and testing and appears to have good psychometric properties.NCT01814215 , March 15, 2013.

Project description:IntroductionClinicians rely on certain physical examination tests to diagnose and potentially grade ankle sprains and ankle instability. Diagnostic error and inaccurate prognosis may have important repercussions for clinical decision-making and patient outcomes. Therefore, it is important to recognize the diagnostic value of orthopaedic tests through understanding the reliability and validity of these tests.ObjectiveTo systematically review and report evidence on the reliability and validity of orthopaedic tests for the diagnosis of ankle sprains and instability.MethodsPubMed, CINAHL, Scopus, and Cochrane databases were searched from inception to December 2021. In addition, the reference list of included studies, located systematic reviews, and orthopaedic textbooks were searched. All articles reporting reliability or validity of physical examination or orthopaedic tests to diagnose ankle instability or sprains were included. Methodological quality of the reliability and the validity studies was assessed with The Quality Appraisal for Reliability studies checklist and the Quality Assessment of Diagnostic Accuracy Studies-2 respectively. We identified the number of times the orthopaedic test was investigated and the validity and/or reliability of each test.ResultsOverall, sixteen studies were included. Three studies assessed reliability, eight assessed validity, and five evaluated both. Overall, fifteen tests were evaluated, none demonstrated robust reliability and validity scores. The anterolateral talar palpation test reported the highest diagnostic accuracy. Further, the anterior drawer test, the anterolateral talar palpation, the reverse anterior lateral drawer test, and palpation of the anterior talofibular ligament reported the highest sensitivity. The highest specificity was attributed to the anterior drawer test, the anterolateral drawer test, the reverse anterior lateral drawer test, tenderness on palpation of the proximal fibular, and the squeeze test.ConclusionOverall, the diagnostic accuracy, reliability, and validity of physical examination tests for the assessment of ankle instability were limited. Physical examination tests should not be used in isolation, but rather in combination with the clinical history to diagnose an ankle sprain. Preliminary evidence suggests that the overall validity of physical examination for the ankle may be better if conducted five days after the injury rather than within 48 h of injury.

Project description:Measuring physical activity (PA) and sedentary time (ST) in young children (<5 years) is complex. Objective measures have high validity but require specialist expertise, are expensive, and can be burdensome for participants. A proxy-report instrument for young children that accurately measures PA and ST is needed. The aim of this study was to assess the reliability and validity of the Early Years Physical Activity Questionnaire (EY-PAQ). In a setting where English and Urdu are the predominant languages spoken by parents of young children, a sample of 196 parents and their young children (mean age 3.2 ± 0.8 years) from Bradford, UK took part in the study. A total of 156 (79.6%) questionnaires were completed in English and 40 (20.4%) were completed in transliterated Urdu. A total of 109 parents took part in the reliability aspect of the study, which involved completion of the EY-PAQ on two occasions (7.2 days apart; standard deviation (SD) = 1.1). All 196 participants took part in the validity aspect which involved comparison of EY-PAQ scores against accelerometry. Validty anaylsis used all data and data falling with specific MVPA and ST boundaries. Reliability was assessed using intra-class correlations (ICC) and validity by Bland⁻Altman plots and rank correlation coefficients. The test re-test reliability of the EY-PAQ was moderate for ST (ICC = 0.47) and fair for moderate-to-vigorous physical activity (MVPA)(ICC = 0.35). The EY-PAQ had poor agreement with accelerometer-determined ST (mean difference = -87.5 min·day-1) and good agreement for MVPA (mean difference = 7.1 min·day-1) limits of agreement were wide for all variables. The rank correlation coefficient was non-significant for ST (rho = 0.19) and significant for MVPA (rho = 0.30). The EY-PAQ has comparable validity and reliability to other PA self-report tools and is a promising population-based measure of young children's habitual MVPA but not ST. In situations when objective methods are not possible for measurement of young children's MVPA, the EY-PAQ may be a suitable alternative but only if boundaries are applied.