Dataset Information

A systematic review of school health policy measurement tools: implementation determinants and outcomes.

ABSTRACT:

Background

Governments in some countries or states/provinces mandate school-based policies intended to improve the health and well-being of primary and secondary students and in some cases the health of school staff. Examples include mandating a minimum time spent per week in programmed physical activity, mandating provision of healthy foods and limiting fat content of school meals, and banning tobacco products or use on school campuses. Although school health researchers have studied whether schools, districts, or states/provinces are meeting requirements, it is unclear to what extent implementation processes and determinants are assessed. The purposes of the present systematic review of quantitative measures of school policy implementation were to (1) identify quantitative school health policy measurement tools developed to measure implementation at the school, district, or state/provincial levels; (2) describe the policy implementation outcomes and determinants assessed and identify the trends in measurement; and (3) assess pragmatic and psychometric properties of identified implementation measures to understand their quality and suitability for broader application.

Methods

Peer-reviewed journal articles published 1995-2020 were included if they (1) had multiple-item quantitative measures of school policy implementation and (2) addressed overall wellness, tobacco, physical activity, nutrition, obesity prevention, or mental health/bullying/social-emotional learning. The final sample comprised 86 measurement tools from 67 peer-review articles. We extracted study characteristics, such as psychometric and pragmatic measure properties, from included articles based on three frameworks: (1) Implementation Outcomes Framework, (2) Consolidated Framework for Implementation Research, and (3) Policy Implementation Determinants Framework.

Results

Most implementation tools were developed to measure overall wellness policies which combined multiple policy topics (n = 35, 40%) and were in survey form (n = 75, 87%). Fidelity was the most frequently prevalent implementation outcome (n = 70, 81%), followed by adoption (n = 32, 81%). The implementation determinants most assessed were readiness for implementation, including resources (n = 43, 50%), leadership (n = 42, 49%), and policy communication (n = 41, 48%). Overall, measures were low-cost and had easy readability. However, lengthy tools and lack of reported validity/reliability data indicate low transferability.

Conclusions

Implementation science can contribute to more complete and rigorous assessment of school health policy implementation processes, which can improve implementation strategies and ultimately the intended health benefits. Several high-quality measures of implementation determinants and implementation outcomes can be applied to school health policy implementation assessment. Dissemination and implementation science researchers can also benefit from measurement experiences of school health researchers.

SUBMITTER: McLoughlin GM

PROVIDER: S-EPMC8235584 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPublic policy has tremendous impacts on population health. While policy development has been extensively studied, policy implementation research is newer and relies largely on qualitative methods. Quantitative measures are needed to disentangle differential impacts of policy implementation determinants (i.e., barriers and facilitators) and outcomes to ensure intended benefits are realized. Implementation outcomes include acceptability, adoption, appropriateness, compliance/fidelity, feasibility, penetration, sustainability, and costs. This systematic review identified quantitative measures that are used to assess health policy implementation determinants and outcomes and evaluated the quality of these measures.MethodsThree frameworks guided the review: Implementation Outcomes Framework (Proctor et al.), Consolidated Framework for Implementation Research (Damschroder et al.), and Policy Implementation Determinants Framework (Bullock et al.). Six databases were searched: Medline, CINAHL Plus, PsycInfo, PAIS, ERIC, and Worldwide Political. Searches were limited to English language, peer-reviewed journal articles published January 1995 to April 2019. Search terms addressed four levels: health, public policy, implementation, and measurement. Empirical studies of public policies addressing physical or behavioral health with quantitative self-report or archival measures of policy implementation with at least two items assessing implementation outcomes or determinants were included. Consensus scoring of the Psychometric and Pragmatic Evidence Rating Scale assessed the quality of measures.ResultsDatabase searches yielded 8417 non-duplicate studies, with 870 (10.3%) undergoing full-text screening, yielding 66 studies. From the included studies, 70 unique measures were identified to quantitatively assess implementation outcomes and/or determinants. Acceptability, feasibility, appropriateness, and compliance were the most commonly measured implementation outcomes. Common determinants in the identified measures were organizational culture, implementation climate, and readiness for implementation, each aspects of the internal setting. Pragmatic quality ranged from adequate to good, with most measures freely available, brief, and at high school reading level. Few psychometric properties were reported.ConclusionsWell-tested quantitative measures of implementation internal settings were under-utilized in policy studies. Further development and testing of external context measures are warranted. This review is intended to stimulate measure development and high-quality assessment of health policy implementation outcomes and determinants to help practitioners and researchers spread evidence-informed policies to improve population health.RegistrationNot registered.

Project description:Background:This systematic review aimed to investigate the psychometric properties of the school health's assessment tools in primary schools through COSMIN Risk of Bias checklist. We examined the studies that have addressed the measurement properties of school-health instruments to give a clear overview of the quality of all available tools measuring school health in primary schools. This systematic review was registered in PROPERO with the Registration ID: CRD42020158158. Method:Databases of EBSCOhost, PubMed, ProQuest, Wily, PROSPERO, and OpenGrey were systematically searched without any time limitation to find all full-text English journal articles studied at least one of the COSMIN checklist measurement properties of a school-health assessment tool in primary schools. The instruments should be constructed based on a school health model. The eligible studies were assessed by COSMIN Risk of Bias checklist to report their quality of methodology for each measurement property and for the whole study by rating high, moderate or low quality. Results:At the final screening just seven studies remained for review. Four studies were tool development, three of them were rated as "adequate" and the other study as "very good"; five studies examined the content validity, three of them were appraised as "very good", and the two remaining as "inadequate". All seven studies measured structural validity, three of them were evaluated as "very good", three other were scored as "adequate", and the last study as "inadequate". All the seven studies investigated the internal consistency, five of them were assessed as "very good", one was rated as "doubtful", and the last one as "inadequate". Just one study examined the cross-cultural validity and was rated as "adequate". Finally, all seven studies measured reliability, two of them were rated as "very good" and the rest five studies were appraised as "doubtful". All rating was based on COSMIN checklist criteria for quality of measurement properties assessment. Conclusion:The number of studies addressing school health assessment tools was very low and therefore not sufficient. Hence, there is a serious need to investigate the psychometric properties of the available instruments measuring school health at primary schools. Moreover, the studies included in the present systematic review did not fulfill all the criteria of the COSMIN checklist for assessing measurement properties. We suggest that future studies consider these criteria for measuring psychometric properties and developing school health assessment tools.

Project description:BackgroundDespite the well-established health benefits of physical activity (PA) for young people (aged 4-19 years), most do not meet PA guidelines. Policies that support PA in schools may be promising, but their impact on PA behavior is poorly understood. The aim of this systematic review was to ascertain the level and type of evidence reported in the international scientific literature for policies within the school setting that contribute directly or indirectly to increasing PA.MethodsThis systematic review is compliant with Preferred Reporting Items for Systematic Review and Meta-Analysis guidelines. Six databases were searched using key concepts of policy, school, evaluation, and PA. Following title and abstract screening of 2323 studies, 25 progressed to data synthesis. Methodological quality was assessed using standardized tools, and the strength of the evidence of policy impact was described based on pre-determined codes: positive, negative, inconclusive, or untested statistically.ResultsEvidence emerged for 9 policy areas that had a direct or indirect effect on PA within the school setting. These were whole school PA policy, physical education, sport/extracurricular PA, classroom-based PA, active breaks/recess, physical environment, shared use agreements, active school transport, and surveillance. The bulk of the evidence was significantly positive (54%), 27% was inconclusive, 9% was significantly negative, and 11% was untested (due to rounding, some numbers add to 99% or 101%). Frequency of evidence was highest in the primary setting (41%), 34% in the secondary setting, and 24% in primary/secondary combined school settings. By policy area, frequency of evidence was highest for sport/extracurricular PA (35%), 17% for physical education, and 12% for whole school PA policy, with evidence for shared use agreements between schools and local communities rarely reported (2%). Comparing relative strength of evidence, the evidence for shared use agreements, though sparse, was 100% positive, while 60% of the evidence for whole school PA policy, 59% of the evidence for sport/extracurricular PA, 57% of the evidence for physical education, 50% of the evidence for PA in classroom, and 50% of the evidence for active breaks/recess were positive.ConclusionThe current evidence base supports the effectiveness of PA policy actions within the school setting but cautions against a "one-size-fits-all" approach and emphasizes the need to examine policy implementation to maximize translation into practice. Greater clarity regarding terminology, measurement, and methods for evaluation of policy interventions is needed.

Project description:BackgroundThe coronavirus pandemic (COVID-19) has also emerged as an infodemic, thereby worsening the harm of the pandemic. This situation has highlighted the need for a deeply rooted understanding of the health information-seeking behaviors (HISBs) of people.ObjectiveThe aim of this paper was to review and provide insight regarding methodologies and the construct of content in HISB surveys by answering the following research question: what are the characteristics of the measurement tools for assessing HISBs in nationally representative surveys around the world?MethodsThe Preferred Reporting Items for Systematic Reviews and Meta-Analyses was used as the framework for this study. A data search was performed through 5 international and 2 Korean databases covering the years between 2008 and 2020. Initially, studies performed among nationally representative samples were included to discover HISB survey instruments. The methodologies of the studies using HISB surveys were analyzed. For content analysis, 2 researchers reached a consensus through discussion by scrutinizing the contents of each survey questionnaire.ResultsA total of 13 survey tools from 8 countries were identified after a review of 2333 records from the search results. Five survey tools (Health Information National Trends Survey, Health Tracking Survey, Annenberg National Health Communication Survey, National Health Interview Survey, and Health Tracking Household Survey) from the United States, 2 instruments from Germany, and 1 tool from each of the countries of the European Union, France, Israel, Poland, South Korea, and Taiwan were identified. Telephone or web-based surveys were commonly used targeting the adult population (≥15 years of age). From the content analysis, the domains of the survey items were categorized as follows: information (information about health and patient medical records), channel (offline and online), and health (overall health, lifestyle, and cancer). All categories encompassed behavioral and attitude dimensions. A theoretical framework, that is, an information-channel-health structure for HISBs was proposed.ConclusionsThe results of our study can contribute to the development and implementation of the survey tools for HISB with integrated questionnaire items. This will help in understanding HISB trends in national health care.

Project description:Background: Sleep of pre-school aged children is important for their health and development, but there are currently no standards for measuring sleep in this age group. We aimed to examine the validity, reliability and feasibility of tools used to assess sleep of pre-school aged children. Methods: Studies were eligible for inclusion if they examined the validity and/or reliability and/or feasibility of a measurement tool used to examine sleep of pre-school aged children (aged 3-7 years). We systematically searched six electronic databases, grey literature and trial registries. We manually searched topic specific journals, reference and citations of included studies, and reference lists of existing reviews. We extracted data and conducted a risk of bias assessment on the included studies using the COnsensus-based Standards for the selection of health Measurement INstruments (COSMIN) risk of bias checklist. We used a narrative synthesis to present the results. Results: Sixteen studies met the inclusion criteria: these explored accelerometers (n = 3) and parental reported tools (n = 13; nine questionnaires, six diaries). Studies assessed construct validity (n = 3), criterion validity (n = 1), convergent validity (n = 13), test-retest reliability (n = 2), internal consistency (n = 4) and feasibility (n = 12). Most studies assessed the convergent validity of questionnaires and diaries compared with accelerometers, but the validity of accelerometers for sleep in this age group is unknown. Of studies with a low risk of bias, one sleep diary was shown to be valid for measuring sleep duration. No measurement tools were appropriate for determining sleep quality. Reporting of reliability and feasibility was minimal. Discussion: The evidence base in this field is limited, and most studies had high risk of bias. Future research on sleep in pre-school aged children should focus on assessing the validity, reliability and feasibility of accelerometers, which in turn will improve the quality of studies that assess questionnaires and diaries against accelerometers. Systematic Review Registration: https://www.crd.york.ac.uk/prospero/display_record.php?ID=CRD42021230900; PROSPERO: CRD42021230900.

Project description:Global initiatives have raised awareness of the need for cross-departmental and cross-sectoral activities to support urban health, sustainability, and equity, with respective indicators routinely used as a way to catalyze and monitor action toward pre-defined goals. Despite the existence of at least 145 urban health indicator (UHI) tools globally, there has been very little research on the use of indicators by policy- and decision-makers; more attention has been devoted to their development and validation. This paper describes the second part of a two-part systematic review of the characteristics (part A) and use (part B, this part) of UHI tools by municipal built environment policy- and decision-makers. Part B is a narrative synthesis of studies on the use of UHI tools. This PRISMA-P compliant review follows a mixed methods sequential explanatory design. The search was conducted using seven bibliographic databases, grey literature searches, and key journal hand searches. Ten studies describing the use of ten UHI tools in seven countries were included in the narrative synthesis, resulting in development of a theory of change (ToC). We found that both expert-led and participatory indicator projects can be underpinned by research evidence and residents' knowledge. Our findings contradict the dominant view of indicator use in policy-making as a linear process, highlighting a number of technical, organizational, political, knowledge, and contextual factors that affect their use. Participatory UHI tools with community involvement were generally more effective at supporting "health in all policies" and "whole-of-society" approaches to governing healthy cities than expert-led processes. UHI tool producers proposed a range of techniques to address urban health complexity characteristics. Finally, in combining data from both parts of the review, we found that potentially important UHI tool features, such as neighbourhood-scale data, were influential in the use of indicators by built environment policy- and decision-makers.