Dataset Information

Online versus in-person comparison of Microscale Audit of Pedestrian Streetscapes (MAPS) assessments: reliability of alternate methods.

ABSTRACT: An online version of the Microscale Audit of Pedestrian Streetscapes (Abbreviated) tool was adapted to virtually audit built environment features supportive of physical activity. The current study assessed inter-rater reliability of MAPS Online between in-person raters and online raters unfamiliar with the regions.In-person and online audits were conducted for a total of 120 quarter-mile routes (60 per site) in Phoenix, AZ and San Diego, CA. Routes in each city included 40 residential origins stratified by walkability and SES, and 20 commercial centers. In-person audits were conducted by raters residing in their region. Online audits were conducted by raters in the alternate location using Google Maps (Aerial and Street View) images. The MAPS Abbreviated Online tool consisted of four sections: overall route, street segments, crossings and cul-de-sacs. Items within each section were grouped into subscales, and inter-rater reliability (ICCs) was assessed for subscales at multiple levels of aggregation.Online and in-person audits showed excellent agreement for overall positive microscale (ICC = 0.86, 95% CI [0.80, 0.90]) and grand scores (ICC = 0.93, 95% CI [0.89, 0.95]). Substantial to near-perfect agreement was found for 21 of 30 (70%) subscales, valence, and subsection scores, with ICCs ranging from 0.62, 95% CI [0.50, 0.72] to 0.95, 95% CI [0.93, 0.97]. Lowest agreement was found for the aesthetics and social characteristics scores, with ICCs ranging from 0.07, 95% CI [-0.12, 0.24] to 0.27, 95% CI [0.10, 0.43].Results support use of the MAPS Abbreviated Online tool to reliably assess microscale neighborhood features that support physical activity and may be used by raters residing in different geographic regions and unfamiliar with the audit areas.

SUBMITTER: Phillips CB

PROVIDER: S-EPMC5545045 | biostudies-other | 2017 Aug

REPOSITORIES: biostudies-other

ACCESS DATA

Similar Datasets

Project description:BackgroundStreetscape (microscale) features of the built environment can influence people's perceptions of their neighborhoods' suitability for physical activity. Many microscale audit tools have been developed, but few have published systematic scoring methods. We present the development, scoring, and reliability of the Microscale Audit of Pedestrian Streetscapes (MAPS) tool and its theoretically-based subscales.MethodsMAPS was based on prior instruments and was developed to assess details of streetscapes considered relevant for physical activity. MAPS sections (route, segments, crossings, and cul-de-sacs) were scored by two independent raters for reliability analyses. There were 290 route pairs, 516 segment pairs, 319 crossing pairs, and 53 cul-de-sac pairs in the reliability sample. Individual inter-rater item reliability analyses were computed using Kappa, intra-class correlation coefficient (ICC), and percent agreement. A conceptual framework for subscale creation was developed using theory, expert consensus, and policy relevance. Items were grouped into subscales, and subscales were analyzed for inter-rater reliability at tiered levels of aggregation.ResultsThere were 160 items included in the subscales (out of 201 items total). Of those included in the subscales, 80 items (50.0%) had good/excellent reliability, 41 items (25.6%) had moderate reliability, and 18 items (11.3%) had low reliability, with limited variability in the remaining 21 items (13.1%). Seventeen of the 20 route section subscales, valence (positive/negative) scores, and overall scores (85.0%) demonstrated good/excellent reliability and 3 demonstrated moderate reliability. Of the 16 segment subscales, valence scores, and overall scores, 12 (75.0%) demonstrated good/excellent reliability, three demonstrated moderate reliability, and one demonstrated poor reliability. Of the 8 crossing subscales, valence scores, and overall scores, 6 (75.0%) demonstrated good/excellent reliability, and 2 demonstrated moderate reliability. The cul-de-sac subscale demonstrated good/excellent reliability.ConclusionsMAPS items and subscales predominantly demonstrated moderate to excellent reliability. The subscales and scoring system represent a theoretically based framework for using these complex microscale data and may be applicable to other similar instruments.

Project description:BackgroundEarly evidence in using online cognitive assessments show that they could offer a feasible and resource-efficient alternative to in-person clinical assessments in evaluating cognitive performance, yet there is currently little understanding about how these assessments relate to traditional, in-person cognitive tests.ObjectivesIn this preliminary study, we assess the feasibility and reliability of NeurOn, a novel online cognitive assessment tool. NeurOn measures various cognitive domains including processing speed, executive functioning, spatial working memory, episodic memory, attentional control, visuospatial functioning, and spatial orientation.DesignThirty-two participants (mean age: 70.19) completed two testing sessions, unsupervised online and in-person, one-week apart. Participants were randomised in the order of testing appointments. For both sessions, participants completed questionnaires prior to a cognitive assessment. Test-retest reliability and concurrent validity of the online cognitive battery was assessed using intraclass correlation coefficients (ICCs) and correlational analysis, respectively. This was conducted by comparing performance in repeated tasks across testing sessions as well as with traditional, in-person cognitive tests.ResultsGlobal cognition in the NeurOn battery moderately validated against MoCA performance, and the battery demonstrated moderate test-retest reliability. Concurrent validity was found only between the online and paper versions of the Trail Making Test -A, as well as global cognitive performance between online and in-person testing sessions.ConclusionsThe NeurOn cognitive battery provides a promising tool for measuring cognitive performance online both longitudinally and across short retesting intervals within healthy older adults. When considering cost-effectiveness, flexible administration, and improved accessibility for wider populations, online cognitive assessments show promise for future screening of neurodegenerative diseases.

Project description:BackgroundDespite the central role of mixed methods in health research, studies evaluating online methods training in the health sciences are nonexistent. The focused goal was to evaluate online training by comparing the self-rated skills of scholars who experienced an in-person retreat to scholars in an online retreat in specific domains of mixed methods research for the health sciences from 2015-2023.MethodsThe authors administered a scholar Mixed Methods Skills Self-Assessment instrument based on an educational competency scale that included domains on: "research questions," "design/approach," "sampling," "analysis," and "dissemination" to participants of the Mixed Methods Research Training Program for the Health Sciences (MMRTP). Self-ratings on confidence on domains were compared before and after retreat participation within cohorts who attended in person (n = 73) or online (n = 57) as well as comparing across in-person to online cohorts. Responses to open-ended questions about experiences with the retreat were analyzed.ResultsScholars in an interactive program to improve mixed methods skills reported significantly increased confidence in ability to define or explain concepts and in ability to apply the concepts to practical problems, whether the program was attended in-person or synchronously online. Scholars in the online retreat had self-rated skill improvements as good or better than scholars who participated in person. With the possible exception of networking, scholars found the online format was associated with advantages such as accessibility and reduced burden of travel and finding childcare. No differences in difficulty of learning concepts was described.ConclusionsKeeping in mind that the retreat is only one component of the MMRTP, this study provides evidence that mixed methods training online was associated with the same increases in self-rated skills as persons attending online and can be a key component to increasing the capacity for mixed methods research in the health sciences.

Project description:BACKGROUND:Improving the neighborhood environment may help address chronic disease and mortality. To identify neighborhood features that are predictors of health, objective assessments of the environment are used. Multiple studies have reported on cross-sectional assessments of health-related neighborhood features using direct observation. As study designs expand to better understand causation and predictors of change, there is a need to test whether direct observation methods are adequate for longitudinal assessment. To our knowledge, this is the first study to report on the reliability of repeated measurements of the neighborhood environment, and their stability, over time. METHODS:The Pittsburgh Hill/Homewood Research on Neighborhood Change and Health (PHRESH) study conducted longitudinal assessments in two low-income, African American neighborhoods at three waves (years 2012, 2015, 2017). The PHRESH audit tool is a modification of earlier validated tools, with an emphasis on environment features relevant for physical activity, sleep, and obesogenic behaviors. Trained data-collector pairs conducted direct observations of a 25% sample of street segments in each neighborhood. At each wave, we audited a sub-sample of street segments twice and assessed reliability using percentage inter-observer agreement and krippendorf's alpha statistics. Stability of these items was assessed as exhibiting moderate or high agreement at every time point. RESULTS:Across waves, a majority (81%) of the items consistently demonstrated moderate to high agreement except for items such as public/communal space, amount of shade, sidewalk features, number of traffic lanes, garden/flower bed/planter, art/statue/monument, amount of trash, and physical disorder. The list of items with poor agreement includes features that are easy to miss (e.g. flower bed/planter), hard to assess from outside (e.g. public/communal space), or may change quickly (e.g. amount of trash). CONCLUSION:In this paper, we have described implementation methods, reliability results and lessons learned to inform future studies of change. We found the use of consistent methods allowed us to conduct reliable, replicable longitudinal assessments of the environment. Items that did not exhibit stability are less useful for detecting real change over time. Overall, the PHRESH direct observation tool is an effective and practical instrument to detect change in the neighborhood environment.

Dataset Information

Online versus in-person comparison of Microscale Audit of Pedestrian Streetscapes (MAPS) assessments: reliability of alternate methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets