Dataset Information

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

ABSTRACT: Background:To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods:The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results:The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616-0.906), and the four subscales had fair reliability (ICC: 0.409-0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448-0.897), and the remaining four had slight reliability (ICC: 0.348-0.364). Conclusions:Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.

SUBMITTER: Zhu W

PROVIDER: S-EPMC5695890 | biostudies-literature | 2017 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Zhu Wenfei W Sun Yuliang Y Kurka Jonathan J Geremia Carrie C Engelberg Jessa K JK Cain Kelli K Conway Terry T Sallis James F JF Hooker Steven P SP Adams Marc A MA

Landscape and urban planning 20170714

<h4>Background</h4>To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona.<h4>Methods</h4>The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San D ...[more]

PMID: 29170571

Similar Datasets

Project description:BackgroundStreetscape (microscale) features of the built environment can influence people's perceptions of their neighborhoods' suitability for physical activity. Many microscale audit tools have been developed, but few have published systematic scoring methods. We present the development, scoring, and reliability of the Microscale Audit of Pedestrian Streetscapes (MAPS) tool and its theoretically-based subscales.MethodsMAPS was based on prior instruments and was developed to assess details of streetscapes considered relevant for physical activity. MAPS sections (route, segments, crossings, and cul-de-sacs) were scored by two independent raters for reliability analyses. There were 290 route pairs, 516 segment pairs, 319 crossing pairs, and 53 cul-de-sac pairs in the reliability sample. Individual inter-rater item reliability analyses were computed using Kappa, intra-class correlation coefficient (ICC), and percent agreement. A conceptual framework for subscale creation was developed using theory, expert consensus, and policy relevance. Items were grouped into subscales, and subscales were analyzed for inter-rater reliability at tiered levels of aggregation.ResultsThere were 160 items included in the subscales (out of 201 items total). Of those included in the subscales, 80 items (50.0%) had good/excellent reliability, 41 items (25.6%) had moderate reliability, and 18 items (11.3%) had low reliability, with limited variability in the remaining 21 items (13.1%). Seventeen of the 20 route section subscales, valence (positive/negative) scores, and overall scores (85.0%) demonstrated good/excellent reliability and 3 demonstrated moderate reliability. Of the 16 segment subscales, valence scores, and overall scores, 12 (75.0%) demonstrated good/excellent reliability, three demonstrated moderate reliability, and one demonstrated poor reliability. Of the 8 crossing subscales, valence scores, and overall scores, 6 (75.0%) demonstrated good/excellent reliability, and 2 demonstrated moderate reliability. The cul-de-sac subscale demonstrated good/excellent reliability.ConclusionsMAPS items and subscales predominantly demonstrated moderate to excellent reliability. The subscales and scoring system represent a theoretically based framework for using these complex microscale data and may be applicable to other similar instruments.

Project description:BackgroundFor diagnostic procedures to be clinically useful, they must be reliable. The interpretation of lumbar spine MRI scans is subject to variability and there is a lack of studies where reliability of multiple degenerative pathologies are rated simultaneously. The objective of our study was to determine the inter-rater reliability of three independent raters evaluating degenerative pathologies seen with lumbar spine MRI.MethodsFifty-nine people, 35 patients with low back pain (LBP) or LBP and leg pain and 24 people without LBP or leg pain, received an MRI of the lumbar spine. Three raters (one radiologist and two chiropractors) evaluated the MRIs for the presence and severity of eight degenerative spinal pathologies using a standardized format: Spondylolisthesis, scoliosis, annular fissure, disc degeneration, disc contour, nerve root compromise, spinal stenosis and facet joint degeneration. Findings were identified and classified at disc level according to type and severity. Raters were instructed to evaluate all study sample persons once to assess inter-rater reliability (fully crossed design). Reliability was calculated using Gwet's Agreement Coefficients (AC1 and AC2) and Cohen's Kappa (κ) and Conger's extension of Cohen's. Gwet's probabilistic benchmarking method to the Landis and Koch scale was used. MRI-findings achieving substantial reliability was considered acceptable.ResultsInter-rater reliability for all raters combined, ranged from (Gwet's AC1 or AC2): 0.64-0.99 and according to probabilistic benchmarking to the Landis and Koch scale equivalent to moderate to almost perfect reliability. Overall reliability level for individual pathologies was almost perfect reliability for spondylolisthesis, spinal stenosis, scoliosis and annular fissure, substantial for nerve root compromise and disc degeneration, and moderate for facet joint degeneration and disc contour.ConclusionInter-rater reliability for 3 raters, evaluating 177 disc levels, was found to be overall acceptable for 6 out of 8 degenerative MRI-findings in the lumbar spine. Ratings of facet joint degeneration and disc contour achieved moderate reliability and was considered unacceptable.

Dataset Information

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Publications

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets