Dataset Information

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

ABSTRACT: Background:To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona. Methods:The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San Diego, California, respectively. Item, subscale scores, and total scores reliability analyses were computed using Kappa or intra-class correlation coefficient (ICC). Results:The route overall score had substantial reliability (ICC: 0.832). Of the route subscale and overall scores, sixteen out of twenty had moderate to substantial reliability (ICC: 0.616-0.906), and the four subscales had fair reliability (ICC: 0.409-0.563). Sixteen out of twenty scores in segment and crossing sections demonstrated fair to substantial reliability (ICC: 0.448-0.897), and the remaining four had slight reliability (ICC: 0.348-0.364). Conclusions:Most of the online MAPS items, subscales, and overall scores demonstrated fair to substantial reliability between raters with varied familiarities of the Phoenix area. Results support use of online MAPS to measure microscale elements of the built environment by raters unfamiliar with a region.

SUBMITTER: Zhu W

PROVIDER: S-EPMC5695890 | biostudies-literature | 2017 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Zhu Wenfei W Sun Yuliang Y Kurka Jonathan J Geremia Carrie C Engelberg Jessa K JK Cain Kelli K Conway Terry T Sallis James F JF Hooker Steven P SP Adams Marc A MA

Landscape and urban planning 20170714

<h4>Background</h4>To test inter-rater reliability of the online Microscale Audit of Pedestrian Streetscapes (MAPS) tool between raters with varying familiarities of Phoenix, Arizona.<h4>Methods</h4>The online MAPS tool, based on the MAPS in-field audit tool and scoring system, was used for audits. Sixty route pairs, 141 segment pairs, and 92 crossing pairs in Phoenix were included. Each route, segment or crossing was audited by two independent raters: one rater in Phoenix and the other in San D ...[more]

PMID: 29170571

Similar Datasets

Project description:BackgroundStreetscape (microscale) features of the built environment can influence people's perceptions of their neighborhoods' suitability for physical activity. Many microscale audit tools have been developed, but few have published systematic scoring methods. We present the development, scoring, and reliability of the Microscale Audit of Pedestrian Streetscapes (MAPS) tool and its theoretically-based subscales.MethodsMAPS was based on prior instruments and was developed to assess details of streetscapes considered relevant for physical activity. MAPS sections (route, segments, crossings, and cul-de-sacs) were scored by two independent raters for reliability analyses. There were 290 route pairs, 516 segment pairs, 319 crossing pairs, and 53 cul-de-sac pairs in the reliability sample. Individual inter-rater item reliability analyses were computed using Kappa, intra-class correlation coefficient (ICC), and percent agreement. A conceptual framework for subscale creation was developed using theory, expert consensus, and policy relevance. Items were grouped into subscales, and subscales were analyzed for inter-rater reliability at tiered levels of aggregation.ResultsThere were 160 items included in the subscales (out of 201 items total). Of those included in the subscales, 80 items (50.0%) had good/excellent reliability, 41 items (25.6%) had moderate reliability, and 18 items (11.3%) had low reliability, with limited variability in the remaining 21 items (13.1%). Seventeen of the 20 route section subscales, valence (positive/negative) scores, and overall scores (85.0%) demonstrated good/excellent reliability and 3 demonstrated moderate reliability. Of the 16 segment subscales, valence scores, and overall scores, 12 (75.0%) demonstrated good/excellent reliability, three demonstrated moderate reliability, and one demonstrated poor reliability. Of the 8 crossing subscales, valence scores, and overall scores, 6 (75.0%) demonstrated good/excellent reliability, and 2 demonstrated moderate reliability. The cul-de-sac subscale demonstrated good/excellent reliability.ConclusionsMAPS items and subscales predominantly demonstrated moderate to excellent reliability. The subscales and scoring system represent a theoretically based framework for using these complex microscale data and may be applicable to other similar instruments.

Project description:BackgroundFor diagnostic procedures to be clinically useful, they must be reliable. The interpretation of lumbar spine MRI scans is subject to variability and there is a lack of studies where reliability of multiple degenerative pathologies are rated simultaneously. The objective of our study was to determine the inter-rater reliability of three independent raters evaluating degenerative pathologies seen with lumbar spine MRI.MethodsFifty-nine people, 35 patients with low back pain (LBP) or LBP and leg pain and 24 people without LBP or leg pain, received an MRI of the lumbar spine. Three raters (one radiologist and two chiropractors) evaluated the MRIs for the presence and severity of eight degenerative spinal pathologies using a standardized format: Spondylolisthesis, scoliosis, annular fissure, disc degeneration, disc contour, nerve root compromise, spinal stenosis and facet joint degeneration. Findings were identified and classified at disc level according to type and severity. Raters were instructed to evaluate all study sample persons once to assess inter-rater reliability (fully crossed design). Reliability was calculated using Gwet's Agreement Coefficients (AC1 and AC2) and Cohen's Kappa (?) and Conger's extension of Cohen's. Gwet's probabilistic benchmarking method to the Landis and Koch scale was used. MRI-findings achieving substantial reliability was considered acceptable.ResultsInter-rater reliability for all raters combined, ranged from (Gwet's AC1 or AC2): 0.64-0.99 and according to probabilistic benchmarking to the Landis and Koch scale equivalent to moderate to almost perfect reliability. Overall reliability level for individual pathologies was almost perfect reliability for spondylolisthesis, spinal stenosis, scoliosis and annular fissure, substantial for nerve root compromise and disc degeneration, and moderate for facet joint degeneration and disc contour.ConclusionInter-rater reliability for 3 raters, evaluating 177 disc levels, was found to be overall acceptable for 6 out of 8 degenerative MRI-findings in the lumbar spine. Ratings of facet joint degeneration and disc contour achieved moderate reliability and was considered unacceptable.

Project description:BackgroundDiagrammatic recording of finger joint angles by using two criss-crossed paper strips can be a quick substitute to the standard goniometry. As a preliminary step toward clinical validation of the diagrammatic technique, the current study employed healthy subjects and non-professional raters to explore whether reliability estimates of the diagrammatic goniometry are comparable with those of the standard procedure.MethodsThe study included two procedurally different parts, which were replicated by assigning 24 medical students to act interchangeably as 12 subjects and 12 raters. A larger component of the study was designed to compare goniometers side-by-side in measurement of finger joint angles varying from subject to subject. In the rest of the study, the instruments were compared by parallel evaluations of joint angles similar for all subjects in a situation of simulated change of joint range of motion over time. The subjects used special guides to position the joints of their left ring finger at varying angles of flexion and extension. The obtained diagrams of joint angles were converted to numerical values by computerized measurements. The statistical approaches included calculation of appropriate intraclass correlation coefficients, standard errors of measurements, proportions of measurement differences of 5 or less degrees, and significant differences between paired observations.ResultsReliability estimates were similar for both goniometers. Intra-rater and inter-rater intraclass correlation coefficients ranged from 0.69 to 0.93. The corresponding standard errors of measurements ranged from 2.4 to 4.9 degrees. Repeated measurements of a considerable number of raters fell within clinically non-meaningful 5 degrees of each other in proportions comparable with a criterion value of 0.95. Data collected with both instruments could be similarly interpreted in a simulated situation of change of joint range of motion over time.ConclusionsThe paper goniometer and the standard goniometer can be used interchangeably by non-professional raters for evaluation of normal finger joints. The obtained results warrant further research to assess clinical performance of the paper strip technique.

Dataset Information

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Publications

Reliability between online raters with varying familiarities of a region: Microscale Audit of Pedestrian Streetscapes (MAPS).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure