ABSTRACT: Background: Little is known about the role of perceived neighborhood environment as a determinant of physical activity (PA) and sedentary time (ST) in understanding obesity-related health behaviors. We focus on a biracial, socioeconomically diverse population using objectively measured ST, which is under-represented in the literature. Methods: We examined the association between self-reported neighborhood perception (Likert-scale questions), PA using the Baecke questionnaire, and both non-sedentary time and ST using accelerometry from wave 4 of the Healthy Aging in Neighborhoods of Diversity across the Life Span (HANDLS) study (n = 2,167). After applying exclusion criteria, the sample size was n = 1,359 for analyses of self-reported PA and n = 404 for analyses of accelerometry data. Factor analysis identified key neighborhood characteristics to develop a total neighborhood perception score (NPS). Higher NPS indicated less favorable neighborhood perception. Linear regression was used to determine the relationship between NPS, PA, non-sedentary time, and ST. Results: Complete data were available for n = 1,359 [age 56.6(9.0) years, 59.5% female, 62.2% African American] for whom we identified four neighborhood perception factors: (1) concern about crime, (2) physical environment, (3) location of violent crime, and (4) social environment. Worsening perception of the overall neighborhood [? = -0.13 (SE = 0.03); p = 0.001], the physical environment [-0.11 (0.05); p = 0.03], and the social environment [-0.46 (0.07); p < 0.0001] were associated with decreased PA. Worsening perception of the overall neighborhood [1.14 (0.49); p = 0.02] and neighborhood social environment [3.59 (1.18); p = 0.003] were associated with increased ST over the day. There were no interactions for race, sex, poverty status, or economic index. Conclusion: Poor overall neighborhood perception, perceived social environment, and perceived neighborhood physical environment are associated with PA and ST in a multi-racial, socioeconomically diverse cohort of urban adults. Clinical Trial Registration: The HANDLS study is registered at ClinicalTrials.gov as NCT01323322.