Dataset Information

Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning.

ABSTRACT: With all the recent attention focused on big data, it is easy to overlook that basic vital statistics remain difficult to obtain in most of the world. What makes this frustrating is that private companies hold potentially useful data, but it is not accessible by the people who can use it to track poverty, reduce disease, or build urban infrastructure. This project set out to test whether we can transform an openly available dataset (Twitter) into a resource for urban planning and development. We test our hypothesis by creating road traffic crash location data, which is scarce in most resource-poor environments but essential for addressing the number one cause of mortality for children over five and young adults. The research project scraped 874,588 traffic related tweets in Nairobi, Kenya, applied a machine learning model to capture the occurrence of a crash, and developed an improved geoparsing algorithm to identify its location. We geolocate 32,991 crash reports in Twitter for 2012-2020 and cluster them into 22,872 unique crashes during this period. For a subset of crashes reported on Twitter, a motorcycle delivery service was dispatched in real-time to verify the crash and its location; the results show 92% accuracy. To our knowledge this is the first geolocated dataset of crashes for the city and allowed us to produce the first crash map for Nairobi. Using a spatial clustering algorithm, we are able to locate portions of the road network (<1%) where 50% of the crashes identified occurred. Even with limitations in the representativeness of the data, the results can provide urban planners with useful information that can be used to target road safety improvements where resources are limited. The work shows how twitter data might be used to create other types of essential data for urban planning in resource poor environments.

SUBMITTER: Milusheva S

PROVIDER: S-EPMC7857609 | biostudies-literature | 2021

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning.

Milusheva Sveta S Marty Robert R Bedoya Guadalupe G Williams Sarah S Resor Elizabeth E Legovini Arianna A

PloS one 20210203 2

With all the recent attention focused on big data, it is easy to overlook that basic vital statistics remain difficult to obtain in most of the world. What makes this frustrating is that private companies hold potentially useful data, but it is not accessible by the people who can use it to track poverty, reduce disease, or build urban infrastructure. This project set out to test whether we can transform an openly available dataset (Twitter) into a resource for urban planning and development. We ...[more]

PMID: 33534801

Similar Datasets

Project description:ImportanceAutomatic curation of consumer-generated, opioid-related social media big data may enable real-time monitoring of the opioid epidemic in the United States.ObjectiveTo develop and validate an automatic text-processing pipeline for geospatial and temporal analysis of opioid-mentioning social media chatter.Design, setting, and participantsThis cross-sectional, population-based study was conducted from December 1, 2017, to August 31, 2019, and used more than 3 years of publicly available social media posts on Twitter, dated from January 1, 2012, to October 31, 2015, that were geolocated in Pennsylvania. Opioid-mentioning tweets were extracted using prescription and illicit opioid names, including street names and misspellings. Social media posts (tweets) (n = 9006) were manually categorized into 4 classes, and training and evaluation of several machine learning algorithms were performed. Temporal and geospatial patterns were analyzed with the best-performing classifier on unlabeled data.Main outcomes and measuresPearson and Spearman correlations of county- and substate-level abuse-indicating tweet rates with opioid overdose death rates from the Centers for Disease Control and Prevention WONDER database and with 4 metrics from the National Survey on Drug Use and Health for 3 years were calculated. Classifier performances were measured through microaveraged F1 scores (harmonic mean of precision and recall) or accuracies and 95% CIs.ResultsA total of 9006 social media posts were annotated, of which 1748 (19.4%) were related to abuse, 2001 (22.2%) were related to information, 4830 (53.6%) were unrelated, and 427 (4.7%) were not in the English language. Yearly rates of abuse-indicating social media post showed statistically significant correlation with county-level opioid-related overdose death rates (n = 75) for 3 years (Pearson r = 0.451, P < .001; Spearman r = 0.331, P = .004). Abuse-indicating tweet rates showed consistent correlations with 4 NSDUH metrics (n = 13) associated with nonmedical prescription opioid use (Pearson r = 0.683, P = .01; Spearman r = 0.346, P = .25), illicit drug use (Pearson r = 0.850, P < .001; Spearman r = 0.341, P = .25), illicit drug dependence (Pearson r = 0.937, P < .001; Spearman r = 0.495, P = .09), and illicit drug dependence or abuse (Pearson r = 0.935, P < .001; Spearman r = 0.401, P = .17) over the same 3-year period, although the tests lacked power to demonstrate statistical significance. A classification approach involving an ensemble of classifiers produced the best performance in accuracy or microaveraged F1 score (0.726; 95% CI, 0.708-0.743).Conclusions and relevanceThe correlations obtained in this study suggest that a social media-based approach reliant on supervised machine learning may be suitable for geolocation-centric monitoring of the US opioid epidemic in near real time.

Project description:BackgroundResearch has found that human associated microbial communities play a role in homeostasis and the disruption of these communities may be important in an array of medical conditions. However outside of the human body many of these communities remain poorly studied. The Metagenomics and Metadesign of the Subways and Urban Biomes (MetaSUB) International Consortium is characterizing the microbiomes of urban environments with the aim to improve design of mass transit systems. As part of the CAMDA 2018 MetaSUB Forensics Challenge 311 city microbiome samples were provided to create urban microbial fingerprints, as well as a further 3 mystery datasets for validation.ResultsMetaSUB samples were clustered using t-SNE in an unsupervised fashion to almost discrete groups, which upon inspection represented city of origin. Based on this clustering, geographically close metropolitan areas appear to display similar microbial profiles such as those of Auckland and Hamilton. Mystery unlabeled samples were provided part of the challenge. A random forest classifier built on the initial dataset of 311 samples was capable of correctly classifying 83.3% of the mystery samples to their city of origin. Random Forest analyses also identified features with the highest discriminatory power, ranking bacterial species such as Campylobacter jejuni and Staphylococcus argenteus as highly predictive of city of origin. The surface from which the sample was collected displayed little detectable impact on the microbial profiles in the data generated here. The proportion of reads classified per sample varied greatly and so de-novo assembly was applied to recover genomic fragments representing organisms not captured in reference databases.ConclusionsCurrent methods can differentiate urban microbiome profiles from each other with relative ease. De-novo assembly indicated that the MetaSUB metagenomic data contains adequate depth to recover metagenomic assembled genomes and that current databases are not sufficient to fully characterize urban microbiomes. Profiles found here indicate there may be a relationship between geographical distance between areas and the urban microbiome composition although this will need further research. The impact of these different profiles on public health is currently unknown but the MetaSUB consortium is uniquely suited to evaluate these and provide a roadmap for the inclusion of urban microbiome information for city planning and public health policy.ReviewersThis article was reviewed by Dimitar Vassilev, Eran Elhaik and Chengsheng Zhu.

Dataset Information

Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning.

Publications

Applying machine learning and geolocation techniques to social media data (Twitter) to develop a resource for urban planning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets