Dataset Information

Real-time community detection in full social networks on a laptop.

ABSTRACT: For a broad range of research and practical applications it is important to understand the allegiances, communities and structure of key players in society. One promising direction towards extracting this information is to exploit the rich relational data in digital social networks (the social graph). As global social networks (e.g., Facebook and Twitter) are very large, most approaches make use of distributed computing systems for this purpose. Distributing graph processing requires solving many difficult engineering problems, which has lead some researchers to look at single-machine solutions that are faster and easier to maintain. In this article, we present an approach for analyzing full social networks on a standard laptop, allowing for interactive exploration of the communities in the locality of a set of user specified query vertices. The key idea is that the aggregate actions of large numbers of users can be compressed into a data structure that encapsulates the edge weights between vertices in a derived graph. Local communities can be constructed by selecting vertices that are connected to the query vertices with high edge weights in the derived graph. This compression is robust to noise and allows for interactive queries of local communities in real-time, which we define to be less than the average human reaction time of 0.25s. We achieve single-machine real-time performance by compressing the neighborhood of each vertex using minhash signatures and facilitate rapid queries through Locality Sensitive Hashing. These techniques reduce query times from hours using industrial desktop machines operating on the full graph to milliseconds on standard laptops. Our method allows exploration of strongly associated regions (i.e., communities) of large graphs in real-time on a laptop. It has been deployed in software that is actively used by social network analysts and offers another channel for media owners to monetize their data, helping them to continue to provide free services that are valued by billions of people globally.

SUBMITTER: Chamberlain BP

PROVIDER: S-EPMC5771570 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Real-time community detection in full social networks on a laptop.

Chamberlain Benjamin Paul BP Levy-Kramer Josh J Humby Clive C Deisenroth Marc Peter MP

PloS one 20180117 1

For a broad range of research and practical applications it is important to understand the allegiances, communities and structure of key players in society. One promising direction towards extracting this information is to exploit the rich relational data in digital social networks (the social graph). As global social networks (e.g., Facebook and Twitter) are very large, most approaches make use of distributed computing systems for this purpose. Distributing graph processing requires solving man ...[more]

PMID: 29342158

Similar Datasets

Project description:Note: Data Use Restrictions: <ul> <li>The data must only be used for genetic research projects solely in the area of health-related social-network research.</li> </ul> This study compiles detailed information on family and social ties linked to participants in the offspring cohort of the Framingham Heart Study. The investigators took computerized information from self-report data used by the FHS for over 30 years to facilitate health exam scheduling. These tracking sheets asked participants to identify people close to them, which were assumed by the investigators to indicate a social tie. Individuals are listed as "egos" (if the ties are from their perspective) and/or "alters" (if they are linked as a friend or family members to one of the other participants). All egos were FHS offspring participants while alters could be FHS participants in any cohort or non-participants. This information was combined with internal FHS pedigree data on family ties to list family member alters (as well as the nature of their relationship). The resultant dataset therefore includes each ego-alter tie, the nature of their relationship and the start and end dates for their ties for each of seven waves for the offspring study. Due to privacy concerns, all exam dates for individuals are listed relative to the initial FHS exam date. The study investigators chose a random, confidential date, to which all initial exam dates are linked (by number of months from random date to initial exam date). With this information, researchers can have access to the temporal relationships among participants' exam dates while remaining unaware of the actual exam dates for individual participants. During each clinic exam cycle, the participants undergo a detailed examination including physical examination, medical history, laboratory testing, and electrocardiogram. Over the years, other tests (that may not be performed at every exam cycle) have included pulmonary function, lifestyle, physical function, cognitive function questionnaires, and various noninvasive cardiovascular tests including echocardiograms. This study involves phenotypic data from these exams including (when available) basic body measurements (height, weight, blood pressure), laboratory values (blood sugar levels, LDL levels), information on smoking and alcohol use, and tests of depression and cognitive functioning. Important links to apply for individual-level data <ol> <li><a href="http://dbgap.ncbi.nlm.nih.gov/aa/wga.cgi?view_pdf&stacc=phs000153.v3.p3" target="_blank">Data Use Certification Requirements (DUC)</a></li> <li><a href="GetPdf.cgi?id=phd001183.1" target="_blank">Instructions to Request Authorized Access</a></li> <li><a href="http://view.ncbi.nlm.nih.gov/dbgap-controlled" target="_blank">Apply here for controlled access to individual level data</a></li> <li><a href="GetPdf.cgi?id=phd000317" target="_blank">Participant Protection Policy FAQ</a></li> </ol> In 1948, researchers recruited men and women from the town of Framingham, Massachusetts, beginning the first round of extensive physical examinations and lifestyle interviews that would later be analyzed for common patterns related to CVD development. Initially, the Framingham Heart Study enrolled 5,209 men and women from the Framingham area who were between the ages of 28 and 62 years. Beginning in 1971, the Framingham Heart Study enrolled 5,124 men and women, who were either offspring of the original cohort or spouses of those offspring. In 2002, 4,095 third generation participants (men and women) were enrolled.

Dataset Information

Real-time community detection in full social networks on a laptop.

Publications

Real-time community detection in full social networks on a laptop.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets