Historical Census Record Linkage.
Ontology highlight
ABSTRACT:
SUBMITTER: Ruggles S
PROVIDER: S-EPMC6200350 | biostudies-literature | 2018 Jul
REPOSITORIES: biostudies-literature
Publications
Historical Census Record Linkage.
Ruggles Steven S Fitch Catherine C Roberts Evan E
Annual review of sociology 20180518
For the past 80 years, social scientists have been linking historical censuses across time to study economic and geographic mobility. In recent decades, the quantity of historical census record linkage has exploded, owing largely to the advent of new machine-readable data created by genealogical organizations. Investigators are examining economic and geographic mobility across multiple generations, but also engaging many new topics. Several analysts are exploring the effects of early-life socioe ...[more]
Similar Datasets
Project description:BackgroundAdvancements in linking publicly available census records with vital and administrative records have enabled novel investigations in epidemiology and social history. However, in the absence of unique identifiers, the linkage of the records may be uncertain or only be successful for a subset of the census cohort, resulting in missing data. For survival analysis, differential ascertainment of event times can impact inference on risk associations and median survival.MethodsWe modify some existing approaches that are commonly used to handle missing survival times to accommodate this imperfect linkage situation including complete case analysis, censoring, weighting, and several multiple imputation methods. We then conduct simulation studies to compare the performance of the proposed approaches in estimating the associations of a risk factor or exposure in terms of hazard ratio (HR) and median survival times in the presence of missing survival times. The effects of different missing data mechanisms and exposure-survival associations on their performance are also explored. The approaches are applied to a historic cohort of residents in Ambler, PA, established using the 1930 US census, from which only 2,440 out of 4,514 individuals (54%) had death records retrievable from publicly available data sources and death certificates. Using this cohort, we examine the effects of occupational and paraoccupational asbestos exposure on survival and disparities in mortality by race and gender.ResultsWe show that imputation based on conditional survival results in less bias and greater efficiency relative to a complete case analysis when estimating log-hazard ratios and median survival times. When the approaches are applied to the Ambler cohort, we find a significant association between occupational exposure and mortality, particularly among black individuals and males, but not between paraoccupational exposure and mortality.DiscussionThis investigation illustrates the strengths and weaknesses of different imputation methods for missing survival times due to imperfect linkage of the administrative or registry data. The performance of the methods may depend on the missingness process as well as the parameter being estimated and models of interest, and such factors should be considered when choosing the methods to address the missing event times.
| S-EPMC10935812 | biostudies-literature
Project description:BackgroundTo assess the feasibility and quality of an anonymous linkage of 1) MONICA (MONItoring of trends and determinants in CArdiovscular disease, three waves between 1984 and 1993) data with 2) census and mortality records of the Swiss National Cohort in order to establish a mortality follow-up until 2008. Many countries feature the defect of lacking general population cohorts because they have missed to provide for follow-up information of health surveys.MethodsRecord linkage procedures were used in a multi-step approach. Kaplan-Meier curves from our data were contrasted with the survival probabilities expected from life tables for the general population, age-standardized mortality rates from our data with those derived from official cross-sectional mortality data. Cox regression models were fit to investigate the influence of covariates on survival.Results97.8% of the eligible 10,160 participants (25-74y at baseline) could be linked to a census (1990: 9,737; 2000: 8,749), mortality (1,526, 1984-2008) and/or emigration record (320, 1990-2008). Linkage success did not differ by any key study characteristic. Results of survival analyses were robust to linkage step or certainty of a correct link. Loss to follow-up between 1990 and 2000 amounted to 4.7%. MONICA participants had lower mortality than the general population, but similar mortality patterns, (e.g. variation by educational level, marital status or region).ConclusionsUsing anonymized census and death records allowed an almost complete mortality follow-up of MONICA study participants of up to 25 years. Lower mortality compared to the general population was in line with a presumable 'healthy participant' selection in the original MONICA study. Apart from that, the derived data set reproduced known mortality patterns and showed only negligible potential for selection bias introduced by the linkage process. Anonymous record linkage was feasible and provided robust results. It can thus provide valuable information, when no cohort study is available.
| S-EPMC2955001 | biostudies-literature
Project description:Studies involving the use of probabilistic record linkage are becoming increasingly common. However, the methods underpinning probabilistic record linkage are not widely taught or understood, and therefore these studies can appear to be a 'black box' research tool. In this article, we aim to describe the process of probabilistic record linkage through a simple exemplar. We first introduce the concept of deterministic linkage and contrast this with probabilistic linkage. We illustrate each step of the process using a simple exemplar and describe the data structure required to perform a probabilistic linkage. We describe the process of calculating and interpreting matched weights and how to convert matched weights into posterior probabilities of a match using Bayes theorem. We conclude this article with a brief discussion of some of the computational demands of record linkage, how you might assess the quality of your linkage algorithm, and how epidemiologists can maximize the value of their record-linked research using robust record linkage methods.
| S-EPMC5005943 | biostudies-literature
Project description:ObjectiveRecord linkage to integrate uncoordinated databases is critical in biomedical research using Big Data. Balancing privacy protection against the need for high quality record linkage requires a human-machine hybrid system to safely manage uncertainty in the ever changing streams of chaotic Big Data.MethodsIn the computer science literature, private record linkage is the most published area. It investigates how to apply a known linkage function safely when linking two tables. However, in practice, the linkage function is rarely known. Thus, there are many data linkage centers whose main role is to be the trusted third party to determine the linkage function manually and link data for research via a master population list for a designated region. Recently, a more flexible computerized third-party linkage platform, Secure Decoupled Linkage (SDLink), has been proposed based on: (1) decoupling data via encryption, (2) obfuscation via chaffing (adding fake data) and universe manipulation; and (3) minimum information disclosure via recoding.ResultsWe synthesize this literature to formalize a new framework for privacy preserving interactive record linkage (PPIRL) with tractable privacy and utility properties and then analyze the literature using this framework.ConclusionsHuman-based third-party linkage centers for privacy preserving record linkage are the accepted norm internationally. We find that a computer-based third-party platform that can precisely control the information disclosed at the micro level and allow frequent human interaction during the linkage process, is an effective human-machine hybrid system that significantly improves on the linkage center model both in terms of privacy and utility.
| S-EPMC3932473 | biostudies-literature
Project description:BackgroundThe linkage of records which refer to the same entity in separate data collections is a common requirement in public health and biomedical research. Traditionally, record linkage techniques have required that all the identifying data in which links are sought be revealed to at least one party, often a third party. This necessarily invades personal privacy and requires complete trust in the intentions of that party and their ability to maintain security and confidentiality. Dusserre, Quantin, Bouzelat and colleagues have demonstrated that it is possible to use secure one-way hash transformations to carry out follow-up epidemiological studies without any party having to reveal identifying information about any of the subjects - a technique which we refer to as "blindfolded record linkage". A limitation of their method is that only exact comparisons of values are possible, although phonetic encoding of names and other strings can be used to allow for some types of typographical variation and data errors.MethodsA method is described which permits the calculation of a general similarity measure, the n-gram score, without having to reveal the data being compared, albeit at some cost in computation and data communication. This method can be combined with public key cryptography and automatic estimation of linkage model parameters to create an overall system for blindfolded record linkage.ResultsThe system described offers good protection against misdeeds or security failures by any one party, but remains vulnerable to collusion between or simultaneous compromise of two or more parties involved in the linkage operation. In order to reduce the likelihood of this, the use of last-minute allocation of tasks to substitutable servers is proposed. Proof-of-concept computer programmes written in the Python programming language are provided to illustrate the similarity comparison protocol.ConclusionAlthough the protocols described in this paper are not unconditionally secure, they do suggest the feasibility, with the aid of modern cryptographic techniques and high speed communication networks, of a general purpose probabilistic record linkage system which permits record linkage studies to be carried out with negligible risk of invasion of personal privacy.
| S-EPMC471556 | biostudies-literature
Project description:There has been substantial recent interest in record linkage, where one attempts to group the records pertaining to the same entities from one or more large databases that lack unique identifiers. This can be viewed as a type of microclustering, with few observations per cluster and a very large number of clusters. We show that the problem is fundamentally hard from a theoretical perspective and, even in idealized cases, accurate entity resolution is effectively impossible unless the number of entities is small relative to the number of records and/or the separation between records from different entities is extremely large. These results suggest conservatism in interpretation of the results of record linkage, support collection of additional data to more accurately disambiguate the entities, and motivate a focus on coarser inference. For example, results from a simulation study suggest that sometimes one may obtain accurate results for population size estimation even when fine-scale entity resolution is inaccurate.
| S-EPMC5963577 | biostudies-literature
Project description:IntroductionThe robustness of record linkage evaluation measures is of high importance since linkage techniques are assessed based on these. However, minimal research has been conducted to evaluate the suitability of existing evaluation measures in the context of linking groups of records. Linkage quality is generally evaluated based on traditional measures such as precision and recall. As we show, these traditional evaluation measures are not suitable for evaluating groups of linked records because they evaluate the quality of individual record pairs rather than the quality of records grouped into clusters.ObjectivesWe set out to highlight the shortcomings of traditional evaluation measures and then propose a novel method to evaluate clustering quality in the context of group-based record linkage.MethodsThe proposed linkage evaluation method assesses how well individual records have been allocated into predicted groups/clusters with respect to ground-truth data. We first identify the best representative predicted cluster for each ground-truth cluster and, based on the resulting mapping, each record in a ground-truth cluster is assigned to one of seven categories. These categories reflect how well the linkage technique assigned records into groups.ResultsWe empirically evaluated our proposed method using real-world data and showed that it better reflects the quality of clusters generated by three group-based record linkage techniques. We also showed that traditional measures such as precision and recall can produce ambiguous results whereas our method does not.ConclusionsThe proposed evaluation method provides unambiguous results regarding the assessed group-based record linkage approaches. The method comprises of seven categories which reflect how each record was predicted, providing more detailed information about the quality of the linkage result. This will help to make better-informed decisions about which linkage technique is best suited for a given linkage application.
| S-EPMC8142966 | biostudies-literature
Project description:BackgroundWithin the field of record linkage, numerous data cleaning and standardisation techniques are employed to ensure the highest quality of links. While these facilities are common in record linkage software packages and are regularly deployed across record linkage units, little work has been published demonstrating the impact of data cleaning on linkage quality.MethodsA range of cleaning techniques was applied to both a synthetically generated dataset and a large administrative dataset previously linked to a high standard. The effect of these changes on linkage quality was investigated using pairwise F-measure to determine quality.ResultsData cleaning made little difference to the overall linkage quality, with heavy cleaning leading to a decrease in quality. Further examination showed that decreases in linkage quality were due to cleaning techniques typically reducing the variability - although correct records were now more likely to match, incorrect records were also more likely to match, and these incorrect matches outweighed the correct matches, reducing quality overall.ConclusionsData cleaning techniques have minimal effect on linkage quality. Care should be taken during the data cleaning process.
| S-EPMC3688507 | biostudies-literature
Project description:ObjectiveTo investigate the time relations between long haul air travel and venous thromboembolism.DesignRecord linkage study using the case crossover approach.SettingWestern Australia.Participants5408 patients admitted to hospital with venous thromboembolism and matched with data for arrivals of international flights during 1981-99.ResultsThe risk of venous thromboembolism is increased for only two weeks after a long haul flight; 46 Australian citizens and 200 non-Australian citizens had an episode of venous thromboembolism during this so called hazard period. The relative risk during this period for Australian citizens was 4.17 (95% confidence interval, 2.94 to 5.40), with 76% of cases (n = 35) attributable to the preceding flight. A "healthy traveller" effect was observed, particularly for Australian citizens.ConclusionsThe annual risk of venous thromboembolism is increased by 12% if one long haul flight is taken yearly. The average risk of death from flight related venous thromboembolism is small compared with that from motor vehicle crashes and injuries at work. The individual risk of death from flight related venous thromboembolism for people with certain pre-existing medical conditions is, however, likely to be greater than the average risk of 1 per 2 million for passengers arriving from a flight. Airlines and health authorities should continue to advise passengers on how to minimise risk.
| S-EPMC261739 | biostudies-literature
Project description:PurposeWe examined the associations between recurrent single- and multisite pain and incident sickness absence (SA) of different lengths and the risk of disability pension (DP).MethodsThe data were derived from the Finnish Helsinki Health Study. Pain measures were recorded for panel 1 in 2000/2 and 2007, and for panel 2 in 2007 and 2012 (altogether 3191 employees). SA data were obtained from the employer's personnel register and DP events from the Finnish Centre for Pensions. Negative binomial regression models with generalized estimation equations were used to model the incidence of self-certified short- (1-3 days), and medically certified medium- (4-14 days) and long-term (more than 14 days) SA episodes. Cox regression models were fitted for the associations between pain and all-cause DP and competing risk models for DP by diagnostic groups. Social and health-related covariates were adjusted for.ResultsRecurrent pain was associated with short-, medium- and long-term SA. Additionally, recurrent single- and multisite pain increased the risk of long-term SA. Recurrent single or multisite pain was further associated with an increased risk of DP, while a single instance of pain did not increase the risk.ConclusionsThese results suggest that recurrent pain is a robust determinant of subsequent SA and DP risk. Improved understanding of determinants of recurrent pain is needed to inform the development of targeted measures to reduce SA and premature exit from employment.
| S-EPMC7118055 | biostudies-literature