Dataset Information

Evaluation measure for group-based record linkage.

ABSTRACT:

Introduction

The robustness of record linkage evaluation measures is of high importance since linkage techniques are assessed based on these. However, minimal research has been conducted to evaluate the suitability of existing evaluation measures in the context of linking groups of records. Linkage quality is generally evaluated based on traditional measures such as precision and recall. As we show, these traditional evaluation measures are not suitable for evaluating groups of linked records because they evaluate the quality of individual record pairs rather than the quality of records grouped into clusters.

Objectives

We set out to highlight the shortcomings of traditional evaluation measures and then propose a novel method to evaluate clustering quality in the context of group-based record linkage.

Methods

The proposed linkage evaluation method assesses how well individual records have been allocated into predicted groups/clusters with respect to ground-truth data. We first identify the best representative predicted cluster for each ground-truth cluster and, based on the resulting mapping, each record in a ground-truth cluster is assigned to one of seven categories. These categories reflect how well the linkage technique assigned records into groups.

Results

We empirically evaluated our proposed method using real-world data and showed that it better reflects the quality of clusters generated by three group-based record linkage techniques. We also showed that traditional measures such as precision and recall can produce ambiguous results whereas our method does not.

Conclusions

The proposed evaluation method provides unambiguous results regarding the assessed group-based record linkage approaches. The method comprises of seven categories which reflect how each record was predicted, providing more detailed information about the quality of the linkage result. This will help to make better-informed decisions about which linkage technique is best suited for a given linkage application.

SUBMITTER: Nanayakkara C

PROVIDER: S-EPMC8142966 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundLinkage of demographic, health, and developmental administrative data can enrich population-based surveillance and research on developmental and educational outcomes. Transparency of the record linkage process and results are required to assess potential biases.ObjectivesTo describe the approach used to link records of kindergarten children from the Early Development Instrument (EDI) in Ontario to health administrative data and test differences in characteristics of children by linkage status. We demonstrate how socio-demographic and medical risk factors amass in their contribution to early developmental vulnerability and test the concordance of health diagnoses in both the EDI and health datasets of linked records.MethodsChildren with records in the 2015 EDI cycle were deterministically linked to a population registry in Ontario, Canada. We compared sociodemographic and developmental vulnerability data between linked and unlinked records. Among linked records, we examined the contribution of medical and social risk factors obtained from health administrative data to developmental vulnerability identified in the EDI using descriptive analyses.ResultsOf 135,937 EDI records, 106,217 (78.1%) linked deterministically to a child in the Ontario health registry using birth date, sex, and postal code. The linked cohort was representative of children who completed the EDI in age, sex, rural residence, immigrant status, language, and special needs status. Linked data underestimated children living in the lowest neighbourhood income quintile (standardized difference [SD] 0.10) and with higher vulnerability in physical health and well-being (SD 0.11) , social competence (SD 0.10), and language and cognitive development (SD 0.12). Analysis of linked records showed developmental vulnerability is sometimes greater in children with social risk factors compared to those with medical risk factors. Common childhood conditions with records in health data were infrequently recorded in EDI records.ConclusionsLinkage of early developmental and health administrative data, in the absence of a single unique identifier, can be successful with few systematic biases introduced. Cross-sectoral linkages can highlight the relative contribution of medical and social risk factors to developmental vulnerability and poor school achievement.

Project description:BackgroundThe linkage of records which refer to the same entity in separate data collections is a common requirement in public health and biomedical research. Traditionally, record linkage techniques have required that all the identifying data in which links are sought be revealed to at least one party, often a third party. This necessarily invades personal privacy and requires complete trust in the intentions of that party and their ability to maintain security and confidentiality. Dusserre, Quantin, Bouzelat and colleagues have demonstrated that it is possible to use secure one-way hash transformations to carry out follow-up epidemiological studies without any party having to reveal identifying information about any of the subjects - a technique which we refer to as "blindfolded record linkage". A limitation of their method is that only exact comparisons of values are possible, although phonetic encoding of names and other strings can be used to allow for some types of typographical variation and data errors.MethodsA method is described which permits the calculation of a general similarity measure, the n-gram score, without having to reveal the data being compared, albeit at some cost in computation and data communication. This method can be combined with public key cryptography and automatic estimation of linkage model parameters to create an overall system for blindfolded record linkage.ResultsThe system described offers good protection against misdeeds or security failures by any one party, but remains vulnerable to collusion between or simultaneous compromise of two or more parties involved in the linkage operation. In order to reduce the likelihood of this, the use of last-minute allocation of tasks to substitutable servers is proposed. Proof-of-concept computer programmes written in the Python programming language are provided to illustrate the similarity comparison protocol.ConclusionAlthough the protocols described in this paper are not unconditionally secure, they do suggest the feasibility, with the aid of modern cryptographic techniques and high speed communication networks, of a general purpose probabilistic record linkage system which permits record linkage studies to be carried out with negligible risk of invasion of personal privacy.