Dataset Information

Two Wrongs Make a Right: Addressing Underreporting in Binary Data from Multiple Sources.

ABSTRACT: Media-based event data-i.e., data comprised from reporting by media outlets-are widely used in political science research. However, events of interest (e.g., strikes, protests, conflict) are often underreported by these primary and secondary sources, producing incomplete data that risks inconsistency and bias in subsequent analysis. While general strategies exist to help ameliorate this bias, these methods do not make full use of the information often available to researchers. Specifically, much of the event data used in the social sciences is drawn from multiple, overlapping news sources (e.g., Agence France-Presse, Reuters). Therefore, we propose a novel maximum likelihood estimator that corrects for misclassification in data arising from multiple sources. In the most general formulation of our estimator, researchers can specify separate sets of predictors for the true-event model and each of the misclassification models characterizing whether a source fails to report on an event. As such, researchers are able to accurately test theories on both the causes of and reporting on an event of interest. Simulations evidence that our technique regularly out performs current strategies that either neglect misclassification, the unique features of the data-generating process, or both. We also illustrate the utility of this method with a model of repression using the Social Conflict in Africa Database.

SUBMITTER: Cook SJ

PROVIDER: S-EPMC5667662 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Two Wrongs Make a Right: Addressing Underreporting in Binary Data from Multiple Sources.

Cook Scott J SJ Blas Betsabe B Carroll Raymond J RJ Sinha Samiran S

Political analysis : an annual publication of the Methodology Section of the American Political Science Association 20170411 2

Media-based event data-i.e., data comprised from reporting by media outlets-are widely used in political science research. However, events of interest (e.g., strikes, protests, conflict) are often underreported by these primary and secondary sources, producing incomplete data that risks inconsistency and bias in subsequent analysis. While general strategies exist to help ameliorate this bias, these methods do not make full use of the information often available to researchers. Specifically, much ...[more]

PMID: 29104409

Similar Datasets

Project description:Invasive pneumococcal disease (IPD) is a vaccine-preventable disease characterized by the presence of Streptococcus pneumoniae in normally sterile sites. Since 2007, Italy has implemented an IPD national surveillance system (IPD-NSS). This system suffers from high rates of underreporting. To estimate the level of underreporting of IPD in 2016-2017 in Tuscany (Italy), we integrated data from IPD-NSS and two other regional data sources, i.e., Tuscany regional microbiological surveillance (Microbiological Surveillance and Antibiotic Resistance in Tuscany, SMART) and hospitalization discharge records (HDRs). We collected (1) notifications to IPD-NSS, (2) SMART records positive for S. pneumoniae from normally sterile sites, and (3) hospitalization records with IPD-related International Classification of Diseases, Ninth Revision, Clinical Modification (ICD9) codes in discharge diagnoses. We performed data linkage of the three sources to obtain a combined surveillance system (CSS). Using the CSS, we calculated the completeness of the three sources and performed a three-source log-linear capture-recapture analysis to estimate total IPD underreporting. In total, 127 IPD cases were identified from IPD-NSS, 320 were identified from SMART, and 658 were identified from HDRs. After data linkage, a total of 904 unique cases were detected. The average yearly CSS notification rate was 12.1/100,000 inhabitants. Completeness was 14.0% for IPD-NSS, 35.4% for SMART, and 72.8% for HDRs. The capture-recapture analysis suggested a total estimate of 3419 cases of IPD (95% confidence interval (CI): 1364-5474), corresponding to an underreporting rate of 73.7% (95% CI: 34.0-83.6) for CSS. This study shows substantial underreporting in the Tuscany IPD surveillance system. Integration of available data sources may be a useful approach to complement notification-based surveillance and provide decision-makers with better information to plan effective control strategies against IPD.

Project description:BackgroundPrior research suggests that United States governmental sources documenting the number of law-enforcement-related deaths (i.e., fatalities due to injuries inflicted by law enforcement officers) undercount these incidents. The National Vital Statistics System (NVSS), administered by the federal government and based on state death certificate data, identifies such deaths by assigning them diagnostic codes corresponding to "legal intervention" in accordance with the International Classification of Diseases-10th Revision (ICD-10). Newer, nongovernmental databases track law-enforcement-related deaths by compiling news media reports and provide an opportunity to assess the magnitude and determinants of suspected NVSS underreporting. Our a priori hypotheses were that underreporting by the NVSS would exceed that by the news media sources, and that underreporting rates would be higher for decedents of color versus white, decedents in lower versus higher income counties, decedents killed by non-firearm (e.g., Taser) versus firearm mechanisms, and deaths recorded by a medical examiner versus coroner.Methods and findingsWe created a new US-wide dataset by matching cases reported in a nongovernmental, news-media-based dataset produced by the newspaper The Guardian, The Counted, to identifiable NVSS mortality records for 2015. We conducted 2 main analyses for this cross-sectional study: (1) an estimate of the total number of deaths and the proportion unreported by each source using capture-recapture analysis and (2) an assessment of correlates of underreporting of law-enforcement-related deaths (demographic characteristics of the decedent, mechanism of death, death investigator type [medical examiner versus coroner], county median income, and county urbanicity) in the NVSS using multilevel logistic regression. We estimated that the total number of law-enforcement-related deaths in 2015 was 1,166 (95% CI: 1,153, 1,184). There were 599 deaths reported in The Counted only, 36 reported in the NVSS only, 487 reported in both lists, and an estimated 44 (95% CI: 31, 62) not reported in either source. The NVSS documented 44.9% (95% CI: 44.2%, 45.4%) of the total number of deaths, and The Counted documented 93.1% (95% CI: 91.7%, 94.2%). In a multivariable mixed-effects logistic model that controlled for all individual- and county-level covariates, decedents injured by non-firearm mechanisms had higher odds of underreporting in the NVSS than those injured by firearms (odds ratio [OR]: 68.2; 95% CI: 15.7, 297.5; p < 0.01), and underreporting was also more likely outside of the highest-income-quintile counties (OR for the lowest versus highest income quintile: 10.1; 95% CI: 2.4, 42.8; p < 0.01). There was no statistically significant difference in the odds of underreporting in the NVSS for deaths certified by coroners compared to medical examiners, and the odds of underreporting did not vary by race/ethnicity. One limitation of our analyses is that we were unable to examine the characteristics of cases that were unreported in The Counted.ConclusionsThe media-based source, The Counted, reported a considerably higher proportion of law-enforcement-related deaths than the NVSS, which failed to report a majority of these incidents. For the NVSS, rates of underreporting were higher in lower income counties and for decedents killed by non-firearm mechanisms. There was no evidence suggesting that underreporting varied by death investigator type (medical examiner versus coroner) or race/ethnicity.

Project description:BackgroundQuestionnaires used in longitudinal studies may have questions added or removed over time for numerous reasons. Data missing completely at a follow-up survey is a unique issue for longitudinal studies. While such excluded questions lack information at one follow-up survey, they are collected at other follow-up surveys, and covariances observed at other follow-up surveys may allow for the recovery of the missing data. This study utilized data from a large longitudinal cohort study to assess the efficiency and feasibility of using multiple imputation (MI) to recover this type of information.MethodsMillennium Cohort Study participants completed the 9-item Patient Health Questionnaire (PHQ) depression module at 2 time points (2004, 2007). The suicidal ideation item in the module was set to missing for the 2007 assessment. Several single-level MI models using different sets of predictors and forms of suicidal ideation were used to compare self-reported values and imputed values for this item in 2007. Additionally, associations with sleep duration and smoking status, which are related constructs, were compared between self-reported and imputed values of suicidal ideation.ResultsAmong 63,028 participants eligible for imputation analysis, 4.05% reported suicidal ideation on the 2007 survey. The imputation models successfully identified suicidal ideation, with a sensitivity ranging between 34 and 66% and a positive predictive value between 36 and 42%. Specificity remained above 96% and negative predictive value above 97% for all imputed models. Similar associations were found for all imputation models on related constructs, though the dichotomous suicidal ideation imputed from the model using only PHQ depression items yielded estimates that were closest with the self-reported associations for all adjusted analyses.ConclusionsAlthough sensitivity and positive predictive value were relatively low, applying MI techniques allowed for inclusion of an otherwise missing variable. Additionally, correlations with related constructs were estimated near self-reported values. Therefore, the other 8 depression items can be used to estimate suicidal ideation that was completely missing from a survey using MI. However, these imputed values should not be used to estimate population prevalence.

Dataset Information

Two Wrongs Make a Right: Addressing Underreporting in Binary Data from Multiple Sources.

Publications

Two Wrongs Make a Right: Addressing Underreporting in Binary Data from Multiple Sources.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets