Dataset Information

Direct estimation of cause-specific mortality fractions from verbal autopsies: multisite validation study using clinical diagnostic gold standards.

ABSTRACT:

Background

Verbal autopsy (VA) is used to estimate the causes of death in areas with incomplete vital registration systems. The King and Lu method (KL) for direct estimation of cause-specific mortality fractions (CSMFs) from VA studies is an analysis technique that estimates CSMFs in a population without predicting individual-level cause of death as an intermediate step. In previous studies, KL has shown promise as an alternative to physician-certified verbal autopsy (PCVA). However, it has previously been impossible to validate KL with a large dataset of VAs for which the underlying cause of death is known to meet rigorous clinical diagnostic criteria.

Methods

We applied the KL method to adult, child, and neonatal VA datasets from the Population Health Metrics Research Consortium gold standard verbal autopsy validation study, a multisite sample of 12,542 VAs where gold standard cause of death was established using strict clinical diagnostic criteria. To emulate real-world populations with varying CSMFs, we evaluated the KL estimations for 500 different test datasets of varying cause distribution. We assessed the quality of these estimates in terms of CSMF accuracy as well as linear regression and compared this with the results of PCVA.

Results

KL performance is similar to PCVA in terms of CSMF accuracy, attaining values of 0.669, 0.698, and 0.795 for adult, child, and neonatal age groups, respectively, when health care experience (HCE) items were included. We found that the length of the cause list has a dramatic effect on KL estimation quality, with CSMF accuracy decreasing substantially as the length of the cause list increases. We found that KL is not reliant on HCE the way PCVA is, and without HCE, KL outperforms PCVA for all age groups.

Conclusions

Like all computer methods for VA analysis, KL is faster and cheaper than PCVA. Since it is a direct estimation technique, though, it does not produce individual-level predictions. KL estimates are of similar quality to PCVA and slightly better in most cases. Compared to other recently developed methods, however, KL would only be the preferred technique when the cause list is short and individual-level predictions are not needed.

SUBMITTER: Flaxman AD

PROVIDER: S-EPMC3160928 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundPhysician review of a verbal autopsy (VA) and completion of a death certificate remains the most widely used approach for VA analysis. This study provides new evidence about the performance of physician-certified verbal autopsy (PCVA) using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 VAs. The study was also designed to analyze issues related to PCVA, such as the impact of a second physician reader on the cause of death assigned, the variation in performance with and without household recall of health care experience (HCE), and the importance of local information for physicians reading VAs.MethodsThe certification was performed by 24 physicians. The assignment of VA was random and blinded. Each VA was certified by one physician. Half of the VAs were reviewed by a different physician with household recall of health care experience included. The completed death certificate was processed for automated ICD-10 coding of the underlying cause of death. PCVA was compared to gold standard cause of death assignment based on strictly defined clinical diagnostic criteria that are part of the Population Health Metrics Research Consortium (PHMRC) gold standard verbal autopsy study.ResultsFor individual cause assignment, the overall chance-corrected concordance for PCVA against the gold standard cause of death is less than 50%, with substantial variability by cause and physician. Physicians assign the correct cause around 30% of the time without HCE, and addition of HCE improves performance in adults to 45% and slightly higher in children to 48%. Physicians estimate cause-specific mortality fractions (CSMFs) with considerable error for adults, children, and neonates. Only for neonates for a cause list of six causes with HCE is accuracy above 0.7. In all three age groups, CSMF accuracy improves when household recall of health care experience is available.ConclusionsResults show that physician coding for cause of death assignment may not be as robust as previously thought. The time and cost required to initially collect the verbal autopsies must be considered in addition to the analysis, as well as the impact of diverting physicians from servicing immediate health needs in a population to review VAs. All of these considerations highlight the importance and urgency of developing better methods to more reliably analyze past and future verbal autopsies to obtain the highest quality mortality data from populations without reliable death certification.

Project description:BackgroundInterVA is a widely disseminated tool for cause of death attribution using information from verbal autopsies. Several studies have attempted to validate the concordance and accuracy of the tool, but the main limitation of these studies is that they compare cause of death as ascertained through hospital record review or hospital discharge diagnosis with the results of InterVA. This study provides a unique opportunity to assess the performance of InterVA compared to physician-certified verbal autopsies (PCVA) and alternative automated methods for analysis.MethodsUsing clinical diagnostic gold standards to select 12,542 verbal autopsy cases, we assessed the performance of InterVA on both an individual and population level and compared the results to PCVA, conducting analyses separately for adults, children, and neonates. Following the recommendation of Murray et al., we randomly varied the cause composition over 500 test datasets to understand the performance of the tool in different settings. We also contrasted InterVA with an alternative Bayesian method, Simplified Symptom Pattern (SSP), to understand the strengths and weaknesses of the tool.ResultsAcross all age groups, InterVA performs worse than PCVA, both on an individual and population level. On an individual level, InterVA achieved a chance-corrected concordance of 24.2% for adults, 24.9% for children, and 6.3% for neonates (excluding free text, considering one cause selection). On a population level, InterVA achieved a cause-specific mortality fraction accuracy of 0.546 for adults, 0.504 for children, and 0.404 for neonates. The comparison to SSP revealed four specific characteristics that lead to superior performance of SSP. Increases in chance-corrected concordance are attained by developing cause-by-cause models (2%), using all items as opposed to only the ones that mapped to InterVA items (7%), assigning probabilities to clusters of symptoms (6%), and using empirical as opposed to expert probabilities (up to 8%).ConclusionsGiven the widespread use of verbal autopsy for understanding the burden of disease and for setting health intervention priorities in areas that lack reliable vital registrations systems, accurate analysis of verbal autopsies is essential. While InterVA is an affordable and available mechanism for assigning causes of death using verbal autopsies, users should be aware of its suboptimal performance relative to other methods.

Project description:BackgroundComputer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs).MethodsThe Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE).ResultsFor all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates.ConclusionsWe found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

Project description:Nigeria's under-five mortality rate is the eighth highest in the world. Identifying the causes of under-five deaths is crucial to achieving Sustainable Development Goal 3 by 2030 and improving child survival. National and international bodies collaborated in this study to provide the first ever direct estimates of the causes of under-five mortality in Nigeria. Verbal autopsy interviews were conducted of a representative sample of 986 neonatal and 2,268 1-59 month old deaths from 2008 to 2013 identified by the 2013 Nigeria Demographic and Health Survey. Cause of death was assigned by physician coding and computerized expert algorithms arranged in a hierarchy. National and regional estimates of age distributions, mortality rates and cause proportions, and zonal- and age-specific mortality fractions and rates for leading causes of death were evaluated. More under-fives and 1-59 month olds in the South, respectively, died as neonates (N = 24.1%, S = 32.5%, p<0.001) and at younger ages (p<0.001) than in the North. The leading causes of neonatal and 1-59 month mortality, respectively, were sepsis, birth injury/asphyxia and neonatal pneumonia, and malaria, diarrhea and pneumonia. The preterm delivery (N = 1.2%, S = 3.7%, p = 0.042), pneumonia (N = 15.0%, S = 21.6%, p = 0.004) and malaria (N = 34.7%, S = 42.2%, p = 0.009) fractions were higher in the South, with pneumonia and malaria focused in the South East and South South; while the diarrhea fraction was elevated in the North (N = 24.8%, S = 13.2%, p<0.001). However, the diarrhea, pneumonia and malaria mortality rates were all higher in the North, respectively, by 222.9% (Z = -10.9, p = 0.000), 27.6% (Z = -2.3, p = 0.020) and 50.6% (Z = -5.7, p = 0.000), with the greatest excesses in older children. The findings support that there is an epidemiological transition ongoing in southern Nigeria, suggest the way forward to a similar transition in the North, and can help guide maternal, neonatal and child health programming and their regional and zonal foci within the country.

Project description:BACKGROUND:Almost all countries without complete vital registration systems have data on deaths collected by hospitals. However, these data have not been widely used to estimate cause of death (COD) patterns in populations because only a non-representative fraction of people in these countries die in health facilities. Methods that can exploit hospital mortality statistics to reliably estimate community COD patterns are required to strengthen the evidence base for disease and injury control programs. We propose a method that weights hospital-certified causes by the probability of death to estimate population cause-specific mortality fractions (CSMFs). METHODS:We used an established verbal autopsy instrument (VAI) to collect data from hospital catchment areas in Chandpur and Comilla Districts, Bangladesh, and Bohol province, the Philippines, between 2011 and 2014, along with demographic covariates for each death. Hospital medical certificates of cause of death (death certificates) were collected and mapped to the corresponding cause categories of the VAI. Tariff 2.0 was used to assign a COD for community deaths. Logistic regression models were created for broad causes in each country to calculate the probability of in-hospital death, given a set of covariate values. The reweighted CSMFs for deaths in the hospital catchment population, represented by each hospital death, were calculated from the corresponding regression models. RESULTS:We collected data on 4228 adult deaths in the Philippines and 3725 deaths in Bangladesh. Short time to hospital and education were consistently associated with in-hospital death in the Philippines and absence of a disability was consistently associated with in-hospital death in Bangladesh. Non-communicable diseases (excluding stroke) and stroke were the leading causes of death in both the Philippines (33.9%, 19.1%) and Bangladesh (46.1%, 21.1%) according to the reweighted method. The reweighted method generally estimated CSMFs that fell between those derived from hospitals and those diagnosed by Tariff 2.0. CONCLUSIONS:Statistical methods can be used to derive estimates of cause-specific probability of death in-hospital for Bangladesh and the Philippines to generate population CSMFs. In regions where hospital death certification is of reasonable quality and routine verbal autopsy is not applied, these estimates could be applied to generate cost-effective and robust CSMFs for the population.