Dataset Information

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods.

ABSTRACT:

Background

Verbal Autopsy (VA) is widely viewed as the only immediate strategy for registering cause of death in much of Africa and Asia, where routine physician certification of deaths is not widely practiced. VA involves a lay interview with family or friends after a death, to record essential details of the circumstances. These data can then be processed automatically to arrive at standardized cause of death information.

Methods

The Population Health Metrics Research Consortium (PHMRC) undertook a study at six tertiary hospitals in low- and middle-income countries which documented over 12,000 deaths clinically and subsequently undertook VA interviews. This dataset, now in the public domain, was compared with the WHO 2012 VA standard and the InterVA-4 interpretative model.

Results

The PHMRC data covered 70% of the WHO 2012 VA input indicators, and categorized cause of death according to PHMRC definitions. After eliminating some problematic or incomplete records, 11,984 VAs were compared. Some of the PHMRC cause definitions, such as 'preterm delivery', differed substantially from the International Classification of Diseases, version 10 equivalent. There were some appreciable inconsistencies between the hospital and VA data, including 20% of the hospital maternal deaths being described as non-pregnant in the VA data. A high proportion of VA cases (66%) reported respiratory symptoms, but only 18% of assigned hospital causes were respiratory-related. Despite these issues, the concordance correlation coefficient between hospital and InterVA-4 cause of death categories was 0.61.

Conclusions

The PHMRC dataset is a valuable reference source for VA methods, but has to be interpreted with care. Inherently inconsistent cases should not be included when using these data to build other VA models. Conversely, models built from these data should be independently evaluated. It is important to distinguish between the internal and external validity of VA models. The effects of using tertiary hospital data, rather than the more usual application of VA to all-community deaths, are hard to evaluate. However, it would still be of value for VA method development to have further studies of population-based post-mortem examinations.

SUBMITTER: Byass P

PROVIDER: S-EPMC3912496 | biostudies-literature | 2014 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods.

Byass Peter P

BMC medicine 20140204

<h4>Background</h4>Verbal Autopsy (VA) is widely viewed as the only immediate strategy for registering cause of death in much of Africa and Asia, where routine physician certification of deaths is not widely practiced. VA involves a lay interview with family or friends after a death, to record essential details of the circumstances. These data can then be processed automatically to arrive at standardized cause of death information.<h4>Methods</h4>The Population Health Metrics Research Consortium ...[more]

PMID: 24495341

Similar Datasets

Project description:BackgroundVerbal autopsy methods are critically important for evaluating the leading causes of death in populations without adequate vital registration systems. With a myriad of analytical and data collection approaches, it is essential to create a high quality validation dataset from different populations to evaluate comparative method performance and make recommendations for future verbal autopsy implementation. This study was undertaken to compile a set of strictly defined gold standard deaths for which verbal autopsies were collected to validate the accuracy of different methods of verbal autopsy cause of death assignment.MethodsData collection was implemented in six sites in four countries: Andhra Pradesh, India; Bohol, Philippines; Dar es Salaam, Tanzania; Mexico City, Mexico; Pemba Island, Tanzania; and Uttar Pradesh, India. The Population Health Metrics Research Consortium (PHMRC) developed stringent diagnostic criteria including laboratory, pathology, and medical imaging findings to identify gold standard deaths in health facilities as well as an enhanced verbal autopsy instrument based on World Health Organization (WHO) standards. A cause list was constructed based on the WHO Global Burden of Disease estimates of the leading causes of death, potential to identify unique signs and symptoms, and the likely existence of sufficient medical technology to ascertain gold standard cases. Blinded verbal autopsies were collected on all gold standard deaths.ResultsOver 12,000 verbal autopsies on deaths with gold standard diagnoses were collected (7,836 adults, 2,075 children, 1,629 neonates, and 1,002 stillbirths). Difficulties in finding sufficient cases to meet gold standard criteria as well as problems with misclassification for certain causes meant that the target list of causes for analysis was reduced to 34 for adults, 21 for children, and 10 for neonates, excluding stillbirths. To ensure strict independence for the validation of methods and assessment of comparative performance, 500 test-train datasets were created from the universe of cases, covering a range of cause-specific compositions.ConclusionsThis unique, robust validation dataset will allow scholars to evaluate the performance of different verbal autopsy analytic methods as well as instrument design. This dataset can be used to inform the implementation of verbal autopsies to more reliably ascertain cause of death in national health information systems.

Project description:BackgroundThe South African national cause of death validation (NCODV 2017/18) project collected a national sample of verbal autopsies (VA) with cause of death (COD) assignment by physician-coded VA (PCVA) and computer-coded VA (CCVA).ObjectiveThe performance of three CCVA algorithms (InterVA-5, InSilicoVA and Tariff 2.0) in assigning a COD was compared with PCVA (reference standard).MethodsSeven performance metrics assessed individual and population level agreement of COD assignment by age, sex and place of death subgroups. Positive predictive value (PPV), sensitivity, overall agreement, kappa, and chance corrected concordance (CCC) assessed individual level agreement. Cause-specific mortality fraction (CSMF) accuracy and Spearman's rank correlation assessed population level agreement.ResultsA total of 5386 VA records were analysed. PCVA and CCVAs all identified HIV/AIDS as the leading COD. CCVA PPV and sensitivity, based on confidence intervals, were comparable except for HIV/AIDS, TB, maternal, diabetes mellitus, other cancers, and some injuries. CCVAs performed well for identifying perinatal deaths, road traffic accidents, suicide and homicide but poorly for pneumonia, other infectious diseases and renal failure. Overall agreement between CCVAs and PCVA for the top single cause (48.2-51.6) indicated comparable weak agreement between methods. Overall agreement, for the top three causes showed moderate agreement for InterVA (70.9) and InSilicoVA (73.8). Agreement based on kappa (-0.05-0.49)and CCC (0.06-0.43) was weak to none for all algorithms and groups. CCVAs had moderate to strong agreement for CSMF accuracy, with InterVA-5 highest for neonates (0.90), Tariff 2.0 highest for adults (0.89) and males (0.84), and InSilicoVA highest for females (0.88), elders (0.83) and out-of-facility deaths (0.85). Rank correlation indicated moderate agreement for adults (0.75-0.79).ConclusionsWhilst CCVAs identified HIV/AIDS as the leading COD, consistent with PCVA, there is scope for improving the algorithms for use in South Africa.

Project description:BackgroundComputer-coded verbal autopsy (CCVA) is a promising alternative to the standard approach of physician-certified verbal autopsy (PCVA), because of its high speed, low cost, and reliability. This study introduces a new CCVA technique and validates its performance using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 verbal autopsies (VAs).MethodsThe Random Forest (RF) Method from machine learning (ML) was adapted to predict cause of death by training random forests to distinguish between each pair of causes, and then combining the results through a novel ranking technique. We assessed quality of the new method at the individual level using chance-corrected concordance and at the population level using cause-specific mortality fraction (CSMF) accuracy as well as linear regression. We also compared the quality of RF to PCVA for all of these metrics. We performed this analysis separately for adult, child, and neonatal VAs. We also assessed the variation in performance with and without household recall of health care experience (HCE).ResultsFor all metrics, for all settings, RF was as good as or better than PCVA, with the exception of a nonsignificantly lower CSMF accuracy for neonates with HCE information. With HCE, the chance-corrected concordance of RF was 3.4 percentage points higher for adults, 3.2 percentage points higher for children, and 1.6 percentage points higher for neonates. The CSMF accuracy was 0.097 higher for adults, 0.097 higher for children, and 0.007 lower for neonates. Without HCE, the chance-corrected concordance of RF was 8.1 percentage points higher than PCVA for adults, 10.2 percentage points higher for children, and 5.9 percentage points higher for neonates. The CSMF accuracy was higher for RF by 0.102 for adults, 0.131 for children, and 0.025 for neonates.ConclusionsWe found that our RF Method outperformed the PCVA method in terms of chance-corrected concordance and CSMF accuracy for adult and child VA with and without HCE and for neonatal VA without HCE. It is also preferable to PCVA in terms of time and cost. Therefore, we recommend it as the technique of choice for analyzing past and current verbal autopsies.

Project description:BACKGROUND:Monitoring progress with disease and injury reduction in many populations will require widespread use of verbal autopsy (VA). Multiple methods have been developed for assigning cause of death from a VA but their application is restricted by uncertainty about their reliability. METHODS:We investigated the validity of five automated VA methods for assigning cause of death: InterVA-4, Random Forest (RF), Simplified Symptom Pattern (SSP), Tariff method (Tariff), and King-Lu (KL), in addition to physician review of VA forms (PCVA), based on 12,535 cases from diverse populations for which the true cause of death had been reliably established. For adults, children, neonates and stillbirths, performance was assessed separately for individuals using sensitivity, specificity, Kappa, and chance-corrected concordance (CCC) and for populations using cause specific mortality fraction (CSMF) accuracy, with and without additional diagnostic information from prior contact with health services. A total of 500 train-test splits were used to ensure that results are robust to variation in the underlying cause of death distribution. RESULTS:Three automated diagnostic methods, Tariff, SSP, and RF, but not InterVA-4, performed better than physician review in all age groups, study sites, and for the majority of causes of death studied. For adults, CSMF accuracy ranged from 0.764 to 0.770, compared with 0.680 for PCVA and 0.625 for InterVA; CCC varied from 49.2% to 54.1%, compared with 42.2% for PCVA, and 23.8% for InterVA. For children, CSMF accuracy was 0.783 for Tariff, 0.678 for PCVA, and 0.520 for InterVA; CCC was 52.5% for Tariff, 44.5% for PCVA, and 30.3% for InterVA. For neonates, CSMF accuracy was 0.817 for Tariff, 0.719 for PCVA, and 0.629 for InterVA; CCC varied from 47.3% to 50.3% for the three automated methods, 29.3% for PCVA, and 19.4% for InterVA. The method with the highest sensitivity for a specific cause varied by cause. CONCLUSIONS:Physician review of verbal autopsy questionnaires is less accurate than automated methods in determining both individual and population causes of death. Overall, Tariff performs as well or better than other methods and should be widely applied in routine mortality surveillance systems with poor cause of death certification practices.

Project description:BackgroundInformation on causes of death (COD) is crucial for measuring the health outcomes of populations and progress towards the Sustainable Development Goals. In many countries such as Vietnam where the civil registration and vital statistics (CRVS) system is dysfunctional, information on vital events will continue to rely on verbal autopsy (VA) methods. This study assesses the validity of VA methods used in Vietnam, and provides recommendations on methods for implementing VA validation studies in Vietnam.MethodsThis validation study was conducted on a sample of 670 deaths from a recent VA study in Quang Ninh province. The study covered 116 cases from this sample, which met three inclusion criteria: a) the death occurred within 30 days of discharge after last hospitalisation, and b) medical records (MRs) for the deceased were available from respective hospitals, and c) the medical record mentioned that the patient was terminally ill at discharge. For each death, the underlying cause of death (UCOD) identified from MRs was compared to the UCOD from VA. The validity of VA diagnoses for major causes of death was measured using sensitivity, specificity and positive predictive value (PPV).ResultsThe sensitivity of VA was at least 75% in identifying some leading CODs such as stroke, road traffic accidents and several site-specific cancers. However, sensitivity was less than 50% for other important causes including ischemic heart disease, chronic obstructive pulmonary diseases, and diabetes. Overall, there was 57% agreement between UCOD from VA and MR, which increased to 76% when multiple causes from VA were compared to UCOD from MR.ConclusionsOur findings suggest that VA is a valid method to ascertain UCOD in contexts such as Vietnam. Furthermore, within cultural contexts in which patients prefer to die at home instead of a healthcare facility, using the available MRs as the gold standard may be meaningful to the extent that recall bias from the interval between last hospital discharge and death can be minimized. Therefore, future studies should evaluate validity of MRs as a gold standard for VA studies in contexts similar to the Vietnamese context.

Project description:BackgroundThe World Health Organization (WHO) aims to reduce tuberculosis (TB) deaths by 95% by 2035; tracking progress requires accurate measurement of TB mortality. International Classification of Diseases (ICD) codes do not differentiate between HIV-associated TB and HIV more generally. Verbal autopsy (VA) is used to estimate cause of death (CoD) patterns but has mostly been validated against a suboptimal gold standard for HIV and TB. This study, conducted among HIV-positive adults, aimed to estimate the accuracy of VA in ascertaining TB and HIV CoD when compared to a reference standard derived from a variety of clinical sources including, in some, minimally-invasive autopsy (MIA).Methods and findingsDecedents were enrolled into a trial of empirical TB treatment or a cohort exploring diagnostic algorithms for TB in South Africa. The WHO 2012 instrument was used; VA CoD were assigned using physician-certified VA (PCVA), InterVA-4, and SmartVA-Analyze. Reference CoD were assigned using MIA, research, and health facility data, as available. 259 VAs were completed: 147 (57%) decedents were female; median age was 39 (interquartile range [IQR] 33-47) years and CD4 count 51 (IQR 22-102) cells/μL. Compared to reference CoD that included MIA (n = 34), VA underestimated mortality due to HIV/AIDS (94% reference, 74% PCVA, 47% InterVA-4, and 41% SmartVA-Analyze; chance-corrected concordance [CCC] 0.71, 0.42, and 0.31, respectively) and HIV-associated TB (41% reference, 32% PCVA; CCC 0.23). For individual decedents, all VA methods agreed poorly with reference CoD that did not include MIA (n = 259; overall CCC 0.14, 0.06, and 0.15 for PCVA, InterVA-4, and SmartVA-Analyze); agreement was better at population level (cause-specific mortality fraction accuracy 0.78, 0.61, and 0.57, for the three methods, respectively).ConclusionsCurrent VA methods underestimate mortality due to HIV-associated TB. ICD and VA methods need modifications that allow for more specific evaluation of HIV-related deaths and direct estimation of mortality due to HIV-associated TB.

Project description:INTRODUCTION:There is a growing recognition of the importance of developing learning health systems which can engage all stakeholders in cycles of evidence generation, reflection, action and learning from action to deal with adaptive problems. There is however limited evaluative evidence of approaches to developing or strengthening such systems, particularly in low-income and middle-income settings. In this protocol, we aim to contribute to developing and sharing knowledge on models of building collaborative learning platforms through our evaluation of the Verbal Autopsy with Participatory Action Research (VAPAR) programme. METHODS AND ANALYSIS:The evaluation takes a participatory approach, focussed on joint learning on whether and how VAPAR contributes to its aims, and what can be learnt for this and similar settings. A realist-informed theory of change was developed by the research team as part of a broader collaboration with other stakeholders. The evaluation will draw on a wide variety of perspectives and data, including programme data and secondary data. This will be supplemented by in-depth interviews and workshops at the end of each cycle to probe the different domains, understand changes to the positions of different actors within the local health system and feedback into improved learning and action in the next cycle. Quantitative data such as verbal autopsy will be analysed for significant trends in health indicators for different population groups. However, the bulk of the data will be qualitative and will be analysed thematically. ETHICS AND DISSEMINATION:Ethics in participatory approaches include a careful focus on the power relationships within the group, such that all groups are given voice and influence, in addition to the usual considerations of informed participation. Within the programme, we will focus on reflexivity, relationship building, two-way learning and learning from failure to reduce power imbalances and mitigate against a blame culture. Local engagement and change will be prioritised in dissemination.

Dataset Information

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods.

Background

Methods

Results

Conclusions

Publications

Usefulness of the Population Health Metrics Research Consortium gold standard verbal autopsy data for general verbal autopsy methods.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets