Dataset Information

Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review.

ABSTRACT:

Background

Computer-coded verbal autopsy (CCVA) methods to assign causes of death (CODs) for medically unattended deaths have been proposed as an alternative to physician-certified verbal autopsy (PCVA). We conducted a systematic review of 19 published comparison studies (from 684 evaluated), most of which used hospital-based deaths as the reference standard. We assessed the performance of PCVA and five CCVA methods: Random Forest, Tariff, InterVA, King-Lu, and Simplified Symptom Pattern.

Methods

The reviewed studies assessed methods' performance through various metrics: sensitivity, specificity, and chance-corrected concordance for coding individual deaths, and cause-specific mortality fraction (CSMF) error and CSMF accuracy at the population level. These results were summarized into means, medians, and ranges.

Results

The 19 studies ranged from 200 to 50,000 deaths per study (total over 116,000 deaths). Sensitivity of PCVA versus hospital-assigned COD varied widely by cause, but showed consistently high specificity. PCVA and CCVA methods had an overall chance-corrected concordance of about 50% or lower, across all ages and CODs. At the population level, the relative CSMF error between PCVA and hospital-based deaths indicated good performance for most CODs. Random Forest had the best CSMF accuracy performance, followed closely by PCVA and the other CCVA methods, but with lower values for InterVA-3.

Conclusions

There is no single best-performing coding method for verbal autopsies across various studies and metrics. There is little current justification for CCVA to replace PCVA, particularly as physician diagnosis remains the worldwide standard for clinical diagnosis on live patients. Further assessments and large accessible datasets on which to train and test combinations of methods are required, particularly for rural deaths without medical attention.

SUBMITTER: Leitao J

PROVIDER: S-EPMC3912516 | biostudies-literature | 2014 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review.

Leitao Jordana J Desai Nikita N Aleksandrowicz Lukasz L Byass Peter P Miasnikof Pierre P Tollman Stephen S Alam Dewan D Lu Ying Y Rathi Suresh Kumar SK Singh Abhishek A Suraweera Wilson W Ram Faujdar F Jha Prabhat P

BMC medicine 20140204

<h4>Background</h4>Computer-coded verbal autopsy (CCVA) methods to assign causes of death (CODs) for medically unattended deaths have been proposed as an alternative to physician-certified verbal autopsy (PCVA). We conducted a systematic review of 19 published comparison studies (from 684 evaluated), most of which used hospital-based deaths as the reference standard. We assessed the performance of PCVA and five CCVA methods: Random Forest, Tariff, InterVA, King-Lu, and Simplified Symptom Pattern ...[more]

PMID: 24495312

Similar Datasets

Project description:BackgroundThe South African national cause of death validation (NCODV 2017/18) project collected a national sample of verbal autopsies (VA) with cause of death (COD) assignment by physician-coded VA (PCVA) and computer-coded VA (CCVA).ObjectiveThe performance of three CCVA algorithms (InterVA-5, InSilicoVA and Tariff 2.0) in assigning a COD was compared with PCVA (reference standard).MethodsSeven performance metrics assessed individual and population level agreement of COD assignment by age, sex and place of death subgroups. Positive predictive value (PPV), sensitivity, overall agreement, kappa, and chance corrected concordance (CCC) assessed individual level agreement. Cause-specific mortality fraction (CSMF) accuracy and Spearman's rank correlation assessed population level agreement.ResultsA total of 5386 VA records were analysed. PCVA and CCVAs all identified HIV/AIDS as the leading COD. CCVA PPV and sensitivity, based on confidence intervals, were comparable except for HIV/AIDS, TB, maternal, diabetes mellitus, other cancers, and some injuries. CCVAs performed well for identifying perinatal deaths, road traffic accidents, suicide and homicide but poorly for pneumonia, other infectious diseases and renal failure. Overall agreement between CCVAs and PCVA for the top single cause (48.2-51.6) indicated comparable weak agreement between methods. Overall agreement, for the top three causes showed moderate agreement for InterVA (70.9) and InSilicoVA (73.8). Agreement based on kappa (-0.05-0.49)and CCC (0.06-0.43) was weak to none for all algorithms and groups. CCVAs had moderate to strong agreement for CSMF accuracy, with InterVA-5 highest for neonates (0.90), Tariff 2.0 highest for adults (0.89) and males (0.84), and InSilicoVA highest for females (0.88), elders (0.83) and out-of-facility deaths (0.85). Rank correlation indicated moderate agreement for adults (0.75-0.79).ConclusionsWhilst CCVAs identified HIV/AIDS as the leading COD, consistent with PCVA, there is scope for improving the algorithms for use in South Africa.

Project description:BackgroundPhysician-coded verbal autopsy (PCVA) is the most widely used method to determine causes of death (CODs) in countries where medical certification of death is uncommon. Computer-coded verbal autopsy (CCVA) methods have been proposed as a faster and cheaper alternative to PCVA, though they have not been widely compared to PCVA or to each other.MethodsWe compared the performance of open-source random forest, open-source tariff method, InterVA-4, and the King-Lu method to PCVA on five datasets comprising over 24,000 verbal autopsies from low- and middle-income countries. Metrics to assess performance were positive predictive value and partial chance-corrected concordance at the individual level, and cause-specific mortality fraction accuracy and cause-specific mortality fraction error at the population level.ResultsThe positive predictive value for the most probable COD predicted by the four CCVA methods averaged about 43% to 44% across the datasets. The average positive predictive value improved for the top three most probable CODs, with greater improvements for open-source random forest (69%) and open-source tariff method (68%) than for InterVA-4 (62%). The average partial chance-corrected concordance for the most probable COD predicted by the open-source random forest, open-source tariff method and InterVA-4 were 41%, 40% and 41%, respectively, with better results for the top three most probable CODs. Performance generally improved with larger datasets. At the population level, the King-Lu method had the highest average cause-specific mortality fraction accuracy across all five datasets (91%), followed by InterVA-4 (72% across three datasets), open-source random forest (71%) and open-source tariff method (54%).ConclusionsOn an individual level, no single method was able to replicate the physician assignment of COD more than about half the time. At the population level, the King-Lu method was the best method to estimate cause-specific mortality fractions, though it does not assign individual CODs. Future testing should focus on combining different computer-coded verbal autopsy tools, paired with PCVA strengths. This includes using open-source tools applied to larger and varied datasets (especially those including a random sample of deaths drawn from the population), so as to establish the performance for age- and sex-specific CODs.

Project description:BackgroundThe most common method for determining cause of death is certification by physicians based either on available medical records, or where such data are not available, through verbal autopsy (VA). The physician-certification approach is costly and inconvenient; however, recent work shows the potential of a computer-based probabilistic model (InterVA) to interpret verbal autopsy data in a more convenient, consistent, and rapid way. In this study we validate separately both physician-certified verbal autopsy (PCVA) and the InterVA probabilistic model against hospital cause of death (HCOD) in adults dying in a district hospital on the coast of Kenya.MethodsBetween March 2007 and June 2010, VA interviews were conducted for 145 adult deaths that occurred at Kilifi District Hospital. The VA data were reviewed by a physician and the cause of death established. A range of indicators (including age, gender, physical signs and symptoms, pregnancy status, medical history, and the circumstances of death) from the VA forms were included in the InterVA for interpretation. Cause-specific mortality fractions (CSMF), Cohen's kappa (κ) statistic, receiver operating characteristic (ROC) curves, sensitivity, specificity, and positive predictive values were applied to compare agreement between PCVA, InterVA, and HCOD.ResultsHCOD, InterVA, and PCVA yielded the same top five underlying causes of adult deaths. The InterVA overestimated tuberculosis as a cause of death compared to the HCOD. On the other hand, PCVA overestimated diabetes. Overall, CSMF for the five major cause groups by the InterVA, PCVA, and HCOD were 70%, 65%, and 60%, respectively. PCVA versus HCOD yielded a higher kappa value (κ = 0.52, 95% confidence interval [CI]: 0.48, 0.54) than the InterVA versus HCOD which yielded a kappa (κ) value of 0.32 (95% CI: 0.30, 0.38). Overall, (κ) agreement across the three methods was 0.41 (95% CI: 0.37, 0.48). The areas under the ROC curves were 0.82 for InterVA and 0.88 for PCVA. The observed sensitivities and specificities across the five major causes of death varied from 43% to 100% and 87% to 99%, respectively, for the InterVA/PCVA against the HCOD.ConclusionBoth the InterVA and PCVA compared well with the HCOD at a population level and determined the top five underlying causes of death in the rural community of Kilifi. We hope that our study, albeit small, provides new and useful data that will stimulate further definitive work on methods of interpreting VA data.

Project description:BackgroundThis study was one of a set of verbal autopsy investigations undertaken by the WHO/UNCEF-supported Child Health Epidemiology Reference Group (CHERG) to derive direct estimates of the causes of neonatal and child deaths in high priority countries of sub-Saharan Africa. The objective of the study was to determine the cause distributions of neonatal (0-27 days) and child (1-59 months) mortality in Niger.MethodsVerbal autopsy interviews were conducted of random samples of 453 neonatal deaths and 620 child deaths from 2007 to 2010 identified by the 2011 Niger National Mortality Survey. The cause of each death was assigned using two methods: computerized expert algorithms arranged in a hierarchy and physician completion of a death certificate for each child. The findings of the two methods were compared to each other, and plausibility checks were conducted to assess which is the preferred method. Comparison of some direct measures from this study with CHERG modeled cause of death estimates are discussed.FindingsThe cause distributions of neonatal deaths as determined by expert algorithms and the physician were similar, with the same top three causes by both methods and all but two other causes within one rank of each other. Although child causes of death differed more, the reasons often could be discerned by analyzing algorithmic criteria alongside the physician's application of required minimal diagnostic criteria. Including all algorithmic (primary and co-morbid) and physician (direct, underlying and contributing) diagnoses in the comparison minimized the differences, with kappa coefficients greater than 0.40 for five of 11 neonatal diagnoses and nine of 13 child diagnoses. By algorithmic diagnosis, early onset neonatal infection was significantly associated (?(2)?=?13.2, P?<?0.001) with maternal infection, and the geographic distribution of child meningitis deaths closely corresponded with that for meningitis surveillance cases and deaths.ConclusionsVerbal autopsy conducted in the context of a national mortality survey can provide useful estimates of the cause distributions of neonatal and child deaths. While the current study found reasonable agreement between the expert algorithm and physician analyses, it also demonstrated greater plausibility for two algorithmic diagnoses and validation work is needed to ascertain the findings. Direct, large-scale measurement of causes of death complement, can strengthen, and in some settings may be preferred over modeled estimates.

Project description:UNLABELLED: BACKGROUND:Through application of the verbal autopsy (VA) approach, trained fieldworkers collect information about the probable cause of death (COD) by using a standardized questionnaire to interview family members who were present at the time of death. The physician-certified VA (PCVA), an independent review of this questionnaire data by up to three physicians trained in VA coding, is currently recommended by the World Health Organization (WHO) and is widely used in the INDEPTH Network. Even given its appropriateness in these contexts, a large percentage of causes of death assigned by VAs remains undetermined. As physicians often do not agree upon a final COD classification, there remains substantial room to improve the standard VA method, potentially leading to a reduction in physician discordance in COD coding. METHODS:We present an extension of the current method of PCVA and compare it to the standard WHO-recommended procedure. We used VA data collected in the Nouna Health and Demographic Surveillance Site (NHDSS) between 2009 and 2010 using a locally-adapted version of an INDEPTH standard verbal autopsy questionnaire. Until 2009, physicians in the NHDSS followed the WHO method (Method 1). As an extension of Method 1, starting in 2010, the use of a panel of physicians was added to the coding process in the case where a third physician's final conclusions resulted in an undetermined COD (Method 2). Two independent samples of VA questionnaires were compared for the year 2009 (using Method 1) and the year 2010 (using Method 2). RESULTS:The WHO-recommended method used for 2009 yielded a high level of undetermined CODs, where the final coding was "undetermined" in 50.8% of all questionnaires due to disagreement among participating physicians (Method 1). By introducing a panel of physicians in 2010 for cases where the principal physicians disagreed on the cause of death, the revised method significantly reduced the proportion of undetermined CODs to 1.5% (Method 2). CONCLUSIONS:As the extended method of PCVA significantly improved the accuracy of the VA procedure, we suggest the adoption of this method for those countries where alternatives like computer-based VA coding are not available. Based on the results of our study, further research should be pursued.

Project description:BackgroundPhysician review of a verbal autopsy (VA) and completion of a death certificate remains the most widely used approach for VA analysis. This study provides new evidence about the performance of physician-certified verbal autopsy (PCVA) using defined clinical diagnostic criteria as a gold standard for a multisite sample of 12,542 VAs. The study was also designed to analyze issues related to PCVA, such as the impact of a second physician reader on the cause of death assigned, the variation in performance with and without household recall of health care experience (HCE), and the importance of local information for physicians reading VAs.MethodsThe certification was performed by 24 physicians. The assignment of VA was random and blinded. Each VA was certified by one physician. Half of the VAs were reviewed by a different physician with household recall of health care experience included. The completed death certificate was processed for automated ICD-10 coding of the underlying cause of death. PCVA was compared to gold standard cause of death assignment based on strictly defined clinical diagnostic criteria that are part of the Population Health Metrics Research Consortium (PHMRC) gold standard verbal autopsy study.ResultsFor individual cause assignment, the overall chance-corrected concordance for PCVA against the gold standard cause of death is less than 50%, with substantial variability by cause and physician. Physicians assign the correct cause around 30% of the time without HCE, and addition of HCE improves performance in adults to 45% and slightly higher in children to 48%. Physicians estimate cause-specific mortality fractions (CSMFs) with considerable error for adults, children, and neonates. Only for neonates for a cause list of six causes with HCE is accuracy above 0.7. In all three age groups, CSMF accuracy improves when household recall of health care experience is available.ConclusionsResults show that physician coding for cause of death assignment may not be as robust as previously thought. The time and cost required to initially collect the verbal autopsies must be considered in addition to the analysis, as well as the impact of diverting physicians from servicing immediate health needs in a population to review VAs. All of these considerations highlight the importance and urgency of developing better methods to more reliably analyze past and future verbal autopsies to obtain the highest quality mortality data from populations without reliable death certification.

Project description:BackgroundVerbal autopsy (VA) plays a vital role in providing cause-of-death information in places where such information is not available. Many low-/middle-income countries (LMICs) including Tanzania are still struggling to yield quality and adequate cause-of-death data for Civil Registration and Vital Statistics (CRVS).ObjectiveTo highlight challenges and recommendations for VA implementation to support LMICs yield quality and adequate mortality statistics for informed decisions on healthcare interventions.DesignCross-sectional study.Study settingIringa region in Tanzania.Participants41 people including 33 community health workers, 1 VA national coordinator, 5 national task force members, 1 VA regional coordinator and 1 member of the VA data management team.ResultsThe perceived challenges of key informants include a weak death notification system, lengthy VA questionnaire, poor data quality and inconsistent responses, lack of clarity in the inclusion criteria, poor commitment to roles and responsibilities, poor coordination, poor financial mechanism and no or delayed feedback to VA implementers. Based on these findings, we recommend the following strategies for effective adaptation and use of VAs: (1) reinforce or implement legislative procedures towards the legal requirement for death notification. (2) Engage key stakeholders in the overall implementation of VAs. (3) Build capacity for data collection, monitoring, processing and use of VA data. (4) Improve the VA questionnaire and quality control mechanism for optimal use in data collection. (5) Create sustainable financing mechanisms and institutionalisation of VA implementation. (6) Integrating VA Implementation in CRVS.ConclusionEffective VA implementation demands through planning, stakeholder engagement, upskilling of local experts and fair compensation for interviewers. Such coordinated endeavours will overcome systemic, technical and behavioural challenges hindering VA's successful implementation.

Project description:IntroductionEach year, more than 10 million children younger than five years of age die. The large majority of these deaths occur in the developing world. The verbal autopsy (VA) is a tool designed to ascertain cause of death in such settings. While VA has been validated against hospital diagnosed cause of death, there has been no research conducted to better understand the factors that may influence individual physicians in determining cause of death from VA.Methodology/principal findingsThis study uses data from over 27,000 neonatal and childhood deaths from The Million Death Study in which 6.3 million people in India were monitored for vital status between 1998 and 2003. The main outcome variable was physician agreement or disagreement of category of death and the variables were assessed for association using the kappa statistic, univariate and multivariate logistic regression using a conceptual hierarchical model, and a sensitivity and specificity analysis using the final VA category of mortality as the gold standard. The main variables found to be significantly associated with increased physician agreement included older ages and male gender of the deceased. When taking into account confounding factors in the multivariate analysis, we did not find consistent significant differences in physician agreement based on the death being in a rural or urban area, at home or in a health care facility, registered or not, or the respondent's gender, religion, relationship to the deceased, or whether or not the respondent lived with the deceased.Conclusions/significanceFactors influencing physician agreement/disagreement to the greatest degree are the gender and age of the deceased; specifically, physicians tend to be less likely to agree on a common category of death in female children and in younger ages, particularly neonates. Additional training of physician reviewers and continued adaptation of the VA itself, with a focus on gender and age of the deceased, may be useful in increasing rates of physician agreement in these groups.

Dataset Information

Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review.

Background

Methods

Results

Conclusions

Publications

Comparison of physician-certified verbal autopsy with computer-coded verbal autopsy for cause of death assignment in hospitalized patients in low- and middle-income countries: systematic review.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets