Comparison of machine learning algorithms applied to symptoms to determine infectious causes of death in children: national survey of 18,000 verbal autopsies in the Million Death Study in India.
Ontology highlight
ABSTRACT: Background
Machine learning (ML) algorithms have been successfully employed for prediction of outcomes in clinical research. In this study, we have explored the application of ML-based algorithms to predict cause of death (CoD) from verbal autopsy records available through the Million Death Study (MDS).Methods
From MDS, 18826 unique childhood deaths at ages 1-59 months during the time period 2004-13 were selected for generating the prediction models of which over 70% of deaths were caused by six infectious diseases (pneumonia, diarrhoeal diseases, malaria, fever of unknown origin, meningitis/encephalitis, and measles). Six popular ML-based algorithms such as support vector machine, gradient boosting modeling, C5.0, artificial neural network, k-nearest neighbor, classification and regression tree were used for building the CoD prediction models.Results
SVM algorithm was the best performer with a prediction accuracy of over 0.8. The highest accuracy was found for diarrhoeal diseases (accuracy = 0.97) and the lowest was for meningitis/encephalitis (accuracy = 0.80). The top signs/symptoms for classification of these CoDs were also extracted for each of the diseases. A combination of signs/symptoms presented by the deceased individual can effectively lead to the CoD diagnosis.Conclusions
Overall, this study affirms that verbal autopsy tools are efficient in CoD diagnosis and that automated classification parameters captured through ML could be added to verbal autopsies to improve classification of causes of death.
SUBMITTER: Idicula-Thomas S
PROVIDER: S-EPMC8488544 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:In regions without complete-coverage civil registration and vital statistics systems there is uncertainty about even the most basic demographic indicators. In such regions the majority of deaths occur outside hospitals and are not recorded. Worldwide, fewer than one-third of deaths are assigned a cause, with the least information available from the most impoverished nations. In populations like this, verbal autopsy (VA) is a commonly used tool to assess cause of death and estimate cause-specific mortality rates and the distribution of deaths by cause. VA uses an interview with caregivers of the decedent to elicit data describing the signs and symptoms leading up to the death. This paper develops a new statistical tool known as InSilicoVA to classify cause of death using information acquired through VA. InSilicoVA shares uncertainty between cause of death assignments for specific individuals and the distribution of deaths by cause across the population. Using side-by-side comparisons with both observed and simulated data, we demonstrate that InSilicoVA has distinct advantages compared to currently available methods.
| S-EPMC5154628 | biostudies-literature
Project description:BackgroundFollowing data access and storage concerns, Government of India transferred the management of its Sample Registration System (SRS) based mortality surveillance (formerly known as the Million Death Study) to an Indian agency. This paper introduces the new system, challenges it faced and its vision for future.MethodsThe All India Institute of Medical Sciences (AIIMS), New Delhi, the new nodal agency, established the "Mortality in India Established through Verbal Autopsy" (MINErVA) platform with state level partners across India in November 2017. The network in its first three years has undertaken capacity building of supervisors conducting verbal autopsy under the SRS, established a panel of trained physician reviewers and developed three IT-based platforms for training, quality control and coding. Coding of VA forms started from January 2015 onwards, and the cause specific mortality fractions (CSMF) of the first 14 185 adult verbal autopsy (VA) records for 2015 were compared with earlier published data for 2010-2013 to check for continuity of system performance.ResultsThe network consists of 25 institutions and a panel of 676 trained physician reviewers. 916 supervisors have been trained in conducting verbal autopsies. More than 75 000 VA forms have been coded to date. The median time taken for finalizing cause of death on the coding platform is 37 days. The level of physician agreement (67%) and proportion of VA forms requiring adjudication (12%) are consistent with published literature. Preliminary CSMF estimates for 2015 were comparable with those for 2010-2013 and identified same top ten causes of death. In addition to the delay, two major challenges identified for coding were language proficiency of physician reviewers vis-à-vis language of narratives and quality of verbal autopsies. While an initial strategic decision was made to consolidate the system to ensure continuity, future vision of the network is to move towards technology-based solutions including electronic data capture of VAs and its analysis and improving the use of mortality data in decision making.ConclusionMINErVA network is now fully functional and is moving towards achieving global standards. It provides valuable lessons for other developing countries to establish their own mortality surveillance systems.
| S-EPMC7688192 | biostudies-literature
Project description:The distribution of deaths by cause provides crucial information for public health planning, response and evaluation. About 60% of deaths globally are not registered or given a cause, limiting our ability to understand disease epidemiology. Verbal autopsy (VA) surveys are increasingly used in such settings to collect information on the signs, symptoms and medical history of people who have recently died. This article develops a novel Bayesian method for estimation of population distributions of deaths by cause using verbal autopsy data. The proposed approach is based on a multivariate probit model where associations among items in questionnaires are flexibly induced by latent factors. Using the Population Health Metrics Research Consortium labeled data that include both VA and medically certified causes of death, we assess performance of the proposed method. Further, we estimate important questionnaire items that are highly associated with causes of death. This framework provides insights that will simplify future data.
| S-EPMC7845920 | biostudies-literature
Project description:BackgroundThe Indian Sample Registration System (SRS) with verbal autopsy methods provides estimations of cause specific mortality for maternal deaths, where the majority of deaths occur at home, unregistered. We aim to examine factors that influence physician agreement and coding choices in assigning causes of death from verbal autopsies.Methodology/principal findingsAmong adult deaths identified in the SRS, pregnancy-related deaths recorded in 2001-2003 were assigned ICD-10 codes by two independent physicians. Inter-rater reliability was estimated using Landis Koch Kappa classification ?0.4--poor to fair agreement; >0.4 ?0.6--moderate agreement; >0.6 ?0.8--substantial agreement; >8--high agreement. We identified factors associated with physician agreement using multivariate logistic regression. A central consensus panel reviewed cases for errors and reclassified as needed based on 2011 ICD-10 coding guidelines. Of 1130 pregnancy-related deaths, 1040 were assigned ICD-10 codes by two physicians. We found substantial agreement regardless of the woman's residence, whether the death was registered, religion, respondent's or deceased's education, age, hospital admission or gestational age. Physician agreement was not influenced by the above variables, with the exception of greater agreement in cases where the respondent did not live with the deceased, or early gestational age at the time of death. A central consensus panel reviewed all cases and recoded 10% of cases due to insufficient use of information in the verbal autopsy by the coding physicians and rationale for this reclassification are discussed.ConclusionIn the absence of complete vital registration and universal healthcare services, physician coded verbal autopsies continues to be heavily relied upon to ascertain pregnancy-related death. From this study, two independent physicians had good inter-rater reliability for assigning pregnancy-related causes of death in a nationally-represented sample, and physician coding does not appear to be heavily influenced by case characteristics or demographics.
| S-EPMC3314652 | biostudies-literature
Project description:BACKGROUND:Verbal autopsies with physician assignment of cause of death (COD) are commonly used in settings where medical certification of deaths is uncommon. It remains unanswered if automated algorithms can replace physician assignment. METHODS:We randomized verbal autopsy interviews for deaths in 117 villages in rural India to either physician or automated COD assignment. Twenty-four trained lay (non-medical) surveyors applied the allocated method using a laptop-based electronic system. Two of 25 physicians were allocated randomly to independently code the deaths in the physician assignment arm. Six algorithms (Naïve Bayes Classifier (NBC), King-Lu, InSilicoVA, InSilicoVA-NT, InterVA-4, and SmartVA) coded each death in the automated arm. The primary outcome was concordance with the COD distribution in the standard physician-assigned arm. Four thousand six hundred fifty-one (4651) deaths were allocated to physician (standard), and 4723 to automated arms. RESULTS:The two arms were nearly identical in demographics and key symptom patterns. The average concordances of automated algorithms with the standard were 62%, 56%, and 59% for adult, child, and neonatal deaths, respectively. Automated algorithms showed inconsistent results, even for causes that are relatively easy to identify such as road traffic injuries. Automated algorithms underestimated the number of cancer and suicide deaths in adults and overestimated other injuries in adults and children. Across all ages, average weighted concordance with the standard was 62% (range 79-45%) with the best to worst ranking automated algorithms being InterVA-4, InSilicoVA-NT, InSilicoVA, SmartVA, NBC, and King-Lu. Individual-level sensitivity for causes of adult deaths in the automated arm was low between the algorithms but high between two independent physicians in the physician arm. CONCLUSIONS:While desirable, automated algorithms require further development and rigorous evaluation. Lay reporting of deaths paired with physician COD assignment of verbal autopsies, despite some limitations, remains a practicable method to document the patterns of mortality reliably for unattended deaths. TRIAL REGISTRATION:ClinicalTrials.gov , NCT02810366. Submitted on 11 April 2016.
| S-EPMC6595581 | biostudies-literature
Project description:Of patients with multidrug-resistant tuberculosis (MDR TB), <50% complete treatment. Most treatment failures for patients with MDR TB are due to death during TB treatment. We sought to determine the proportion of deaths during MDR TB treatment attributable to TB itself. We used a structured verbal autopsy tool to interview family members of patients who died during MDR TB treatment in India during January-December 2016. A committee triangulated information from verbal autopsy, death certificate, or other medical records available with the family members to ascertain the underlying cause of death. For 66% of patient deaths (47/71), TB was the underlying cause of death. We assigned TB as the underlying cause of death for an additional 6 patients who died of suicide and 2 of pulmonary embolism. Deaths during TB treatment signify program failure; accurately determining the cause of death is the first step to designing appropriate, timely interventions to prevent premature deaths.
| S-EPMC5823351 | biostudies-literature
Project description:BackgroundInterVA is a widely disseminated tool for cause of death attribution using information from verbal autopsies. Several studies have attempted to validate the concordance and accuracy of the tool, but the main limitation of these studies is that they compare cause of death as ascertained through hospital record review or hospital discharge diagnosis with the results of InterVA. This study provides a unique opportunity to assess the performance of InterVA compared to physician-certified verbal autopsies (PCVA) and alternative automated methods for analysis.MethodsUsing clinical diagnostic gold standards to select 12,542 verbal autopsy cases, we assessed the performance of InterVA on both an individual and population level and compared the results to PCVA, conducting analyses separately for adults, children, and neonates. Following the recommendation of Murray et al., we randomly varied the cause composition over 500 test datasets to understand the performance of the tool in different settings. We also contrasted InterVA with an alternative Bayesian method, Simplified Symptom Pattern (SSP), to understand the strengths and weaknesses of the tool.ResultsAcross all age groups, InterVA performs worse than PCVA, both on an individual and population level. On an individual level, InterVA achieved a chance-corrected concordance of 24.2% for adults, 24.9% for children, and 6.3% for neonates (excluding free text, considering one cause selection). On a population level, InterVA achieved a cause-specific mortality fraction accuracy of 0.546 for adults, 0.504 for children, and 0.404 for neonates. The comparison to SSP revealed four specific characteristics that lead to superior performance of SSP. Increases in chance-corrected concordance are attained by developing cause-by-cause models (2%), using all items as opposed to only the ones that mapped to InterVA items (7%), assigning probabilities to clusters of symptoms (6%), and using empirical as opposed to expert probabilities (up to 8%).ConclusionsGiven the widespread use of verbal autopsy for understanding the burden of disease and for setting health intervention priorities in areas that lack reliable vital registrations systems, accurate analysis of verbal autopsies is essential. While InterVA is an affordable and available mechanism for assigning causes of death using verbal autopsies, users should be aware of its suboptimal performance relative to other methods.
| S-EPMC3160943 | biostudies-literature
Project description:BackgroundRecently, a new algorithm for automatic computer certification of verbal autopsy data named InSilicoVA was published. The authors presented their algorithm as a statistical method and assessed its performance using a single set of model predictors and one age group.MethodsWe perform a standard procedure for analyzing the predictive accuracy of verbal autopsy classification methods using the same data and the publicly available implementation of the algorithm released by the authors. We extend the original analysis to include children and neonates, instead of only adults, and test accuracy using different sets of predictors, including the set used in the original paper and a set that matches the released software.ResultsThe population-level performance (i.e., predictive accuracy) of the algorithm varied from 2.1 to 37.6% when trained on data preprocessed similarly as in the original study. When trained on data that matched the software default format, the performance ranged from -11.5 to 17.5%. When using the default training data provided, the performance ranged from -59.4 to -38.5%. Overall, the InSilicoVA predictive accuracy was found to be 11.6-8.2 percentage points lower than that of an alternative algorithm. Additionally, the sensitivity for InSilicoVA was consistently lower than that for an alternative diagnostic algorithm (Tariff 2.0), although the specificity was comparable.ConclusionsThe default format and training data provided by the software lead to results that are at best suboptimal, with poor cause-of-death predictive performance. This method is likely to generate erroneous cause of death predictions and, even if properly configured, is not as accurate as alternative automated diagnostic methods.
| S-EPMC5907465 | biostudies-literature
Project description:Population-based information on causes of death (CoD) by age, sex, and area is critical for countries with limited resources to identify and address key public health issues. This study analysed the demographic surveillance and verbal autopsy (VA) data to estimate age- and sex-specific mortality rates and cause-specific mortality fractions in two well-defined rural populations within the demographic surveillance system in Abhoynagar and Mirsarai subdistricts, located in different climatic zones.During 2004-2010, the sample demographic surveillance system registered 1,384 deaths in Abhoynagar and 1,847 deaths in Mirsarai. Trained interviewers interviewed the main caretaker of the deceased with standard VA questionnaires to record signs and symptoms of diseases or conditions that led to death and health care experiences before death. The computer-automated InterVA-4 method was used to analyse VAs to determine probable CoD.Age- and sex-specific death rates revealed a higher neonatal mortality rate in Abhoynagar than Mirsarai, and death rates and sex ratios of male to female death rates were higher in the ages after infancy. Communicable diseases (CDs) accounted for 16.7% of all deaths in Abhoynagar and 21.2% in Mirsarai--the difference was due mostly to more deaths from acute respiratory infections, pneumonia, and tuberculosis in Mirsarai. Non-communicable diseases (NCDs) accounted for 56.2 and 55.3% of deaths in each subdistrict, respectively, with leading causes being stroke (16.5-19.3%), neoplasms (13.2% each), cardiac diseases (8.9-11.6%), chronic obstructive pulmonary diseases (5.1-6.3%), diseases of the digestive system (3.1-4.1%), and diabetes (2.8-3.5%), together accounting for 49.2-51.2% points of the NCD deaths in the two subdistricts. Injury and other external causes accounted for another 7.5-7.7% deaths, with self-harm being higher among females in Abhoynagar.The computer-automated coding of VA to determine CoD reconfirmed that NCDs were the leading CoD with some differences between the sites. Incorporating VA into the national sample vital registration system can help policy makers to identify the leading CoDs for public health planning.
| S-EPMC4220132 | biostudies-literature
Project description:BackgroundVerbal autopsy (VA) has been proposed to determine the cause of death (COD) distributions in settings where most deaths occur without medical attention or certification. We develop performance criteria for VA-based COD systems and apply these to the Registrar General of India's ongoing, nationally-representative Indian Million Death Study (MDS).MethodsPerformance criteria include a low ill-defined proportion of deaths before old age; reproducibility, including consistency of COD distributions with independent resampling; differences in COD distribution of hospital, home, urban or rural deaths; age-, sex- and time-specific plausibility of specific diseases; stability and repeatability of dual physician coding; and the ability of the mortality classification system to capture a wide range of conditions.ResultsThe introduction of the MDS in India reduced the proportion of ill-defined deaths before age 70 years from 13% to 4%. The cause-specific mortality fractions (CSMFs) at ages 5 to 69 years for independently resampled deaths and the MDS were very similar across 19 disease categories. By contrast, CSMFs at these ages differed between hospital and home deaths and between urban and rural deaths. Thus, reliance mostly on urban or hospital data can distort national estimates of CODs. Age-, sex- and time-specific patterns for various diseases were plausible. Initial physician agreement on COD occurred about two-thirds of the time. The MDS COD classification system was able to capture more eligible records than alternative classification systems. By these metrics, the Indian MDS performs well for deaths prior to age 70 years. The key implication for low- and middle-income countries where medical certification of death remains uncommon is to implement COD surveys that randomly sample all deaths, use simple but high-quality field work with built-in resampling, and use electronic rather than paper systems to expedite field work and coding.ConclusionsSimple criteria can evaluate the performance of VA-based COD systems. Despite the misclassification of VA, the MDS demonstrates that national surveys of CODs using VA are an order of magnitude better than the limited COD data previously available.
| S-EPMC3912490 | biostudies-literature