Dataset Information

Paradata analyses to inform population-based survey capture of pregnancy outcomes: EN-INDEPTH study.

ABSTRACT:

Background

Paradata are (timestamped) records tracking the process of (electronic) data collection. We analysed paradata from a large household survey of questions capturing pregnancy outcomes to assess performance (timing and correction processes). We examined how paradata can be used to inform and improve questionnaire design and survey implementation in nationally representative household surveys, the major source for maternal and newborn health data worldwide.

Methods

The EN-INDEPTH cross-sectional population-based survey of women of reproductive age in five Health and Demographic Surveillance System sites (in Bangladesh, Guinea-Bissau, Ethiopia, Ghana, and Uganda) randomly compared two modules to capture pregnancy outcomes: full pregnancy history (FPH) and the standard DHS-7 full birth history (FBH+). We used paradata related to answers recorded on tablets using the Survey Solutions platform. We evaluated the difference in paradata entries between the two reproductive modules and assessed which question characteristics (type, nature, structure) affect answer correction rates, using regression analyses. We also proposed and tested a new classification of answer correction types.

Results

We analysed 3.6 million timestamped entries from 65,768 interviews. 83.7% of all interviews had at least one corrected answer to a question. Of 3.3 million analysed questions, 7.5% had at least one correction. Among corrected questions, the median number of corrections was one, regardless of question characteristics. We classified answer corrections into eight types (no correction, impulsive, flat (simple), zigzag, flat zigzag, missing after correction, missing after flat (zigzag) correction, missing/incomplete). 84.6% of all corrections were judged not to be problematic with a flat (simple) mistake correction. Question characteristics were important predictors of probability to make answer corrections, even after adjusting for respondent's characteristics and location, with interviewer clustering accounted as a fixed effect. Answer correction patterns and types were similar between FPH and FBH+, as well as the overall response duration. Avoiding corrections has the potential to reduce interview duration and reproductive module completion by 0.4?min.

Conclusions

The use of questionnaire paradata has the potential to improve measurement and the resultant quality of electronic data. Identifying sections or specific questions with multiple corrections sheds light on typically hidden challenges in the survey's content, process, and administration, allowing for earlier real-time intervention (e.g.,, questionnaire content revision or additional staff training). Given the size and complexity of paradata, additional time, data management, and programming skills are required to realise its potential.

SUBMITTER: Gordeev VS

PROVIDER: S-EPMC7869213 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Paradata analyses to inform population-based survey capture of pregnancy outcomes: EN-INDEPTH study.

Gordeev Vladimir Sergeevich VS Akuze Joseph J Baschieri Angela A Thysen Sanne M SM Dzabeng Francis F Haider M Moinuddin MM Smuk Melanie M Wild Michael M Lokshin Michael M MM Yitayew Temesgen Azemeraw TA Abebe Solomon Mokonnen SM Natukwatsa Davis D Gyezaho Collins C Amenga-Etego Seeba S Lawn Joy E JE Blencowe Hannah H

Population health metrics 20210208 Suppl 1

<h4>Background</h4>Paradata are (timestamped) records tracking the process of (electronic) data collection. We analysed paradata from a large household survey of questions capturing pregnancy outcomes to assess performance (timing and correction processes). We examined how paradata can be used to inform and improve questionnaire design and survey implementation in nationally representative household surveys, the major source for maternal and newborn health data worldwide.<h4>Methods</h4>The EN-I ...[more]

PMID: 33557853

Similar Datasets

Project description:BackgroundRisks of neonatal death, stillbirth and miscarriage are highest in low- and middle-income countries (LMICs), where data has most gaps and estimates rely on household surveys, dependent on women reporting these events. Underreporting of pregnancy and adverse pregnancy outcomes (APOs) is common, but few studies have investigated barriers to reporting these in LMICs. The EN-INDEPTH multi-country study applied qualitative approaches to explore barriers and enablers to reporting pregnancy and APOs in surveys, including individual, community, cultural and interview level factors.MethodsThe study was conducted in five Health and Demographic Surveillance System sites in Guinea-Bissau, Ethiopia, Uganda, Bangladesh and Ghana. Using an interpretative paradigm and phenomenology methodology, 28 focus group discussions were conducted with 82 EN-INDEPTH survey interviewers and supervisors and 172 women between February and August 2018. Thematic analysis was guided by an a priori codebook.ResultsSurvey interview processes influenced reporting of pregnancy and APOs. Women found questions about APOs intrusive and of unclear relevance. Across all sites, sociocultural and spiritual beliefs were major barriers to women reporting pregnancy, due to fear that harm would come to their baby. We identified several factors affecting reporting of APOs including reluctance to speak about sad memories and variation in recognition of the baby's value, especially for APOs at earlier gestation. Overlaps in local understanding and terminology for APOs may also contribute to misreporting, for example between miscarriages and stillbirths. Interviewers' skills and training were the keys to enabling respondents to open up, as was privacy during interviews.ConclusionSociocultural beliefs and psycho-social impacts of APOs play a large part in underreporting these events. Interviewers' skills, careful tool development and translation are the keys to obtaining accurate information. Reporting could be improved with clearer explanations of survey purpose and benefits to respondents and enhanced interviewer training on probing, building rapport and empathy.

Project description:BackgroundHousehold surveys remain important sources of stillbirth data, but omission and misclassification are common. Classifying adverse pregnancy outcomes as stillbirths requires accurate reporting of vital status at birth and gestational age or birthweight for every pregnancy. Further categorisation, e.g. by sex, or timing (intrapartum/antepartum) improves data to understand and prevent stillbirth.MethodsWe undertook a cross-sectional population-based survey of women of reproductive age in five health and demographic surveillance system sites in Bangladesh, Ethiopia, Ghana, Guinea-Bissau and Uganda (2017-2018). All women answered a full birth history with pregnancy loss questions (FBH+) or a full pregnancy history (FPH). A sub-sample across both groups were asked additional stillbirth questions. Questions were evaluated using descriptive measures. Using an interpretative paradigm and phenomenology methodology, focus group discussions with women exploring barriers to reporting birthweight for stillbirths were conducted. Thematic analysis was guided by an a priori codebook.ResultsOverall 69,176 women reported 98,483 livebirths (FBH+) and 102,873 pregnancies (FPH). Additional questions were asked for 1453 stillbirths, 1528 neonatal deaths and 12,620 surviving children born in the 5 years prior to the survey. Completeness was high (> 99%) for existing FBH+/FPH questions on signs of life at birth and gestational age (months). Discordant responses in signs of life at birth between different questions were common; nearly one-quarter classified as stillbirths on FBH+/FPH were reported born alive on additional questions. Availability of information on gestational age (weeks) (58.1%) and birthweight (13.2%) was low amongst stillbirths, and heaping was common. Most women (93.9%) were able to report the sex of their stillborn baby. Response completeness for stillbirth timing (18.3-95.1%) and estimated proportion intrapartum (15.6-90.0%) varied by question and site. Congenital malformations were reported in 3.1% stillbirths. Perceived value in weighing a stillborn baby varied and barriers to weighing at birth a nd knowing birthweight were common.ConclusionsImproving stillbirth data in surveys will require investment in improving the measurement of vital status, gestational age and birthweight by healthcare providers, communication of these with women, and overcoming reporting barriers. Given the large burden and effect on families, improved data must be made available to end preventable stillbirths.

Project description:BackgroundElectronic data collection is increasingly used for household surveys, but factors influencing design and implementation have not been widely studied. The Every Newborn-INDEPTH (EN-INDEPTH) study was a multi-site survey using electronic data collection in five INDEPTH health and demographic surveillance system sites.MethodsWe described experiences and learning involved in the design and implementation of the EN-INDEPTH survey, and undertook six focus group discussions with field and research team to explore their experiences. Thematic analyses were conducted in NVivo12 using an iterative process guided by a priori themes.ResultsFive steps of the process of selecting, adapting and implementing electronic data collection in the EN-INDEPTH study are described. Firstly, we reviewed possible electronic data collection platforms, and selected the World Bank's Survey Solutions® as the most suited for the EN-INDEPTH study. Secondly, the survey questionnaire was coded and translated into local languages, and further context-specific adaptations were made. Thirdly, data collectors were selected and trained using standardised manual. Training varied between 4.5 and 10 days. Fourthly, instruments were piloted in the field and the questionnaires finalised. During data collection, data collectors appreciated the built-in skip patterns and error messages. Internet connection unreliability was a challenge, especially for data synchronisation. For the fifth and final step, data management and analyses, it was considered that data quality was higher and less time was spent on data cleaning. The possibility to use paradata to analyse survey timing and corrections was valued. Synchronisation and data transfer should be given special consideration.ConclusionWe synthesised experiences using electronic data collection in a multi-site household survey, including perceived advantages and challenges. Our recommendations for others considering electronic data collection include ensuring adaptations of tools to local context, piloting/refining the questionnaire in one site first, buying power banks to mitigate against power interruption and paying attention to issues such as GPS tracking and synchronisation, particularly in settings with poor internet connectivity.

Project description:BackgroundAn estimated 40% of pregnancies globally are unintended. Measurement of pregnancy intention in low- and middle-income countries relies heavily on surveys, notably Demographic and Health Surveys (DHS), yet few studies have evaluated survey questions. We examined questions for measuring pregnancy intention, which are already in the DHS, and additional questions and investigated associations with maternity care utilisation and adverse pregnancy outcomes.MethodsThe EN-INDEPTH study surveyed 69,176 women of reproductive age in five Health and Demographic Surveillance System sites in Ghana, Guinea-Bissau, Ethiopia, Uganda and Bangladesh (2017-2018). We investigated responses to survey questions regarding pregnancy intention in two ways: (i) pregnancy-specific intention and (ii) desired-versus-actual family size. We assessed data completeness for each and level of agreement between the two questions, and with future fertility desire. We analysed associations between pregnancy intention and number and timing of antenatal care visits, place of delivery, and stillbirth, neonatal death and low birthweight.ResultsMissing data were <2% in all questions. Responses to pregnancy-specific questions were more consistent with future fertility desire than desired-versus-actual family size responses. Using the pregnancy-specific questions, 7.4% of women who reported their last pregnancy as unwanted reported wanting more children in the future, compared with 45.1% of women in the corresponding desired family size category. Women reporting unintended pregnancies were less likely to attend 4+ antenatal care visits (aOR 0.73, 95% CI 0.64-0.83), have their first visit during the first trimester (aOR 0.71, 95% CI 0.63-0.79), and report stillbirths (aOR 0.57, 95% CI 0.44-0.73) or neonatal deaths (aOR 0.79, 95% CI 0.64-0.96), compared with women reporting intended pregnancies. We found no associations for desired-versus-actual family size intention.ConclusionsWe found the pregnancy-specific intention questions to be a much more reliable assessment of pregnancy intention than the desired-versus-actual family size questions, despite a reluctance to report pregnancies as unwanted rather than mistimed. The additional questions were useful and may complement current DHS questions, although these are not the only possibilities. As women with unintended pregnancies were more likely to miss timely and frequent antenatal care, implementation research is required to improve coverage and quality of care for those women.

Project description:BackgroundLow birthweight (< 2500 g) is an important marker of maternal health and is associated with neonatal mortality, long-term development and chronic diseases. Household surveys remain an important source of population-based birthweight information, notably Demographic and Health Surveys (DHS) and UNICEF's Multiple Indicator Cluster Surveys (MICS); however, data quality concerns remain. Few studies have addressed how to close these gaps in surveys.MethodsThe EN-INDEPTH population-based survey of 69,176 women was undertaken in five Health and Demographic Surveillance System sites (Matlab-Bangladesh, Dabat-Ethiopia, Kintampo-Ghana, Bandim-Guinea-Bissau, IgangaMayuge-Uganda). Responses to existing DHS/MICS birthweight questions on 14,411 livebirths were analysed and estimated adjusted odds ratios (aORs) associated with reporting weighing, birthweight and heaping reported. Twenty-eight focus group discussions with women and interviewers explored barriers and enablers to reporting birthweight.ResultsAlmost all women provided responses to birthweight survey questions, taking on average 0.2 min to answer. Of all babies, 62.4% were weighed at birth, 53.8% reported birthweight and 21.1% provided health cards with recorded birthweight. High levels of heterogeneity were observed between sites. Home births and neonatal deaths were less likely to be weighed at birth (home births aOR 0.03(95%CI 0.02-0.03), neonatal deaths (aOR 0.19(95%CI 0.16-0.24)), and when weighed, actual birthweight was less likely to be known (aOR 0.44(95%CI 0.33-0.58), aOR 0.30(95%CI 0.22-0.41)) compared to facility births and post-neonatal survivors. Increased levels of maternal education were associated with increases in reporting weighing and knowing birthweight. Half of recorded birthweights were heaped on multiples of 500 g. Heaping was more common in IgangaMayuge (aOR 14.91(95%CI 11.37-19.55) and Dabat (aOR 14.25(95%CI 10.13-20.3) compared to Bandim. Recalled birthweights were more heaped than those recorded by card (aOR 2.59(95%CI 2.11-3.19)). A gap analysis showed large missed opportunity between facility birth and known birthweight, especially for neonatal deaths. Qualitative data suggested that knowing their baby's weight was perceived as valuable by women in all sites, but lack of measurement and poor communication, alongside social perceptions and spiritual beliefs surrounding birthweight, impacted women's ability to report birthweight.ConclusionsSubstantial data gaps remain for birthweight data in household surveys, even amongst facility births. Improving the accuracy and recording of birthweights, and better communication with women, for example using health cards, could improve survey birthweight data availability and quality.

Project description:BackgroundPreterm birth (gestational age (GA) <37 weeks) is the leading cause of child mortality worldwide. However, GA is rarely assessed in population-based surveys, the major data source in low/middle-income countries. We examined the performance of new questions to measure GA in household surveys, a subset of which had linked early pregnancy ultrasound GA data.MethodsThe EN-INDEPTH population-based survey of 69,176 women was undertaken (2017-2018) in five Health and Demographic Surveillance System sites in Bangladesh, Ethiopia, Ghana, Guinea-Bissau and Uganda. We included questions regarding GA in months (GAm) for all women and GA in weeks (GAw) for a subset; we also asked if the baby was 'born before expected' to estimate preterm birth rates. Survey data were linked to surveillance data in two sites, and to ultrasound pregnancy dating at <24 weeks in one site. We assessed completeness and quality of reported GA. We examined the validity of estimated preterm birth rates by sensitivity and specificity, over/under-reporting of GAw in survey compared to ultrasound by multinomial logistic regression, and explored perceptions about GA and barriers and enablers to its reporting using focus group discussions (n = 29).ResultsGAm questions were almost universally answered, but heaping on 9 months resulted in underestimation of preterm birth rates. Preference for reporting GAw in even numbers was evident, resulting in heaping at 36 weeks; hence, over-estimating preterm birth rates, except in Matlab where the peak was at 38 weeks. Questions regarding 'born before expected' were answered but gave implausibly low preterm birth rates in most sites. Applying ultrasound as the gold standard in Matlab site, sensitivity of survey-GAw for detecting preterm birth (GAw <37) was 60% and specificity was 93%. Focus group findings suggest that women perceive GA to be important, but usually counted in months. Antenatal care attendance, women's education and health cards may improve reporting.ConclusionsThis is the first published study assessing GA reporting in surveys, compared with the gold standard of ultrasound. Reporting GAw within 5 years' recall is feasible with high completeness, but accuracy is affected by heaping. Compared to ultrasound-GAw, results are reasonably specific, but sensitivity needs to be improved. We propose revised questions based on the study findings for further testing and validation in settings where pregnancy ultrasound data and/or last menstrual period dates/GA recorded in pregnancy are available. Specific training of interviewers is recommended.

Project description:BackgroundHousehold surveys remain important sources of maternal and child health data, but until now, standard surveys such as Demographic and Health Surveys (DHS) have not collected information on maternity care for women who have experienced a stillbirth. Thus, nationally representative data are lacking to inform programmes to address the millions of stillbirths which occur annually.MethodsThe EN-INDEPTH population-based survey of women of reproductive age was undertaken in five Health and Demographic Surveillance System sites in Bangladesh, Ethiopia, Ghana, Guinea-Bissau and Uganda (2017-2018). All women answered a full birth history with additional questions on pregnancy losses (FBH+) or full pregnancy history (FPH). A sub-sample, including all women reporting a recent stillbirth or neonatal death, was asked additional maternity care questions. These were evaluated using descriptive measures. Associations between stillbirth and maternal socio-demographic characteristics, babies' characteristics and maternity care use were assessed using a weighted logistic regression model for women in the FBH+ group.ResultsA total of 15,591 women reporting a birth since 1 January 2012 answered maternity care questions. Completeness was very high (> 99%), with similar proportions of responses for both live and stillbirths. Amongst the 14,991 births in the FBH+ group, poorer wealth status, higher parity, large perceived baby size-at-birth, preterm or post-term birth, birth in a government hospital compared to other locations and vaginal birth were associated with increased risk of stillbirth after adjusting for potential confounding factors. Regarding association with reported postnatal care, women with a stillbirth were more likely to report hospital stays of > 1 day. However, women with a stillbirth were less likely to report having received a postnatal check compared to those with a live birth.ConclusionsWomen who had experienced stillbirth were able to respond to questions about pregnancy and birth, and we found no reason to omit questions to these women in household surveys. Our analysis identified several potentially modifiable factors associated with stillbirth, adding to the evidence-base for policy and action in low- and middle-income contexts. Including these questions in DHS-8 would lead to increased availability of population-level data to inform action to end preventable stillbirths.

Project description:BackgroundBirth registration is a child's first right. Registration of live births, stillbirths and deaths is foundational for national planning. Completeness of birth registration for live births in low- and middle-income countries is measured through population-based surveys which do not currently include completeness of stillbirth or death registration.MethodsThe EN-INDEPTH population-based survey of women of reproductive age was undertaken in five Health and Demographic Surveillance System sites in Bangladesh, Ethiopia, Ghana, Guinea-Bissau and Uganda (2017-2018). In four African sites, we included new/modified questions regarding registration for 1177 stillbirths and 11,881 livebirths (1333 neonatal deaths and 10,548 surviving the neonatal period). Questions were evaluated for completeness of responses, data quality, time to administer and estimates of registration completeness using descriptive statistics. Timing of birth registration, factors associated with non-registration and reported barriers were assessed using descriptive statistics and logistic regression.ResultsAlmost all women, irrespective of their baby's survival, responded to registration questions, taking an average of < 1 min. Reported completeness of birth registration was 30.7% (6.1-53.5%) for babies surviving the neonatal period, compared to 1.7% for neonatal deaths (0.4-5.7%). Women were able to report age at birth registration for 93.6% of babies. Non-registration of babies surviving the neonatal period was significantly higher for home-born children (aOR 1.43 (95% CI 1.27-1.60)) and in Dabat (Ethiopia) (aOR 4.11 (95% CI 3.37-5.01)). Other socio-demographic factors associated with non-registration included younger age of mother, more prior births, little or no education, and lower socio-economic status. Neonatal death registration questions were feasible (100% women responded; only 1% did not know), revealing extremely low completeness with only 1.2% of neonatal deaths reported as registered. Despite > 70% of stillbirths occurring in facilities, only 2.5% were reported as registered.ConclusionsQuestions on birth, stillbirth and death registration were feasible in a household survey. Completeness of birth registration is low in all four sites, but stillbirth and neonatal death registration was very low. Closing the registration gap amongst facility births could increase registration of both livebirths and facility deaths, including stillbirths, but will require co-ordination between civil registration systems and the often over-stretched health sector. Investment and innovation is required to capture birth and especially deaths in both facility and community systems.

Project description:Background:Under-five and maternal mortality were halved in the Millennium Development Goals (MDG) era, with slower reductions for 2.6 million neonatal deaths and 2.6 million stillbirths. The Every Newborn Action Plan aims to accelerate progress towards national targets, and includes an ambitious Measurement Improvement Roadmap. Population-based household surveys, notably Demographic and Health Surveys (DHS) and Multiple Indicator Cluster Surveys, are major sources of population-level data on child mortality in countries with weaker civil registration and vital statistics systems, where over two-thirds of global child deaths occur. To estimate neonatal/child mortality and pregnancy outcomes (stillbirths, miscarriages, birthweight, gestational age) the most common direct methods are: (1) the standard DHS-7 with Full Birth History with additional questions on pregnancy losses in the past 5 years (FBH+) or (2) a Full Pregnancy History (FPH). No direct comparison of these two methods has been undertaken, although descriptive analyses suggest that the FBH+ may underestimate mortality rates particularly for stillbirths. Methods:This is the protocol paper for the Every Newborn-INDEPTH study (INDEPTH Network, International Network for the Demographic Evaluation of Populations and their Health Every Newborn, Every Newborn Action Plan), aiming to undertake a randomised comparison of FBH+ and FPH to measure pregnancy outcomes in a household survey in five selected INDEPTH Network sites in Africa and South Asia (Bandim in urban and rural Guinea-Bissau; Dabat in Ethiopia; IgangaMayuge in Uganda; Kintampo in Ghana; Matlab in Bangladesh). The survey will reach >68 000 pregnancies to assess if there is ≥15% difference in stillbirth rates. Additional questions will capture birthweight, gestational age, birth/death certification, termination of pregnancy and fertility intentions. The World Bank's Survey Solutions platform will be tailored for data collection, including recording paradata to evaluate timing. A mixed methods assessment of barriers and enablers to reporting of pregnancy and adverse pregnancy outcomes will be undertaken. Conclusions:This large-scale study is the first randomised comparison of these two methods to capture pregnancy outcomes. Results are expected to inform the evidence base for survey methodology, especially in DHS, regarding capture of stillbirths and other outcomes, notably neonatal deaths, abortions (spontaneous and induced), birthweight and gestational age. In addition, this study will inform strategies to improve health and demographic surveillance capture of neonatal/child mortality and pregnancy outcomes.

Project description:BackgroundGlobal mortality estimates remain heavily dependent on household surveys in low- and middle-income countries, where most under-five deaths occur. Few studies have assessed the accuracy of mortality data or determinants of capturing births in surveys.MethodsThe Every Newborn-INDEPTH study (EN-INDEPTH) included a large, multi-country survey of women aged 15-49 interviewed about livebirths and their survival status in five Health and Demographic Surveillance Systems (HDSSs). The HDSSs undertake regular household visits to register births and deaths for a given population. We analysed EN-INDEPTH survey data to assess background factors associated with not recalling a complete date-of-birth. We calculated Kaplan-Meier survival estimates for both survey and HDSS data and describe age-at-death distributions during the past 5 years for children born to the same women. We assessed the proportion of HDSS-births that could be matched on month-of-birth to survey-births and used regression models to identify factors associated with matching.Results69,176 women interviewed in the survey reported 109,817 births and 3064 deaths in children under 5 years in the 5 years prior to the survey. In the HDSS data, the same women had 83,768 registered births and 2335 under-five deaths in the same period. A complete date-of-birth was not reported for 1-7% of survey-births. Birthdates were less likely to be complete for dead children and children born to women of higher parity or with little/no education. Distributions of reported age-at-death indicated heaping at full weeks (neonatal period) and at 12 months. Heaping was more pronounced in the survey data. Survey estimates of under-five mortality rates were similar to HDSS estimates of under-five mortality in two of five sites, higher in the survey in two sites (15%, 41%) and lower (24%) in one site. The proportion of HDSS-births matched to survey-births ranged from 51 to 89% across HDSSs and births of children who had died were less likely to be matched.ConclusionsMortality estimates in the survey and HDSS were not markedly different for most sites. However, neither source is a "gold standard" and both sources miss some events. Research is required to improve capture and accuracy to better track newborn and child survival targets.

Dataset Information

Paradata analyses to inform population-based survey capture of pregnancy outcomes: EN-INDEPTH study.

Background

Methods

Results

Conclusions

Publications

Paradata analyses to inform population-based survey capture of pregnancy outcomes: EN-INDEPTH study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets