Dataset Information

Automating document classification with distant supervision to increase the efficiency of systematic reviews: A case study on identifying studies with HIV impacts on female sex workers.

ABSTRACT: There remains a limited understanding of the HIV prevention and treatment needs among female sex workers in many parts of the world. Systematic reviews of existing literature can help fill this gap; however, well-done systematic reviews are time-demanding and labor-intensive. Here, we propose an automatic document classification approach to a systematic review to significantly reduce the effort in reviewing documents and optimizing empiric decision making. We first describe a manual document classification procedure that is used to curate a pertinent training dataset and then propose three classifiers: a keyword-guided method, a cluster analysis-based method, and a random forest approach that utilizes a large set of feature tokens. This approach is used to identify documents studying female sex workers that contain content relevant to either HIV or experienced violence. We compare the performance of the three classifiers by cross-validation in terms of area under the curve of the receiver operating characteristic and precision and recall plot, and found random forest approach reduces the amount of manual reading for our example by 80%; in sensitivity analysis, we found that even trained with only 10% of data, the classifier can still avoid reading 75% of future documents (68% of total) while retaining 80% of relevant documents. In sum, the automated procedure of document classification presented here could improve both the precision and efficiency of systematic reviews and facilitate live reviews, where reviews are updated regularly. We expect to obtain a reasonable classifier by taking 20% of retrieved documents as training samples. The proposed classifier could also be used for more meaningfully assembling literature in other research areas and for rapid documents screening with a tight schedule, such as COVID-related work during the crisis.

SUBMITTER: Li X

PROVIDER: S-EPMC9246134 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Effective supportive supervision of healthcare services is crucial for improving and maintaining quality of care. However, this process can be challenging in an environment with chronic shortage of qualified human resources, overburdened healthcare providers, multiple roles of district managers, weak supply chains, high donor fragmentation and inefficient allocation of limited financial resources. Operating in this environment, we systematically evaluated an approach developed in Tanzania to strengthen the implementation of routine supportive supervision of primary healthcare providers. The approach included a systematic quality assessment at health facilities using an electronic tool and subsequent result dissemination at council level. Mixed methods were used to compare the new supportive supervision approach with routine supportive supervision. Qualitative data was collected through in-depth interviews in three councils. Observational data and informal communication as well as secondary data complemented the data set. Additionally, an economic costing analysis was carried out in the same councils. Compared to routine supportive supervision, the new approach increased healthcare providers' knowledge and skills, as well as quality of data collected and acceptance of supportive supervision amongst stakeholders involved. It also ensured better availability of evidence for follow-up actions, including budgeting and planning, and higher stakeholder motivation and ownership of subsequent quality improvement measures. The new approach reduced time and cost spent during supportive supervision. This increased feasibility of supportive supervision and hence the likelihood of its implementation. Thus, the results presented together with previous findings suggested that if used as the standard approach for routine supportive supervision the new approach offers a suitable option to make supportive supervision more efficient and effective and therewith more sustainable. Moreover, the new approach also provides informed guidance to overcome several problems of supportive supervision and healthcare quality assessments in low- and middle income countries.

Dataset Information

Automating document classification with distant supervision to increase the efficiency of systematic reviews: A case study on identifying studies with HIV impacts on female sex workers.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets