An automated essay scoring systems: a systematic literature review.
Ontology highlight
ABSTRACT: Supplementary information
The online version contains supplementary material available at 10.1007/s10462-021-10068-2.
SUBMITTER: Ramesh D
PROVIDER: S-EPMC8460059 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Automated essay scoring (AES) is the task of automatically assigning scores to essays as an alternative to human grading. Conventional AES methods typically rely on manually tuned features, which are laborious to effectively develop. To obviate the need for feature engineering, many deep neural network (DNN)-based AES models have been proposed and have achieved state-of-the-art accuracy. DNN-AES models require training on a large dataset of graded essays. However, assigned grades in such datasets are known to be strongly biased due to effects of rater bias when grading is conducted by assigning a few raters in a rater set to each essay. Performance of DNN models rapidly drops when such biased data are used for model training. In the fields of educational and psychological measurement, item response theory (IRT) models that can estimate essay scores while considering effects of rater characteristics have recently been proposed. This study therefore proposes a new DNN-AES framework that integrates IRT models to deal with rater bias within training data. To our knowledge, this is a first attempt at addressing rating bias effects in training data, which is a crucial but overlooked problem.
| S-EPMC7334153 | biostudies-literature
Project description:Clinical coding and classification processes transform natural language descriptions in clinical text into data that can subsequently be used for clinical care, research, and other purposes. This systematic literature review examined studies that evaluated all types of automated coding and classification systems to determine the performance of such systems. Studies indexed in Medline or other relevant databases prior to March 2009 were considered. The 113 studies included in this review show that automated tools exist for a variety of coding and classification purposes, focus on various healthcare specialties, and handle a wide variety of clinical document types. Automated coding and classification systems themselves are not generalizable, nor are the results of the studies evaluating them. Published research shows these systems hold promise, but these data must be considered in context, with performance relative to the complexity of the task and the desired outcome.
| S-EPMC3000748 | biostudies-literature
Project description:IntroductionSeveral versions of Early Warning Systems (EWS) are used in obstetrics to detect and treat early clinical deterioration to avert morbidity and mortality. EWS can potentially be useful to improve the quality of care and reduce the risk of maternal mortality in resource-limited settings. We conducted a systematic literature review of published obstetric early warning systems, define their predictive accuracy for morbidity and mortality, and their effectiveness in triggering corrective actions and improving health outcomes.MethodsWe systematically searched for primary research articles on obstetric EWS published in peer-reviewed journals between January 1997 and March 2018 in Medline, CINAHL, SCOPUS, Science Direct, and Science Citation Index. We also searched reference lists of relevant articles and websites of professional societies. We included studies that assessed the predictive accuracy of EWS to detect clinical deterioration, or/and their effectiveness in improving clinical outcomes in obstetric inpatients. We excluded studies with a paediatric or non-obstetric adult population. Cross-sectional and qualitative studies were also excluded. We performed a narrative synthesis since the outcomes reported were heterogeneous.ResultsA total of 381 papers were identified, 17 of which met the inclusion criteria. Eleven of the included studies evaluated the predictive accuracy of EWS for obstetric morbidity and mortality, 5 studies assessed the effectiveness of EWS in improving clinical outcomes, while one study addressed both. Sixteen published EWS versions were reviewed, 14 of which included five basic clinical observations (pulse rate, respiratory rate, temperature, blood pressure, and consciousness level). The obstetric EWS identified had very high median (inter-quartile range) sensitivity-89% (72% to 97%) and specificity-85% (67% to 98%) but low median (inter-quartile range) positive predictive values-41% (25% to 74%) for predicting morbidity or ICU admission. Obstetric EWS had a very high accuracy in predicting death (AUROC >0.80) among critically ill obstetric patients. Obstetric EWS improves the frequency of routine vital sign observation, reduces the interval between the recording of specifically defined abnormal clinical observations and corrective clinical actions, and can potentially reduce the severity of obstetric morbidity.ConclusionObstetric EWS are effective in predicting severe morbidity (in general obstetric population) and mortality (in critically ill obstetric patients). EWS can contribute to improved quality of care, prevent progressive obstetric morbidity and improve health outcomes. There is limited evidence of the effectiveness of EWS in reducing maternal death across all settings. Clinical parameters in most obstetric EWS versions are routinely collected in resource-limited settings, therefore implementing EWS may be feasible in such settings.
| S-EPMC6544303 | biostudies-literature
Project description:Available information on chest Computed Tomography (CT) findings of the 2019 novel coronavirus disease (COVID-19) is constantly evolving. Ground glass opacities and consolidation with bilateral and peripheral distribution were reported as the most common CT findings, but also less typical features could be identified. All radiologists should be aware of the imaging spectrum of the COVID-19 pneumonia and imaging changes in the course of the disease. Our aim is to display the chest CT findings at first assessment and follow-up through a pictorial essay, to help in the recognition of these features for an accurate diagnosis.
| S-EPMC7315402 | biostudies-literature
Project description:BackgroundAcute exacerbations of asthma are common in children. Multiple asthma severity scores exist, but current emergency department (ED) use of severity scores is not known.MethodsA systematic review was undertaken to identify the parameters collected in pediatric asthma severity scores. A survey of Paediatric Emergency Research in the United Kingdom and Ireland (PERUKI) sites was undertaken to ascertain routinely collected asthma data and information about severity scores. Included studies examined severity of asthma exacerbation in children 5-18 years of age with extractable severity parameters.ResultsSixteen articles were eligible, containing 17 asthma severity scores. The severity scores assessed combinations of 15 different parameters (median, 6; range, 2-8). The most common parameters considered were expiratory wheeze (15/17), inspiratory wheeze (13/17), respiratory rate (10/17), and general accessory muscle use (9/17). Fifty-nine PERUKI centers responded to the questionnaire. Twenty centers (33.1%) currently assess severity, but few use a published score. The most commonly recorded routine data required for severity scores were oxygen saturations (59/59, 100%), heart rate, and respiratory rate (58/59, 98.3% for both). Among well-validated scores like the Pulmonary Index Score (PIS), Pediatric Asthma Severity Score (PASS), Childhood Asthma Score (CAS), and the Pediatric Respiratory Assessment Measure (PRAM), only 6/59 (10.2%), 3/59 (5.1%), 1/59 (1.7%), and 0 (0%) of units respectively routinely collect the data required to calculate them.ConclusionStandardized published pediatric asthma severity scores are infrequently used. Improved routine data collection focusing on the key parameters common to multiple scores could improve this, facilitating research and audit of pediatric acute asthma.
| S-EPMC7593416 | biostudies-literature
Project description:Background & aimsStenosis is a common complication of Crohn's disease (CD) that has no effective medical therapy. Development of antifibrotic agents will require testing in randomized controlled trials. Computed tomography enterography- and magnetic resonance enterography-based technologies might be used to measure outcomes in these trials. These approaches have been validated in studies of patients with symptomatic strictures who underwent imaging evaluations followed by resection with histopathologic grading of the intestinal tissue for inflammation and/or fibrosis (the reference standard). Imaging findings have correlated with findings from quantitative or semiquantitative histologic evaluation of the degree of fibromuscular stenosis and/or inflammation on the resection specimen. However, it is not clear whether histologic findings are an accurate reference standard. We performed a systematic review of all published histologic scoring systems used to assess stenosing CD.MethodsWe performed a comprehensive search of Embase and MEDLINE of studies through March 13, 2019, that used a histologic scoring system to characterize small bowel CD and assessed inflammatory and fibrotic alterations within the same adult individual. All scores fitting the criteria were included in our analysis, independent of the presence of stricturing disease, as long as inflammation and fibrosis were evaluated separately but in the same scoring system.ResultsWe observed substantial heterogeneity among the scoring systems, which were not derived from modern principles for evaluative index development. None had undergone formal validity or reliability testing. None of the existing indices had been constructed according to accepted methods for the development of evaluative indices. Basic knowledge regarding their operating properties were lacking. Specific indices for evaluating the important pathologic component of myofibroblast hypertrophy or hyperplasia have not been proposed.ConclusionsIn a systematic review of publications, we found a lack of validated histopathologic scoring systems for assessment of fibromuscular stenosis. Data that describe the operating properties of existing cross-sectional imaging techniques for stenosing CD should be questioned. Development and validation of a histopathology index is an important research priority.
| S-EPMC7649049 | biostudies-literature
Project description:INTRODUCTION:Timely notification of infectious diseases is crucial for prompt response by public health services. Adequate notification systems facilitate timely notification. A systematic literature review was performed to assess outcomes of studies on notification timeliness and to determine which aspects of notification systems are associated with timely notification. METHODOLOGY:Articles reviewing timeliness of notifications published between 2000 and 2017 were searched in Pubmed and Scopus. Using a standardized notification chain, timeliness of reporting system for each article was defined as either sufficient (≥ 80% notifications in time), partly sufficient (≥ 50-80%), or insufficient (< 50%) according to the article's predefined timeframe, a standardized timeframe for all articles, and a disease specific timeframe. Electronic notification systems were compared with conventional methods (postal mail, fax, telephone, email) and mobile phone reporting. RESULTS:48 articles were identified. In almost one third of the studies with a predefined timeframe (39), timeliness of notification systems was either sufficient or insufficient (11/39, 28% and 12/39, 31% resp.). Applying the standardized timeframe (45 studies) revealed similar outcomes (13/45, 29%, sufficient notification timeframe, vs 15/45, 33%, insufficient). The disease specific timeframe was not met by any study. Systems involving reporting by laboratories most often complied sufficiently with predefined or standardized timeframes. Outcomes were not related to electronic, conventional notification systems or mobile phone reporting. Electronic systems were faster in comparative studies (10/13); this hardly resulted in sufficient timeliness, neither according to predefined nor to standardized timeframes. CONCLUSION:A minority of notification systems meets either predefined, standardized or disease specific timeframes. Systems including laboratory reporting are associated with timely notification. Electronic systems reduce reporting delay, but implementation needs considerable effort to comply with notification timeframes. During outbreak threats, patient, doctors and laboratory testing delays need to be reduced to achieve timely detection and notification. Public health authorities should incorporate procedures for this in their preparedness plans.
| S-EPMC6002046 | biostudies-literature
Project description:Hidradenitis suppurativa (HS) is an inflammatory disorder characterized by chronic deep-seated nodules, abscesses, fistulae, sinus tracts, and scars in apocrine gland-bearing regions. Assessing its severity is challenging because of its clinical heterogeneity, lack of a standardized tool, and increasing severity scores. This article provides a chronological overview of HS grading scales to aid in the understanding and comparison of different scoring systems. A literature review of articles published in English on PubMed was conducted searched from 1989 to 2023. The review includes 15 scores that are the most relevant and widely used and acknowledges the existence of over 30 scoring systems for HS. The expanding landscape of HS scoring systems presents challenges when patients evaluated using different systems are compared. A universally accepted scoring system is required for consistent application across diverse populations. A comprehensive assessment should balance subjective and objective items, considering observer-reported signs and patient-reported symptoms to make meaningful treatment decisions.
| S-EPMC10861310 | biostudies-literature
Project description:IntroductionSeveral different forms of automated insulin delivery systems (AID systems) have recently been developed and are now licensed for type 1 diabetes (T1D). We undertook a systematic review of reported trials and real-world studies for commercial hybrid closed-loop (HCL) systems.MethodsPivotal, phase III and real-world studies using commercial HCL systems that are currently approved for use in type 1 diabetes were reviewed with a devised protocol using the Medline database.ResultsFifty-nine studies were included in the systematic review (19 for 670G; 8 for 780G; 11 for Control-IQ; 14 for CamAPS FX; 4 for Diabeloop; and 3 for Omnipod 5). Twenty were real-world studies, and 39 were trials or sub-analyses. Twenty-three studies, including 17 additional studies, related to psychosocial outcomes and were analysed separately.ConclusionsThese studies highlighted that HCL systems improve time In range (TIR) and arouse minimal concerns around severe hypoglycaemia. HCL systems are an effective and safe option for improving diabetes care. Real-world comparisons between systems and their effects on psychological outcomes require further study.
| S-EPMC10126177 | biostudies-literature
Project description:BackgroundThe purpose of antenatal care (ANC) is to identify 'at-risk' pregnant women, to provide quality care for all, and maximize the allocation of resources for those who need them the most. To address the synergistic effect of risk factors, clinicians across the globe developed antenatal scoring systems.ObjectiveThis review aims to investigate various antenatal risk scoring systems developed and used in India to predict adverse neonatal outcome.MethodsWe reviewed articles published between January 2000 and April 2020, which have either developed a scoring system or used a scoring system, among the Indian population. This systematic review is reported based on Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) guidelines. Prediction model study Risk Of Bias Assessment Tool (PROBAST) was employed for the assessment of the quality of included studies. Data sources such as Embase, MEDLINE/Pubmed, APA PsycExtra, PsycINFO, CINHAL Plus, Cochrane Library, IndMED, LILACS, Scopus, WHO Reproductive Health Library and Web of science were searched.ResultsAn initial search retrieved a total of 6246 articles. This systematic review identified six studies, of which one study developed an antenatal scoring system and the other five studies used two antenatal systems for predicting adverse neonatal outcome. The study which developed a risk scoring system had a high risk of bias (ROB) and concern for applicability. The overall sensitivity of the antenatal scoring system was high (77.4%), but the specificity was low (45%). Similarly, the positive predictive value is low (15.3%), and the negative predictive value is high (94.2%). A meta-analysis was not conducted due to the heterogeneity of the studies and insufficient data.ConclusionsThere is a need for a systematically developed antenatal scoring system for India. Such scoring systems can be promising in public health, proposing a paradigm shift in the implementation of effective mother and child health programmes locally as well as nationally.Supplementary informationThe online version contains supplementary material available at 10.1007/s13224-021-01484-z.
| S-EPMC9207162 | biostudies-literature