Dataset Information

ELIPS: Development and Validation of an Observational Tool for Examining Early Language in Play Settings.

ABSTRACT: Intervention in the early years can help to mitigate the risks that early language and communication difficulties pose for later learning and well-being. Critical to this is the capacity of early years educators to evaluate language development accurately in the classroom in order to target individual support effectively. This article reports on the development and testing of the Early Language in Play Settings (eLIPS) tool, an observational measure of child language. An action research model was used in the design of the tool with the result that the methodology adopted was compatible with an early years child-centered approach. Observations of children in play settings were used to gather information about early language through subscales measuring social communication, receptive and expressive language. A series of preliminary trials with 3- to 5-year-olds, established that the eLIPS measures have concurrent validity with scores from a standardized language assessment, the Clinical Evaluation of Language Fundamentals-Preschool 2 ^UK . Investigation of internal consistency showed reliability for use by researchers and early years educators together with inter-rater reliability across these groups. It was concluded that eLIPS has potential as a tool to assist early years educators in understanding individual patterns of language acquisition in a play-based environment and for framing team discussions about approaches to early language support.

SUBMITTER: Duncan LG

PROVIDER: S-EPMC7412886 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundBest Practice Alerts (BPAs) are alert messages to physicians in the electronic health record that are used to encourage appropriate use of health care resources. While these alerts are helpful in both improving care and reducing costs, BPAs are often broadly applied nonselectively across entire patient populations. The development of large language models (LLMs) provides an opportunity to selectively identify patients for BPAs.ObjectiveIn this paper, we present an example case where an LLM screening tool is used to select patients appropriate for a BPA encouraging the prescription of deep vein thrombosis (DVT) anticoagulation prophylaxis. The artificial intelligence (AI) screening tool was developed to identify patients experiencing acute bleeding and exclude them from receiving a DVT prophylaxis BPA.MethodsOur AI screening tool used a BioMed-RoBERTa (Robustly Optimized Bidirectional Encoder Representations from Transformers Pretraining Approach; AllenAI) model to perform classification of physician notes, identifying patients without active bleeding and thus appropriate for a thromboembolism prophylaxis BPA. The BioMed-RoBERTa model was fine-tuned using 500 history and physical notes of patients from the MIMIC-III (Medical Information Mart for Intensive Care) database who were not prescribed anticoagulation. A development set of 300 MIMIC patient notes was used to determine the model's hyperparameters, and a separate test set of 300 patient notes was used to evaluate the screening tool.ResultsOur MIMIC-III test set population of 300 patients included 72 patients with bleeding (ie, were not appropriate for a DVT prophylaxis BPA) and 228 without bleeding who were appropriate for a DVT prophylaxis BPA. The AI screening tool achieved impressive accuracy with a precision-recall area under the curve of 0.82 (95% CI 0.75-0.89) and a receiver operator curve area under the curve of 0.89 (95% CI 0.84-0.94). The screening tool reduced the number of patients who would trigger an alert by 20% (240 instead of 300 alerts) and increased alert applicability by 14.8% (218 [90.8%] positive alerts from 240 total alerts instead of 228 [76%] positive alerts from 300 total alerts), compared to nonselectively sending alerts for all patients.ConclusionsThese results show a proof of concept on how language models can be used as a screening tool for BPAs. We provide an example AI screening tool that uses a HIPAA (Health Insurance Portability and Accountability Act)-compliant BioMed-RoBERTa model deployed with minimal computing power. Larger models (eg, Generative Pre-trained Transformers-3, Generative Pre-trained Transformers-4, and Pathways Language Model) will exhibit superior performance but require data use agreements to be HIPAA compliant. We anticipate LLMs to revolutionize quality improvement in hospital medicine.

Project description:ObjectiveThe Table Talk tool is an observational assessment of early care and education teacher (ECET) mealtime practices. The Table Talk Revised (TT-R) tool incorporates new constructs that emerged from qualitative research and teases apart existing categories to improve nuance of data capture. The objective of this study was to evaluate the TT-R, document interrater reliability for the TT-R, and report on ECET feeding communications in broader settings than previously studied (i.e., beyond a single Lunch and Head Start only).MethodsTrained observers conducted mealtime observations in classrooms (Nclassroms?=?63, 10 sites) during Breakfast and two Lunches for both Lead and Assistant ECETs (N?=?126). Classrooms were spread across Head Start in an urban area (60%), Head Starts in a rural area (24%), and a state-funded preschool (16%).ResultsOn average, there were 22.17 (SD?=?10.92) total verbal feeding communications at Breakfast, 37.72 (SD?=?15.83) at Lunch1, and 34.39 (SD?=?15.05) at Lunch2 with meals averaging 25?min. The most commonly observed supportive statement category was Exploring Foods for Lead (Breakfast?=?1.61, Lunch1?=?3.23, Lunch2?=?2.70) and Assistant ECETs (Breakfast?=?.89, Lunch1?=?2.03) except for Lunch2 which was Encourages Trying in a Positive Way (Lunch2?=?1.30). The most commonly observed unsupportive statement category was Firm Behavioral Control for both Lead (Breakfast?=?3.61, Lunch1?=?5.84, Lunch2?=?5.51) and Assistants ECETs (Breakfast?=?3.11, Lunch1?=?6.38, Lunch2?=?4.32). The majority of Interclass Correlation Coefficients indicating interrater reliability were in the excellent range (64%) for commonly occurring statement categories, and 14 of the 19 low frequency statement categories had >?80% agreement.Conclusions and implicationsOverall, items added to the Table Talk tool performed well, and interrater reliability was favorable. Our study also documented differences between Lead and Assistant teachers in mealtime practices and illustrated differing patterns of interaction between lunches and breakfast, important findings to inform future research and practice. The TT-R may be a useful measurement tool for monitoring and evaluating ECET practices in mealtime environments as well as informing intervention.

Project description:BackgroundAlthough 80% of children with disabilities live in developing countries, there are few culturally appropriate developmental assessment tools available for these settings. Often tools from the West provide misleading findings in different cultural settings, where some items are unfamiliar and reference values are different from those of Western populations.Methods and findingsFollowing preliminary and qualitative studies, we produced a draft developmental assessment tool with 162 items in four domains of development. After face and content validity testing and piloting, we expanded the draft tool to 185 items. We then assessed 1,426 normal rural children aged 0-6 y from rural Malawi and derived age-standardized norms for all items. We examined performance of items using logistic regression and reliability using kappa statistics. We then considered all items at a consensus meeting and removed those performing badly and those that were unnecessary or difficult to administer, leaving 136 items in the final Malawi Developmental Assessment Tool (MDAT). We validated the tool by comparing age-matched normal children with those with malnutrition (120) and neurodisabilities (80). Reliability was good for items remaining with 94%-100% of items scoring kappas >0.4 for interobserver immediate, delayed, and intra-observer testing. We demonstrated significant differences in overall mean scores (and individual domain scores) for children with neurodisabilities (35 versus 99 [p<0.001]) when compared to normal children. Using a pass/fail technique similar to the Denver II, 3% of children with neurodisabilities passed in comparison to 82% of normal children, demonstrating good sensitivity (97%) and specificity (82%). Overall mean scores of children with malnutrition (weight for height <80%) were also significantly different from scores of normal controls (62.5 versus 77.4 [p<0.001]); scores in the separate domains, excluding social development, also differed between malnourished children and controls. In terms of pass/fail, 28% of malnourished children versus 94% of controls passed the test overall.ConclusionsA culturally relevant developmental assessment tool, the MDAT, has been created for use in African settings and shows good reliability, validity, and sensitivity for identification of children with neurodisabilities.

Project description:BackgroundThe use of obstetric early-warning-systems (EWS) has been recommended to improve timely recognition, management and early referral of women who have or are developing a critical illness. Development of such prediction models should involve a statistical combination of predictor clinical observations into a multivariable model which should be validated. No obstetric EWS has been developed and validated for low resource settings. We report on the development and validation of a simple prediction model for obstetric morbidity and mortality in resource-limited settings.MethodsWe performed a multivariate logistic regression analysis using a retrospective case-control analysis of secondary data with clinical indices predictive of severe maternal outcome (SMO). Cases for design and validation were randomly selected (n = 500) from 4360 women diagnosed with SMO in 42 Nigerian tertiary-hospitals between June 2012 and mid-August 2013. Controls were 1000 obstetric admissions without SMO diagnosis. We used clinical observations collected within 24 h of SMO occurrence for cases, and normal births for controls. We created a combined dataset with two controls per case, split randomly into development (n = 600) and validation (n = 900) datasets. We assessed the model's validity using sensitivity and specificity measures and its overall performance in predicting SMO using receiver operator characteristic (ROC) curves. We then fitted the final developmental model on the validation dataset and assessed its performance. Using the reference range proposed in the United Kingdom Confidential-Enquiry-into-Maternal-and-Child-Health 2007-report, we converted the model into a simple score-based obstetric EWS algorithm.ResultsThe final developmental model comprised abnormal systolic blood pressure-(SBP > 140 mmHg or < 90 mmHg), high diastolic blood pressure-(DBP > 90 mmHg), respiratory rate-(RR > 40/min), temperature-(> 38 °C), pulse rate-(PR > 120/min), caesarean-birth, and the number of previous caesarean-births. The model was 86% (95% CI 81-90) sensitive and 92%- (95% CI 89-94) specific in predicting SMO with area under ROC of 92% (95% CI 90-95%). All parameters were significant in the validation model except DBP. The model maintained good discriminatory power in the validation (n = 900) dataset (AUC 92, 95% CI 88-94%) and had good screening characteristics. Low urine output (300mls/24 h) and conscious level (prolonged unconsciousness-GCS < 8/15) were strong predictors of SMO in the univariate analysis.ConclusionWe developed and validated statistical models that performed well in predicting SMO using data from a low resource settings. Based on these, we proposed a simple score based obstetric EWS algorithm with RR, temperature, systolic BP, pulse rate, consciousness level, urinary output and mode of birth that has a potential for clinical use in low-resource settings..

Project description:BackgroundIdentifying pneumonia using diagnosis codes alone may be insufficient for research on clinical decision making. Natural language processing (NLP) may enable the inclusion of cases missed by diagnosis codes.ObjectivesThis article (1) develops a NLP tool that identifies the clinical assertion of pneumonia from physician emergency department (ED) notes, and (2) compares classification methods using diagnosis codes versus NLP against a gold standard of manual chart review to identify patients initially treated for pneumonia.MethodsAmong a national population of ED visits occurring between 2006 and 2012 across the Veterans Affairs health system, we extracted 811 physician documents containing search terms for pneumonia for training, and 100 random documents for validation. Two reviewers annotated span- and document-level classifications of the clinical assertion of pneumonia. An NLP tool using a support vector machine was trained on the enriched documents. We extracted diagnosis codes assigned in the ED and upon hospital discharge and calculated performance characteristics for diagnosis codes, NLP, and NLP plus diagnosis codes against manual review in training and validation sets.ResultsAmong the training documents, 51% contained clinical assertions of pneumonia; in the validation set, 9% were classified with pneumonia, of which 100% contained pneumonia search terms. After enriching with search terms, the NLP system alone demonstrated a recall/sensitivity of 0.72 (training) and 0.55 (validation), and a precision/positive predictive value (PPV) of 0.89 (training) and 0.71 (validation). ED-assigned diagnostic codes demonstrated lower recall/sensitivity (0.48 and 0.44) but higher precision/PPV (0.95 in training, 1.0 in validation); the NLP system identified more "possible-treated" cases than diagnostic coding. An approach combining NLP and ED-assigned diagnostic coding classification achieved the best performance (sensitivity 0.89 and PPV 0.80).ConclusionSystem-wide application of NLP to clinical text can increase capture of initial diagnostic hypotheses, an important inclusion when studying diagnosis and clinical decision-making under uncertainty.

Project description:ImportanceAdherence to the Consolidated Standards of Reporting Trials (CONSORT) for randomized clinical trials is associated with improvingquality because inadequate reporting in randomized clinical trials may complicate the interpretation and the application of findings to clinical care.ObjectiveTo evaluate an automated reporting checklist generation tool that uses natural language processing (NLP), called CONSORT-NLP.Design, setting, and participantsThis study used published journal articles as training, testing, and validation sets to develop, refine, and evaluate the CONSORT-NLP tool. Articles reporting randomized clinical trials were selected from 25 high-impact-factor journals under the following categories: (1) general and internal medicine, (2) oncology, and (3) cardiac and cardiovascular systems.Main outcomes and measuresFor an evaluation of the performance of this tool, an accuracy metric defined as the number of correct assessments divided by all assessments was calculated.ResultsThe CONSORT-NLP tool uses the widely used Portable Document Format as an input file. Of the 37 CONSORT reporting items, 34 (92%) were included in the tool. Of these 34 reporting items, 30 were fully implemented; 28 (93%) of the fully implemented CONSORT reporting items had an accuracy of more than 90% for the validation set. The remaining 2 (7%) had an accuracy between 80% and 90% for the validation set. Two to 5 articles were selected from each of these journals for a total of 158 articles to establish a training set of 111 articles to train CONSORT-NLP for CONSORT reporting items, a testing set of 25 articles to refine CONSORT-NLP, and a validation set of 22 articles to assess the performance of CONSORT-NLP. The CONSORT-NLP tool used the Portable Document Format of the articles as input files. A CONSORT-NLP graphical user interface was built using Java in 2019. The time required to complete the CONSORT checklist manually vs using the CONSORT-NLP tool was compared for 30 articles. Two case studies for randomized clinical trials are provided as an illustration for the CONSORT-NLP tool. For the 30 articles investigated, CONSORT-NLP required a mean (SD) 23.0 (4.1) seconds, whereas the manual reviewer required a mean (SD) 11.9 (2.2), 22.6 (4.6), and 57.6 (7.1) minutes, for 3 reviewers, respectively.Conclusions and relevanceThe CONSORT-NLP tool is designed to assist in the reporting of randomized clinical trials. Potential users of CONSORT-NLP include clinicians, researchers, and scientists who plan to publish a randomized trial study in a peer-reviewed journal. The use of CONSORT-NLP may help them save substantial time when generating the CONSORT checklist. This tool may also be useful for manuscript reviewers and journal editors who review these articles.

Dataset Information

ELIPS: Development and Validation of an Observational Tool for Examining Early Language in Play Settings.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets