Dataset Information

Universal language development screening: comparative performance of two questionnaires

ABSTRACT:

Background and objective

Low language ability in early childhood is a strong predictor of later psychopathology as well as reduced school readiness, lower educational attainment, employment problems and involvement with the criminal justice system. Assessment of early language development is universally offered in many countries, but there has been little evaluation of assessment tools. We planned to compare the screening performance of two commonly used language assessment instruments.

Methods

A pragmatic diagnostic accuracy study was carried out in five areas of England comparing the performance of two screening tools (Ages and Stages Questionnaire (ASQ) and Sure Start Language Measure (SSLM)) against a reference test (Preschool Language Scale, 5th edition).

Results

Results were available for 357 children aged 23–30 months. The ASQ Communication Scale using optimal cut-off values had a sensitivity of 0.55, a specificity of 0.95 and positive and negative predictive values of 0.53 and 0.95, respectively. The SSLM had corresponding values of 0.83, 0.81, 0.33 and 0.98, respectively. Both screening tools performed relatively poorly in families not using English exclusively in the home.

Conclusion

The very widely used ASQ Communication Scale performs poorly as a language screening tool, missing over one-third of cases of low language ability. The SSLM performed better as a screening tool.

SUBMITTER: Wilson P

PROVIDER: S-EPMC8739429 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:ImportanceUniversal developmental screening is widely recommended, yet studies of the accuracy of commonly used questionnaires reveal mixed results, and previous comparisons of these questionnaires are hampered by important methodological differences across studies.ObjectiveTo compare the accuracy of 3 developmental screening instruments as standardized tests of developmental status.Design, setting, and participantsThis cross-sectional diagnostic accuracy study recruited consecutive parents in waiting rooms at 10 pediatric primary care offices in eastern Massachusetts between October 1, 2013, and January 31, 2017. Parents were included if they were sufficiently literate in the English or Spanish language to complete a packet of screening questionnaires and if their child was of eligible age. Parents completed all questionnaires in counterbalanced order. Participants who screened positive on any questionnaire plus 10% of those who screened negative on all questionnaires (chosen at random) were invited to complete developmental testing. Analyses were weighted for sampling and nonresponse and were conducted from October 1, 2013, to January 31, 2017.ExposuresThe 3 screening instruments used were the Ages & Stages Questionnaire, Third Edition (ASQ-3); Parents' Evaluation of Developmental Status (PEDS); and Survey of Well-being of Young Children (SWYC): Milestones.Main outcomes and measuresReference tests administered were Bayley Scales of Infant and Toddler Development, Third Edition, for children aged 0 to 42 months, and Differential Ability Scales, Second Edition, for older children. Age-standardized scores were used as indicators of mild (80-89), moderate (70-79), or severe (<70) delays.ResultsA total of 1495 families of children aged 9 months to 5.5 years participated. The mean (SD) age of the children at enrollment was 2.6 (1.3) years, and 779 (52.1%) were male. Parent respondents were primarily female (1325 [88.7%]), with a mean (SD) age of 33.4 (6.3) years. Of the 20.5% to 29.0% of children with a positive score on each questionnaire, 35% to 60% also received a positive score on a second questionnaire, demonstrating moderate co-occurrence. Among younger children (<42 months), the specificity of the ASQ-3 (89.4%; 95% CI, 85.9%-92.1%) and SWYC Milestones (89.0%; 95% CI, 86.1%-91.4%) was higher than that of the PEDS (79.6%; 95% CI, 75.7%-83.1%; P?<?.001 and P?=?.002, respectively), but differences in sensitivity were not statistically significant. Among older children (43-66 months), specificity of the ASQ-3 (92.1%; 95% CI, 85.1%-95.9%) was higher than that of the SWYC Milestones (70.7%; 95% CI, 60.9%-78.8%) and the PEDS (73.7%; 95% CI, 64.3%-81.3%; P?<?.001), but sensitivity to mild delays of the SWYC Milestones (54.8%; 95% CI, 38.1%-70.4%) and of the PEDS (61.8%; 95% CI, 43.1%-77.5%) was higher than that of the ASQ-3 (23.5%; 95% CI, 9.0%-48.8%; P?=?.012 and P?=?.002, respectively). Sensitivity exceeded 70% only with respect to severe delays, with 73.7% (95% CI, 50.1%-88.6%) for the SWYC Milestones among younger children, 78.9% (95% CI, 55.4%-91.9%) for the PEDS among younger children, and 77.8% (95% CI, 41.8%-94.5%) for the PEDS among older children. Attending to parents' concerns was associated with increased sensitivity of all questionnaires.Conclusions and relevanceThis study found that 3 frequently used screening questionnaires offer adequate specificity but modest sensitivity for detecting developmental delays among children aged 9 months to 5 years. The results suggest that trade-offs in sensitivity and specificity occurred among the questionnaires, with no one questionnaire emerging superior overall.

Project description:ImportanceLarge language models (LLMs) are promising as tools for citation screening in systematic reviews. However, their applicability has not yet been determined.ObjectiveTo evaluate the accuracy and efficiency of an LLM in title and abstract literature screening.Design, setting, and participantsThis prospective diagnostic study used the data from the title and abstract screening process for 5 clinical questions (CQs) in the development of the Japanese Clinical Practice Guidelines for Management of Sepsis and Septic Shock. The LLM decided to include or exclude citations based on the inclusion and exclusion criteria in terms of patient, population, problem; intervention; comparison; and study design of the selected CQ and was compared with the conventional method for title and abstract screening. This study was conducted from January 7 to 15, 2024.ExposuresLLM (GPT-4 Turbo)-assisted citation screening or the conventional method.Main outcomes and measuresThe sensitivity and specificity of the LLM-assisted screening process was calculated, and the full-text screening result using the conventional method was set as the reference standard in the primary analysis. Pooled sensitivity and specificity were also estimated, and screening times of the 2 methods were compared.ResultsIn the conventional citation screening process, 8 of 5634 publications in CQ 1, 4 of 3418 in CQ 2, 4 of 1038 in CQ 3, 17 of 4326 in CQ 4, and 8 of 2253 in CQ 5 were selected. In the primary analysis of 5 CQs, LLM-assisted citation screening demonstrated an integrated sensitivity of 0.75 (95% CI, 0.43 to 0.92) and specificity of 0.99 (95% CI, 0.99 to 0.99). Post hoc modifications to the command prompt improved the integrated sensitivity to 0.91 (95% CI, 0.77 to 0.97) without substantially compromising specificity (0.98 [95% CI, 0.96 to 0.99]). Additionally, LLM-assisted screening was associated with reduced time for processing 100 studies (1.3 minutes vs 17.2 minutes for conventional screening methods; mean difference, -15.25 minutes [95% CI, -17.70 to -12.79 minutes]).Conclusions and relevanceIn this prospective diagnostic study investigating the performance of LLM-assisted citation screening, the model demonstrated acceptable sensitivity and reasonably high specificity with reduced processing time. This novel method could potentially enhance efficiency and reduce workload in systematic reviews.

Project description:BackgroundMost valid methods to measure treatment adherence require time and resources, and they are not easily applied in highly demanding Primary Health Care Clinics (PHCC). The objective of this study was to determine sensitivity, specificity, predictive values, likelihood ratios, and post-test probabilities of two novel questionnaires as proxy measurements of treatment adherence in Type-2 diabetic patients.MethodsTwo questionnaires were developed by a group of experts to identify the patient's medical prescription knowledge (knowledge) and their attitudes toward treatment adherence (attitudes) as proxy measurements of adherence. The questionnaires were completed by patients receiving care in PHCC pertaining to the Mexican Institute of Social Security in Aguascalientes (Mexico). Pill count was used as gold standard. Participants were selected randomly, and their oral hypoglycemic prescriptions were studied. The main outcome measures for each questionnaire were sensitivity, specificity, predictive values, likelihood ratios, and post-test probabilities, all as an independent questionnaire test and in a serial analysis.ResultsAdherence prevalence was 27.0% using pill count. Knowledge questionnaire showed the highest sensitivity (68.1%) and negative predictive value (82.2%), the lowest negative likelihood ratio (0.58) and post-test probability for a negative result (0.16). Serial analysis showed the highest specificity (77.4%) and positive predictive value (40.1%) as well as the highest positive likelihood ratio (1.8) and post-test probability for a positive result (0.39).ConclusionMedical Prescription Knowledge questionnaire showed the best performance as proxy measurement to identify non-adherence in type 2 diabetic patients regarding negative predictive value, negative likelihood ratio, and post-test probability for a negative result. However, Medical Prescription Knowledge questionnaire performance may change in contexts with higher adherence prevalence. Therefore, more research is needed before using this method in other contexts.

Project description:BACKGROUND:Dental disease is one of the most common health problems in dogs. However, no studies have investigated the attitudes, opinions and practices of dog owners, veterinarians and veterinary nurses regarding dental health and preventative dental home care in dogs. The objective of this study was therefore to develop and validate questionnaires for this purpose, in accordance with survey methodology guidelines. METHODS:Questionnaire items were determined based on the authors' academic knowledge and clinical experience, and modified throughout the validation process. Several measures were taken to reduce sampling, coverage, measurement and non-response errors. Content validity was assessed by Subject-Matter Experts (SME) and cognitive interviews were conducted in accordance with the "think-aloud protocol". Non-response analysis was performed using several methods. RESULTS:Constructs were identified using exploratory factor analysis and two predefined constructs from the dog owner questionnaire were confirmed "Dog owners' attitudes towards brushing their dog's teeth" (Cronbach's ? = 0.86) and "Dog owners' assessment of their dog's dental health" (? = 0.76). Additionally, exploratory factor analysis identified three potential constructs. In the veterinary health practitioner questionnaire, two constructs were identified: "Veterinary health practitioners' attitudes towards dental chews and dental feed" (? = 0.78) and "Veterinary health practitioners' attitudes and opinions on dental problems and dental cleaning" (? = 0.73). Non-response analysis showed a higher proportion of women in the sample of dog owners and veterinarians compared to the target populations. Veterinarians in the sample were also younger. In contrast, gender and age distributions in veterinary nurses did not differ between sample and target. CONCLUSION:The validation presented in this work showed that the developed questionnaires could be used as accurate and reliable tools for measuring attitudes and practices regarding dental home care in dogs among Swedish dog owners, veterinarians and veterinary nurses.