Dataset Information

Evaluation of Online Information in University Students: Development and Scaling of the Screening Instrument EVON.

ABSTRACT: As Internet sources provide information of varying quality, it is an indispensable prerequisite skill to evaluate the relevance and credibility of online information. Based on the assumption that competent individuals can use different properties of information to assess its relevance and credibility, we developed the EVON (evaluation of online information), an interactive computer-based test for university students. The developed instrument consists of eight items that assess the skill to evaluate online information in six languages. Within a simulated search engine environment, students are requested to select the most relevant and credible link for a respective task. To evaluate the developed instrument, we conducted two studies: (1) a pre-study for quality assurance and observing the response process (cognitive interviews of n = 8 students) and (2) a main study aimed at investigating the psychometric properties of the EVON and its relation to other variables (n = 152 students). The results of the pre-study provided first evidence for a theoretically sound test construction with regard to students' item processing behavior. The results of the main study showed acceptable psychometric outcomes for a standardized screening instrument with a small number of items. The item design criteria affected the item difficulty as intended, and students' choice to visit a website had an impact on their task success. Furthermore, the probability of task success was positively predicted by general cognitive performance and reading skill. Although the results uncovered a few weaknesses (e.g., a lack of difficult items), and the efforts of validating the interpretation of EVON outcomes still need to be continued, the overall results speak in favor of a successful test construction and provide first indication that the EVON assesses students' skill in evaluating online information in search engine environments.

SUBMITTER: Hahnel C

PROVIDER: S-EPMC7773327 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Evaluation of Online Information in University Students: Development and Scaling of the Screening Instrument EVON.

Hahnel Carolin C Eichmann Beate B Goldhammer Frank F

Frontiers in psychology 20201216

As Internet sources provide information of varying quality, it is an indispensable prerequisite skill to evaluate the relevance and credibility of online information. Based on the assumption that competent individuals can use different properties of information to assess its relevance and credibility, we developed the EVON (evaluation of online information), an interactive computer-based test for university students. The developed instrument consists of eight items that assess the skill to evalu ...[more]

PMID: 33391077

Similar Datasets

Project description:BACKGROUND:The screening instrument for borderline personality disorder (SI-Bord) consists of a 5-item self-reported questionnaire on the key features of BPD from the DSM-5 using a 5-point Likert scale. This study investigated its validity and reliability in screening for BPD in university students. METHODS:A cross-sectional study was conducted on a sample of university students in Thailand between November and December 2019. An online assessment gathered demographic data and results from the SI-Bord, the Perceived Stress Scale-10 (PSS-10) and the Patient Health Questionnaire-9 (PHQ-9). Participants whose SI-Bord scores were???1 were randomly selected to be interviewed and assessed for a BPD diagnosis by four psychiatrists using the Structured Clinical Interview for DSM-IV Axis II Personality Disorders (SCID-II) as a reference point. An intraclass correlation coefficient (ICC) of 0.925 (95% CI, 0.805-0.979) ensured inter-rater reliability between the four psychiatrists. The diagnostic sensitivity and specificity of the SI-Bord, as compared to that of the SCID-II, were determined to indicate the cut-off score. The Receiver Operating Characteristics (ROC) was analyzed to evaluate its diagnostic accuracy. RESULTS:The study included 342 students aged 18-25?years (the mean age was 20.25?±?1.4?years), 80.4% of whom were female. Among the 68 participants selected for an online interview, 16 were diagnosed with BPD. The cut-off score of the SI-Bord was >?9, as suggested by the Youden index, yielding a sensitivity of 56.3% and a specificity of 92.3%. It had a positive predictive value of 69.2% and negative predictive value of 87.3%. The SI-Bord had adequate discriminative power between cases and non-cases of BPD, with the area under the ROC curve being 0.83. Cronbach's alpha for the SI-Bord was 0.76, indicating acceptable internal consistency. The SI-Bord score was positively correlated to PHQ-9 and PSS-10 scores (r?=?0.67 and r?=?0.69, p?<?0.001, respectively) and negatively correlated to MSPSS (r?=?-?0.50, p?<?0.001). The prevalence of BPD in the sample was 6.4%, according to the cut-off score?>?9. CONCLUSION:The SI-Bord demonstrated good reliability and validity for screening BPD in university students. However, a study in non-Thai and other population groups should be warranted.

Project description:BackgroundThe method by which mental health screening result reports are given affects the user's health behavior. Lists with the distribution of scores in various mental health areas is difficult for users to understand, and if the results are negative, they may feel more embarrassed than necessary. Therefore, we propose using group-tailored feedback, grouping people of similar mental health types by cluster analysis for comprehensive explanations of multidimensional mental health.MethodsThis cross-sectional, observational study was conducted using a qualitative approach based on cluster analysis. Data were collected via a developed mental screening website, with depression, anxiety, sleep problems, perfectionism, procrastination, and attention assessed for 2 weeks in January 2020 in Korea. Participants were randomly recruited, and sample size was 174. Total was divided into 25 with severe depression/anxiety (SDA+) and 149 without severe depression/anxiety (SDA-) according to the PHQ-9 and GAD-7 criteria. Cluster analysis was conducted in each group, and an ANOVA was performed to find significant clusters. Thereafter, structured discussion was performed with mental health professionals to define the features of the clusters and construct the feedback content initially. Thirteen expert counselors were interviewed to reconstruct the content and validate the effectiveness of the developed feedback.ResultsSDA- was divided into 3 using the k-means algorithm, which showed the best performance (silhouette score = 0.32, CH score = 91.67) among the clustering methods. Perfectionism and procrastination were significant factors in discretizing the groups. SDA+ subgroups were integrated because only 25 people belonged to this group, and they need professional help rather than self-care. Mental status and treatment recommendations were determined for each group, and group names were assigned to represent their features. The developed feedback was assessed to improve mental health literacy (MHL) through integrative and understandable explanations of multidimensional mental health. Moreover, it appeared that a sense of belonging was induced to reduce reluctance to face the feedback.ConclusionsThis study suggests group-tailored feedback using cluster analysis, which identifies groups of university students by integrating multidimensions of mental health. These methods can help students increase their interest in mental health and improve MHL to enable timely help.

Project description:BackgroundDecreased experiential learning opportunities exacerbated by the COVID-19 pandemic have increased development of online surgical educational courses. To what extent may such courses provide exposure to broad and accessible surgical education?MethodsSurge is a 6-week online surgical elective hosted within a virtual learning environment, covering all surgical specialties. Course content is mapped to the Royal College of Surgeons' Undergraduate Curriculum in Surgery. Each week consultant surgeons discuss their specialty in short videos on anatomy, pathology and lifestyle of a surgeon. Students also engage with learning activities; further reading lists; formative quizzes and live sessions including suturing practice. Participants were medical students undertaking third-year electives at the University of Sheffield. Pre- and post-course questionnaires investigated student interest in surgery, understanding of steps required to pursue a surgical career and confidence in surgical environments. Qualitative data was collected via free-text responses and analysed with content analysis. Quantitative data was collected using 5-point Likert scales (1 = Strongly Disagree; 5 = Strongly Agree) and analysed using the Wilcoxon signed-rank test.ResultsTwenty-two students participated in Surge over five 6-week cohorts. Examination of free-text responses revealed students gained increased understanding of available surgical career options. Students felt better informed regarding different surgical specialties (median score 2.5 vs. 4, p = 0.000) and steps required to develop a surgical portfolio (median score 2 vs. 5, p = 0.000). Additionally, confidence in understanding of relevant intraoperative steps improved (median score 3 vs. 4, p = 0.000).ConclusionThese data demonstrate Surge increased student confidence and understanding of surgical careers despite reduced in-person opportunities to engage with surgical education. Surge will continue to be developed and evaluated on a larger scale.

Project description:BACKGROUND:With abundant personal health information at hand, individuals are faced with a critical challenge in evaluating the informational value of health care records to keep useful information and discard that which is determined useless. Young, healthy college students who were previously dependents of adult parents or caregivers are less likely to be concerned with disease management. Personal health information management (PHIM) is a special case of personal information management (PIM) that is associated with multiple interactions among varying stakeholders and systems. However, there has been limited evidence to understand informational or behavioral underpinning of the college students' PHIM activities, which can influence their health in general throughout their lifetime. OBJECTIVE:This study aimed to investigate demographic and academic profiles of college students with relevance to PHIM activities. Next, we sought to construct major PHIM-related activity components and perceptions among college students. Finally, we sought to discover major factors predicting core PHIM activities among college students we sampled. METHODS:A Web survey was administered to collect responses about PHIM behaviors and perceptions among college students from the University of Kentucky from January through March 2017. A total of 1408 college students were included in the analysis. PHIM perceptions, demographics, and academic variations were used as independent variables to predict diverse PHIM activities using a principal component analysis (PCA) and hierarchical regression analyses (SPSS v.24, IBM Corp, Armonk, NY, USA). RESULTS:Majority of the participants were female (956/1408, 67.90%), and the age distribution of this population included an adequate representation of college students of all ages. The most preferred health information resources were family (612/1408, 43.47%), health care professionals (366/1408, 26.00%), friends (27/1408, 1.91%), and the internet (157/1408, 11.15%). Organizational or curatorial activities such as Arranging, Labeling, Categorizing, and Discarding were rated low (average=3.21, average=3.02, average=2.52, and average=2.42, respectively). The PCA results suggested 3 components from perception factors labeled as follows: Assistance (alpha=.85), Awareness (alpha=.716), and Difficulty (alpha=.558). Overall, the Demographics and Academics variables were not significant in predicting dependent variables such as Labeling, Categorizing, Health Education Materials, and Discarding, whereas they were significant for other outcome variables such as Sharing, Collecting, Knowing, Insurance Information, Using, and Owning. CONCLUSIONS:College years are a significant time for students to learn decision-making skills for maintaining information, a key aspect of health records, as well as for educators to provide appropriate educational and decision aids in the environment of learning as independent adults. Our study will contribute to better understand knowledge about specific skills and perceptions for college students' practice of effective PHIM throughout their lives.

Project description:BackgroundToo many young people engage in unhealthy behaviours such as eating unhealthily, being physically inactive, binge drinking and smoking. This study aimed to estimate the short-term and long-term cost-effectiveness of a theory-based online health behaviour intervention ("U@Uni") in comparison with control in young people starting university.MethodsA costing analysis was conducted to estimate the full cost of U@Uni and the cost of U@Uni roll-out. The short-term cost-effectiveness of U@Uni was estimated using statistical analysis of 6-month cost and health-related quality of life data from the U@Uni randomised controlled trial. An economic modelling analysis combined evidence from the trial with published evidence of the effect of health behaviours on mortality risk and general population data on health behaviours, to estimate the lifetime cost-effectiveness of U@Uni in terms of incremental cost per QALY. Costs and effects were discounted at 1.5% per annum. A full probabilistic sensitivity analysis was conducted to account for uncertainty in model inputs and provide an estimate of the value of information for groups of important parameters.ResultsTo implement U@Uni for the randomised controlled trial was estimated to cost £292 per participant, whereas roll-out to another university was estimated to cost £19.71, both giving a QALY gain of 0.0128 per participant. The short-term (6-month) analysis suggested that U@Uni would not be cost-effective at a willingness-to-pay threshold of £20,000 per QALY (incremental cost per QALY gained = £243,926). When a lifetime horizon was adopted the results suggest that the full implementation of U@Uni is unlikely to be cost-effective, whereas the roll-out of U@Uni to another university is extremely likely to be cost-effective. The value of information analysis suggests that the most important drivers of decision uncertainty are uncertainties in the effect of U@Uni on health behaviours.ConclusionsThe study provides the first estimate of the costs and cost-effectiveness of an online health behaviour intervention targeted at new university students. The results suggest that the roll-out, but not the full implementation, of U@Uni would be a cost-effective decision for the UK Department of Health, given a lifetime perspective and a willingness-to pay threshold of £20,000 per QALY.Trial registrationCurrent Controlled Trials ISRCTN67684181.

Project description:Seminal to the process of a health sciences curriculum evaluation is the periodic review of clinical assessment instruments that measure competency. An assessment of quality is facilitated by using a well-structured, authentic and reliable instrument. This process rests on designing and measuring the instrument against a sound framework and validating it for scientific merit. This paper documents the pedagogy and the process taken in developing an improved formative competency-based assessment instrument for the final year students of the Bachelor of Oral Health program (BOH) at the University of the Western Cape (UWC). Methods: A qualitative research study design employing the Nominal Group Technique (NGT) was used as a method for gaining small group consensus on the clinical assessment instrument for exit level Oral Hygiene (BOH3) students within the parameters of assessment principles. The key contributors to the instrument development process were the academic staff of the Department of Oral Hygiene, involved in clinical teaching and assessment of student competency. Results: The domains of ethics and professionalism, patient assessment, diagnosis, treatment planning and implementation was identified as the core elements in the assessment. The principles of assessment, which include, alignment with outcomes, feedback, transparency and validity, were used to guide the instrument development. The assessment criteria were cross examined for alignment to the learning outcomes of the module and the program whilst formative feedback was foregrounded as a central feature to support student learning and progress monitoring. Transparency was obtained by providing students access to the instrument before and after the assessment including the written feedback on their performance. The instrument embodied a range of criteria to be assessed rather than on the awarding of a cumulative score. This allowed for the identification of the criteria or domain within which a student is struggling or excelling. Consensus on the instrument design was achieved using the NGT phases throughout the instrument development process including the weighting of the domains and grading. This level of engagement together with the application of scientifically sound assessment principles contributed to the validation of the instrument. Conclusion: The development of a competency-based assessment instrument was the result of a structured, collaborative and scientifically engaged process framed around specific assessment principles. The process culminated in the development of a formative competency-based clinical assessment instrument that was fit for purpose in the Bachelor of Oral Health program. The Nominal Group Technique served to be a valuable approach for small group consensus in developing the instrument. It served to promote individual perspectives and to generate debate and group discussion between academics that were proficient in clinical teaching and, finally to facilitate group consensus on the instrument structure and system for administration. Supplementary Information The online version contains supplementary material available at 10.1186/s12903-022-02498-3.

Dataset Information

Evaluation of Online Information in University Students: Development and Scaling of the Screening Instrument EVON.

Publications

Evaluation of Online Information in University Students: Development and Scaling of the Screening Instrument EVON.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets