A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity.
Ontology highlight
ABSTRACT:
SUBMITTER: Butler-Henderson K
PROVIDER: S-EPMC7508171 | biostudies-literature | 2020 Dec
REPOSITORIES: biostudies-literature
Publications
A systematic review of online examinations: A pedagogical innovation for scalable authentication and integrity.
Butler-Henderson Kerryn K Crawford Joseph J
Computers & education 20200922
Digitization and automation across all industries has resulted in improvements in efficiencies and effectiveness to systems and process, and the higher education sector is not immune. Online learning, e-learning, electronic teaching tools, and digital assessments are not innovations. However, there has been limited implementation of online invigilated examinations in many countries. This paper provides a brief background on online examinations, followed by the results of a systematic review on t ...[more]
Similar Datasets
Project description:Academic integrity is at the heart of excellent education. However, resources explaining the concept tend to be definition-driven, while using complex language and sometimes even an austere tone designed to discourage students from breaches. This study aims to design and evaluate an online module at a UK University across 2 years, designed to improve students' understanding of concepts of academic integrity and practice. The module includes a range of interactive resources (e.g., gamified quizzes and e-booklets) and was made available to a large cohort of postgraduate students (448). The study adopts a mixed-methods approach composed of three sequential phases involving first collecting students' views on existing academic integrity resources (7 students participating in a focus group and 39 competing a questionnaire), then developing a range of new ones based on students' feedback to form the content of the module, and finally gathering students' evaluation on the newly created resources (sample size: 361 students). Results illustrate a clear improvement in relation to the accessibility, usefulness and understandability of new resources. Results also highlight a remarkable increase in student confidence levels regarding academic integrity. Students also considered the new module as more appealing and informative. This manuscript offers a good example of a pedagogical approach aimed at promoting academic integrity in an innovative and engaging fashion.
| S-EPMC9581242 | biostudies-literature
Project description:BACKGROUND:Patient participation in the health care domain has surged dramatically through the availability of digital health platforms and online health communities (OHCs). Such patient-driven service innovation has both potential and challenges for health care organizations. Over the last 5 years, articles have surfaced that focus on value cocreation in health care services and the importance of engaging patients and other actors in service delivery. However, a theoretical understanding of how to use OHCs for this purpose is still underdeveloped within the health care service ecosystem. OBJECTIVE:This paper aimed to introduce a theoretical discussion for better understanding of the potential of OHCs for health care organizations, in particular, for patient empowerment. METHODS:This literature review study involved a comprehensive search using 12 electronic databases (EMBASE, PsycINFO, Web of Science, Scopus, ScienceDirect, Medical Literature Analysis and Retrieval System Online, PubMed, Elton B Stephens Co [academic], Cumulative Index of Nursing and Allied Health Literature, Accelerated Information Sharing for Law Enforcement, Association for Computing Machinery, and Google Scholar) from 2013 to 2019. A total of 1388 studies were identified from the database search. After removing duplicates and applying inclusion criteria, we thematically analyzed 56 articles using the Braun and Clarke thematic analysis approach. RESULTS:We identified a list of 5 salient themes: communication extension, improved health literacy for patients and health care organizations, communication transparency with patients, informational and social support for patients, and patient empowerment in self-management. The most frequent theme was communication extension, which covers 39% (22/56) of the literature. This theme reported that an extension of communication between patients, caregivers, and physicians and organizations led to new opportunities to create value with minimal time and cost restrictions. Improved health literacy and communication transparency with patients were the second and third most frequent themes, respectively, covering 26% (15/56) and 25% (14/56) of the literature, respectively. The frequency of these themes indicated that the use of OHCs to generate new knowledge from patients' interactions helped health care organizations to customize treatment plans and establish transparent and effective communication between health care organizations and patients. Furthermore, of the 56 studies, 13 (23%) and 10 (17%) studies contended the opportunity of using OHCs in terms of informational and emotional support and empowering patients in their self-management of diseases. CONCLUSIONS:This review enables better understanding of the current state of the art of the online value cocreation and its potential for health care organizations. This study found that the opportunities for health care organizations through enhancement of patient participation and their cocreation of value in digital health platforms have been rapidly increasing. The identified gaps and opportunities in this study would identify avenues for future directions in modernized and more effective value-oriented health care informatics research.
| S-EPMC7210492 | biostudies-literature
Project description:The coronavirus disease 2019 (COVID-19) pandemic has required educators to adapt the in-person objective structured clinical examination (OSCE) to online settings in order for it to remain a critical component of the multifaceted assessment of a student’s competency. This systematic scoping review aimed to summarize the assessment methods and validity and reliability of the measurement tools used in current online OSCE (hereafter, referred to as teleOSCE) approaches. A comprehensive literature review was undertaken following the Preferred Reporting Items for Systematic Reviews and Meta-Analyses extension for Scoping Reviews guidelines. Articles were eligible if they reported any form of performance assessment, in any field of healthcare, delivered in an online format. Two reviewers independently screened the results and analyzed relevant studies. Eleven articles were included in the analysis. Pre-recorded videos were used in 3 studies, while observations by remote examiners through an online platform were used in 7 studies. Acceptability as perceived by students was reported in 2 studies. This systematic scoping review identified several insights garnered from implementing teleOSCEs, the components transferable from telemedicine, and the need for systemic research to establish the ideal teleOSCE framework. TeleOSCEs may be able to improve the accessibility and reproducibility of clinical assessments and equip students with the requisite skills to effectively practice telemedicine in the future.
| S-EPMC8212027 | biostudies-literature
Project description:Tutorial tactics are policies for an Intelligent Tutoring System (ITS) to decide the next action when there are multiple actions available. Recent research has demonstrated that when the learning contents were controlled so as to be the same, different tutorial tactics would make difference in students' learning gains. However, the Reinforcement Learning (RL) techniques that were used in previous studies to induce tutorial tactics are insufficient when encountering large problems and hence were used in offline manners. Therefore, we introduced a Genetic-Based Reinforcement Learning (GBML) approach to induce tutorial tactics in an online-learning manner without basing on any preexisting dataset. The introduced method can learn a set of rules from the environment in a manner similar to RL. It includes a genetic-based optimizer for rule discovery task by generating new rules from the old ones. This increases the scalability of a RL learner for larger problems. The results support our hypothesis about the capability of the GBML method to induce tutorial tactics. This suggests that the GBML method should be favorable in developing real-world ITS applications in the domain of tutorial tactics induction.
| S-EPMC4439506 | biostudies-other
Project description:BackgroundWriting multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive medical knowledge, time and effort from medical educators. This systematic review focuses on the application of large language models (LLMs) in generating medical MCQs.MethodsThe authors searched for studies published up to November 2023. Search terms focused on LLMs generated MCQs for medical examinations. Non-English, out of year range and studies not focusing on AI generated multiple-choice questions were excluded. MEDLINE was used as a search database. Risk of bias was evaluated using a tailored QUADAS-2 tool.ResultsOverall, eight studies published between April 2023 and October 2023 were included. Six studies used Chat-GPT 3.5, while two employed GPT 4. Five studies showed that LLMs can produce competent questions valid for medical exams. Three studies used LLMs to write medical questions but did not evaluate the validity of the questions. One study conducted a comparative analysis of different models. One other study compared LLM-generated questions with those written by humans. All studies presented faulty questions that were deemed inappropriate for medical exams. Some questions required additional modifications in order to qualify.ConclusionsLLMs can be used to write MCQs for medical examinations. However, their limitations cannot be ignored. Further study in this field is essential and more conclusive evidence is needed. Until then, LLMs may serve as a supplementary tool for writing medical examinations. 2 studies were at high risk of bias. The study followed the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) guidelines.
| S-EPMC10981304 | biostudies-literature
Project description:Purpose of the articleDuring the Covid-19 pandemic, formative OSCE were transformed into online OSCE, and senior students (near peers) substituted experienced clinical teachers. The aims of the study were to evaluate quality of the feedbacks given by near peers during online OSCEs and explore the experience of near-peer feedback from both learner's and near peer's perspectives.Materials and methodsAll 2nd year medical students (n = 158) attended an online OSCE under the supervision of twelve senior medical students. Outcome measures were 1) students' perception of the quality of the feedback through an online survey (Likert 1-5); 2) objective assessment of the quality of the feedback focusing on both the process and the content using a feedback scale (Likert 1-5); 3) experience of near peer feedback in two different focus groups.ResultsOne hundred six medical students answered the questionnaire and had their feedback session videotaped. The mean perceived overall quality of senior students' overall feedback was 4.75 SD 0.52. They especially valued self-evaluation (mean 4.80 SD 0.67), balanced feedback (mean 4.93 SD 0.29) and provision of simulated patient's feedback (mean 4.97 SD 0.17). The overall objective assessment of the feedback quality was 3.73 SD 0.38: highly scored skills were subjectivity (mean 3.95 SD 1.12) and taking into account student's self-evaluation (mean 3.71 (SD 0.87). Senior students mainly addressed history taking issues (mean items 3.53 SD 2.37) and communication skills (mean items 4.89 SD 2.43) during feedback. Participants reported that near peer feedback was less stressful and more tailored to learning needs- challenges for senior students included to remain objective and to provide negative feedback.ConclusionIncreased involvement of near peers in teaching activities is strongly supported for formative OSCE and should be implemented in parallel even if experience teachers are again involved in such teaching activities. However, it requires training not only on feedback skills but also on the specific content of the formative OSCE.
| S-EPMC9310367 | biostudies-literature
Project description:We undertook a systematic review evaluating the effectiveness and cost of health-focused innovation design contests. We followed Cochrane guidance and systematically searched eight databases. Articles were included if they reported an open contest focused on improving health, required participants submit finished design solutions and contained a prize structure. We searched 3579 citations, identifying 29 health-focused innovation design contests which collectively received 15494 contest submissions. Contests solicited submissions worldwide (7) and exclusively from North America (13), Asia (4), Africa (2), Australia (2) and Europe (1). Submissions per contest ranged from 3 to 11354 (median of 73). Contest entrants were tasked with helping develop health promotion messages (HPM) (25) and improve predictive clinical models, protocols and/or algorithms (4). Two types of contests emerged-those focused on high-quality, innovative outcomes and those focused on the process of mass community engagement. All outcome-oriented contests had innovation design contest solutions equivalent or superior to the comparator (100%; 7/7). Two of two studies found that innovation design contests saved money (100%; 2/2). Five of seven process-oriented contests concluded the contest improved at least one health indicator (71%; 5/7). Innovation design contests are an effective way to solicit innovative solutions to health problems and spur mass community engagement.
| S-EPMC5863925 | biostudies-literature
Project description:The aim of the present article was to systematically review the ethics of surgical innovation and introduce the components of the learning health care system to guide future research and debate on surgical innovation.Although the call for evidence-based practice in surgery is increasingly high on the agenda, most surgeons feel that the format of the randomized controlled trial is not suitable for surgery. Innovation in surgery has aspects of, but should be distinguished from both research and clinical care and raises its own ethical challenges.To answer the question "What are the main ethical aspects of surgical innovation?", we systematically searched PubMed and Embase. Papers expressing an opinion, point of view, or position were included, that is, normative ethical papers.We included 59 studies discussing ethical aspects of surgical innovation. These studies discussed 4 major themes: oversight, informed consent, learning curve, and vulnerable patient groups. Although all papers addressed the ethical challenges raised by surgical innovation, surgeons hold no uniform view of surgical innovation, and there is no agreement on the distinction between innovation and research. Even though most agree to some sort of oversight, they offer different alternatives ranging from the formation of new surgical innovation committees to establishing national registries. Most agree that informed consent is necessary for innovative procedures and that surgeons should be adequately trained to assure their competence to tackle the learning curve problem. All papers agree that in case of vulnerable patients, alternatives must be found for the informed consent procedure.We suggest that the concept of the learning health care system might provide guidance for thinking about surgical innovation. The underlying rationale of the learning health care system is to improve the quality of health care by embedding research within clinical care. Two aspects of a learning health care system might particularly enrich the necessary future discussion on surgical innovation: integration of research and practice and a moral emphasis on "learning activities." Future research should evaluate whether the learning health care system and its adjacent moral framework provides ethical guidance for evidence-based surgery.
| S-EPMC4998304 | biostudies-literature
Project description:Diffusion of innovation can be interpreted as a social spreading phenomenon governed by the impact of media and social interactions. Although these mechanisms have been identified by quantitative theories, their role and relative importance are not entirely understood, as empirical verification has so far been hindered by the lack of appropriate data. Here we analyse a dataset recording the spreading dynamics of the world's largest Voice over Internet Protocol service to empirically support the assumptions behind models of social contagion. We show that the rate of spontaneous service adoption is constant, the probability of adoption via social influence is linearly proportional to the fraction of adopting neighbours, and the rate of service termination is time-invariant and independent of the behaviour of peers. By implementing the detected diffusion mechanisms into a dynamical agent-based model, we are able to emulate the adoption dynamics of the service in several countries worldwide. This approach enables us to make medium-term predictions of service adoption and disclose dependencies between the dynamics of innovation spreading and the socio-economic development of a country.
| S-EPMC4223898 | biostudies-other
Project description:BackgroundSurgical innovation is different from the introduction of novel pharmaceuticals. To help address this, in 2009 the IDEAL Collaboration (Idea, Development, Exploration, Assessment, Long-term follow-up) introduced the five-stage framework for surgical innovation. To evaluate the framework feasibility for novel neurosurgical procedure introduction, two innovative surgical procedures were examined: the endoscopic endonasal approach for skull base meningiomas (EEMS) and the WovenEndobridge (WEB device) for endovascular treatment of intracranial aneurysms.MethodsThe published literature on EEMS and WEB devices was systematically reviewed. Identified studies were classified according to the IDEAL framework stage. Next, studies were evaluated for possible categorization according to the IDEAL framework.ResultsFive hundred seventy-six papers describing EEMS were identified of which 26 papers were included. No prospective studies were identified, and no studies reported on ethical approval or patient informed consent for the innovative procedure. Therefore, no clinical studies could be categorized according to the IDEAL Framework. For WEB devices, 6229 articles were screened of which 21 were included. In contrast to EEMS, two studies were categorized as 2a and two as 2b.ConclusionThe results of this systematic review demonstrate that both EEMS and WEB devices were not introduced according to the (later developed in the case of EEMS) IDEAL framework. Elements of the framework such as informed consent, ethical approval, and rigorous outcomes reporting are important and could serve to improve the quality of neurosurgical research. Alternative study designs and the use of big data could be useful modifications of the IDEAL framework for innovation in neurosurgery.
| S-EPMC5590028 | biostudies-literature