Using Kahoot! as a formative assessment tool in medical education: a phenomenological study.
Ontology highlight
ABSTRACT:
SUBMITTER: Ismail MA
PROVIDER: S-EPMC6593549 | biostudies-other | 2019 Jun
REPOSITORIES: biostudies-other
Publications
Using Kahoot! as a formative assessment tool in medical education: a phenomenological study.
Ismail Muhd Al-Aarifin MA Ahmad Anisa A Mohammad Jamilah Al-Muhammady JA Fakri Nik Mohd Rizal Mohd NMRM Nor Mohd Zarawi Mat MZM Pa Mohamad Najib Mat MNM
BMC medical education 20190625 1
<h4>Background</h4>Gamification is an increasingly common phenomenon in education. It is a technique to facilitate formative assessment and to promote student learning. It has been shown to be more effective than traditional methods. This phenomenological study was conducted to explore the advantages of gamification through the use of the Kahoot! platform for formative assessment in medical education.<h4>Methods</h4>This study employed a phenomenological design. Five focus groups were conducted ...[more]
Similar Datasets
Project description:In self-directed learning environments, students must monitor and assess their learning progress themselves. Formative assessments play a central role as instructional tools to train students in these self-assessment skills. Key to the effectiveness of a formative assessment as a training instrument is the degree of student involvement in the design and implementation of the formative assessment used to train them. Five characteristics are essential to secure a sufficient level of student involvement: (1) congruence (i.e., the tasks of a formative assessment must reflect the instructional content), (2) authenticity (i.e., the tasks of a formative assessment should be related to students' background and study context), (3) consultation (i.e., students should have a say in how their answers are evaluated based on which criteria), (4) transparency (i.e., no "mystery items"-the wording of the items must clearly addresses the targeted content), and (5) accommodation (i.e., all students should have the same chance to complete the items of a formative assessment). These five characteristics of student involvement were used to evaluate the quality of formative assessments that are employed to train undergraduates in a medical school in self-assessment skills. Different from extant approaches that typically rely on instructors' judgments to infer the degree of student involvement, this study uses a paradigm in higher educational research asking students directly about their perceptions of an instructional tool. Thus, here, students had to rate their level of involvement in the design and implementation of the formative assessments used to train them. A total of 140 medical students participated in the study. Multivariate statistical methods were used to analyze the data. The majority of medical students appreciate the use of formative assessments to teach them self-directed learning skills; but the results also show that there is still room for increasing student involvement-especially, for minority students.
| S-EPMC8368904 | biostudies-literature
Project description:ObjectiveTo characterize the psychometric properties of a novel Obstetric Communication Assessment Tool (OCAT) in a pilot study of standardized difficult OB communication scenarios appropriate for undergraduate medical evaluation.MethodsWe developed and piloted four challenging OB Standardized Patient (SP) scenarios in a sample of twenty-one third year OB/GYN clerkship students: Religious Beliefs (RB), Angry Father (AF), Maternal Smoking (MS), and Intimate Partner Violence (IPV). Five trained Standardized Patient Reviewers (SPRs) independently scored twenty-four randomized video-recorded encounters using the OCAT. Cronbach's alpha and Intraclass Correlation Coefficient-2 (ICC-2) were used to estimate internal consistency (IC) and inter-rater reliability (IRR), respectively. Systematic variation in reviewer scoring was assessed using the Stuart-Maxwell test.ResultsIC was acceptable to excellent with Cronbach's alpha values (and 95% Confidence Intervals [CI]): RB 0.91 (0.86, 0.95), AF 0.76 (0.62, 0.87), MS 0.91 (0.86, 0.95), and IPV 0.94 (0.91, 0.97). IRR was unacceptable to poor with ICC-2 values: RB 0.46 (0.40, 0.53), AF 0.48 (0.41, 0.54), MS 0.52 (0.45, 0.58), and IPV 0.67 (0.61, 0.72). Stuart-Maxwell analysis indicated systematic differences in reviewer stringency.ConclusionsOur initial characterization of the OCAT demonstrates important issues in communications assessment. We identify scoring inconsistencies due to differences in SPR rigor that require enhanced training to improve assessment reliability. We outline a rational process for initial communication tool validation that may be useful in undergraduate curriculum development, and acknowledge that rigorous validation of OCAT training and implementation is needed to create a valuable OB communication assessment tool.
| S-EPMC4912696 | biostudies-literature
Project description:ObjectivesFormative peer assessment focuses on learning and development of the student learning process. This implies that students are taking responsibility for assessing the work of their peers by giving and receiving feedback to each other. The aim was to compile research about formative peer assessment presented in higher healthcare education, focusing on the rationale, the interventions, the experiences of students and teachers and the outcomes of formative assessment interventions.DesignA scoping review.Data sourcesSearches were conducted until May 2019 in PubMed, Cumulative Index to Nursing and Allied Health Literature, Education Research Complete and Education Research Centre. Grey literature was searched in Library Search, Google Scholar and Science Direct.Eligibility criteriaStudies addressing formative peer assessment in higher education, focusing on medicine, nursing, midwifery, dentistry, physical or occupational therapy and radiology published in peer-reviewed articles or in grey literature.Data extractions and synthesisOut of 1452 studies, 37 met the inclusion criteria and were critically appraised using relevant Critical Appraisal Skills Programme, Joanna Briggs Institute and Mixed Methods Appraisal Tool tools. The pertinent data were analysed using thematic analysis.ResultThe critical appraisal resulted in 18 included studies with high and moderate quality. The rationale for using formative peer assessment relates to giving and receiving constructive feedback as a means to promote learning. The experience and outcome of formative peer assessment interventions from the perspective of students and teachers are presented within three themes: (1) organisation and structure of the formative peer assessment activities, (2) personal attributes and consequences for oneself and relationships and (3) experience and outcome of feedback and learning.ConclusionHealthcare education must consider preparing and introducing students to collaborative learning, and thus develop well-designed learning activities aligned with the learning outcomes. Since peer collaboration seems to affect students' and teachers' experiences of formative peer assessment, empirical investigations exploring collaboration between students are of utmost importance.
| S-EPMC7875268 | biostudies-literature
Project description:ObjectivesContinuous formative assessment with appropriate feedback is the pillar of effective clinical teaching and learning. Group Objective Structured Clinical Examination (GOSCE) has been reported as a resource-effective method of formative assessment. The present study aims to describe the development and evaluation of GOSCE as a formative assessment for pre-clerkship medical students.MethodsAt the University of Sharjah, GOSCE was introduced to medical students in Years 1, 2, and 3. The GOSCE was conducted as a formative assessment in which groups of 4-5 students were observed while they performed various clinical skills, followed by structured feedback from clinical tutors and peers. GOSCE was evaluated both quantitatively and qualitatively and appropriate statistical analysis was applied to evaluate their responses.ResultsA total of 232 students who attended the GOSCE responded to the questionnaires. Most of the students and clinical tutors preferred formative GOSCE over individual feedback. Both students and clinical tutors valued the experience as it helped students to identify gaps and to share knowledge and skills among group members.ConclusionThis study found that formative GOSCE provided a valuable and feasible educational opportunity for students to receive feedback about their clinical skills.
| S-EPMC6694983 | biostudies-literature
Project description:IntroductionIn formative peer assessment, the students give and receive feedback from each other and expand their knowledge in a social context of interaction and collaboration. The ability to collaborate and communicate are essential parts of the healthcare professionals' competence and delivery of safe patient care. Thereby, it is of utmost importance to support students with activities fostering these competences during their healthcare education. The aim of the scoping review is to compile research on peer assessment presented in healthcare education programmes, focusing formative assessment. The result of the scoping review will form the basis for developing and conducting an intervention focusing collaborative learning and peer assessment in a healthcare education programme.Methods and analysisThe scoping review will be conducted by using the framework presented by Arksey & O'Malley and Levac et al. The primary research question is: How are formative peer assessment interventions delivered in healthcare education? The literature search will be conducted in the peer-reviewed databases PubMed, Cumulative Index to Nursing and Allied Health Literature, Education Research Complete and Education Research Centre between September and December 2018. Additional search will be performed in Google Scholar, hand-searching of reference lists of included studies and Libsearch for identification of grey literature. Two researchers will independently screen title and abstract. Full-text articles will be screened by three researchers using a charting form. Studies meeting the inclusion criteria will be critically evaluated using the Critical Appraisal Skills Programme. A flow diagram will present the included and excluded studies. A narrative synthesis will be conducted by using thematic analysis as presented by Braun and Clarke. The findings will be presented under thematic headings using a summary table. To enhance validity, stakeholders from healthcare education programmes and healthcare institutions will be provided with an overview of the preliminary results.Ethics and disseminationResearch ethics approval is not required for the scoping review.
| S-EPMC6231592 | biostudies-literature
Project description:BackgroundThe adoption of competency-based medical education requires objective assessments of a learner's capability to carry out clinical tasks within workplace-based learning settings. This study involved an evaluation of the use of mobile technology to record entrustable professional activity assessments in an undergraduate clerkship curriculum.ApproachA paper-based form was adapted to a mobile platform called eClinic Card. Students documented workplace-based assessments throughout core clerkship and preceptors confirmed accuracy via mobile phones. Assessment scores for the 2017-2018 academic year were collated and analyzed for all core rotations, and preceptors and students were surveyed regarding the mobile assessment experience.EvaluationThe mobile system enabled 80 students and 624 preceptors to document 6850 assessment submissions across 47 clinical sites over a 48-week core clerkship curriculum. Students' scores demonstrated progressive improvement across all entrustable professional activities with stage-appropriate levels of independence reported by end of core clerkship. Preceptors and students were satisfied with ease of use and dependability of the mobile assessment platform; however, students felt quality of formative coaching feedback could be improved.ReflectionOur preliminary evaluation suggests the use of mobile technology to assess entrustable professional activity achievement across a core clerkship curriculum is a feasible and acceptable modality for workplace-based assessment. The use of mobile technology supported a programmatic assessment approach. However, meaningful coaching feedback, as well as faculty development and support, emerged as key factors influencing successful adoption and usage of entrustable professional activities within an undergraduate medical curriculum.
| S-EPMC8633342 | biostudies-literature
Project description:To evaluate the merit of the Clinical Audit Project (CAP) in an assessment program for undergraduate medical education using a systematic assessment validation framework.A cross-sectional assessment validation study at one medical school in Western Australia, with retrospective qualitative analysis of the design, development, implementation and outcomes of the CAP, and quantitative analysis of assessment data from four cohorts of medical students (2011- 2014).The CAP is fit for purpose with clear external and internal alignment to expected medical graduate outcomes. Substantive validity in students' and examiners' response processes is ensured through relevant methodological and cognitive processes. Multiple validity features are built-in to the design, planning and implementation process of the CAP. There is evidence of high internal consistency reliability of CAP scores (Cronbach's alpha > 0.8) and inter-examiner consistency reliability (intra-class correlation>0.7). Aggregation of CAP scores is psychometrically sound, with high internal consistency indicating one common underlying construct. Significant but moderate correlations between CAP scores and scores from other assessment modalities indicate validity of extrapolation and alignment between the CAP and the overall target outcomes of medical graduates. Standard setting, score equating and fair decision rules justify consequential validity of CAP scores interpretation and use.This study provides evidence demonstrating that the CAP is a meaningful and valid component in the assessment program. This systematic framework of validation can be adopted for all levels of assessment in medical education, from individual assessment modality, to the validation of an assessment program as a whole.
| S-EPMC5056027 | biostudies-literature
Project description:BACKGROUND:Social networking sites, in particular Facebook, are not only predominant in students' social life but are to varying degrees interwoven with the medical curriculum. Particularly, Facebook groups have been identified for their potential in higher education. However, there is a paucity of data on user types, content, and dynamics of study-related Facebook groups. OBJECTIVE:The aim of this study was to identify the role of study-related Facebook group use, characterize medical students that use or avoid using Facebook groups (demographics, participation pattern, and motivation), and analyze student posting behavior, covered topics, dynamics, and limitations in Facebook groups with regards to educational usage. METHODS:Using a multi-method approach (interviews, focus groups, and qualitative and quantitative analysis of Facebook posts), we analyzed two representative Facebook groups of medical preclinical semesters at Ludwig-Maximilians-University (LMU) Munich. Facebook primary posts and replies over one semester were extracted and evaluated by using thematic content analysis. We developed and applied a coding scheme for studying the frequency and distribution of these posts. Additionally, we interviewed students with various degrees of involvement in the groups, as well as "new minorities," students not registered on Facebook. RESULTS:Facebook groups seem to have evolved as the main tool for medical students at LMU to complement the curriculum and to discuss study-related content. These Facebook groups are self-organizing and quickly adapt to organizational or subject-related challenges posed by the curriculum. A wide range of topics is covered, with a dominance of organization-related posts (58.35% [6916/11,853] of overall posts). By measuring reply rates and comments per category, we were able to identify learning tips and strategies, material sharing, and course content discussions as the most relevant categories. Rates of adequate replies in these categories ranged between 78% (11/14) and 100% (13/13), and the number of comments per post ranged from 8.4 to 13.7 compared with the average overall reply rate of 68.69% (1167/1699) and 3.9 comments per post. User typology revealed social media drivers (>30 posts per semester) as engines of group function, frequent users (11-30 posts), and a majority of average users acting rather as consumers or lurkers (1-10 posts). CONCLUSIONS:For the moment, the medical faculty has no active involvement in these groups and therefore no influence on accuracy of information, professionalism, and ethical issues. Nevertheless, faculty could in the future benefit by extracting relevant information, identifying common problems, and understanding semester-related dynamics.
| S-EPMC5756318 | biostudies-literature
Project description:Introduction:Prior research has identified seven elements of a good assessment, but the elements have not been operationalized in the form of a rubric to rate assessment utility. It would be valuable for medical educators to have a systematic way to evaluate the utility of an assessment in order to determine if the assessment used is optimal for the setting. Methods:We developed and refined an assessment utility rubric using a modified Delphi process. Twenty-nine graduate students pilot-tested the rubric in 2016 with hypothetical data from three examinations, and interrater reliability of rubric scores was measured with interclass correlation coefficients (ICCs). Results:Consensus for all rubric items was reached after three rounds. The resulting assessment utility rubric includes four elements (equivalence, educational effect, catalytic effect, acceptability) with three items each, one element (validity evidence) with five items, and space to provide four feasibility items relating to time and cost. Rater scores had ICC values greater than .75. Discussion:The rubric shows promise in allowing educators to evaluate the utility of an assessment specific to their setting. The medical education field needs to give more consideration to how an assessment drives learning forward, how it motivates trainees, and whether it produces acceptable ranges of scores for all stakeholders.
| S-EPMC6338154 | biostudies-other
Project description:INTRODUCTION:Assessment in Medical Education fills many roles and is under constant scrutiny. Assessments must be of good quality, and supported by validity evidence. Given the high-stakes consequences of assessment, and the many audiences within medical education (e.?g., training level, specialty-specific), we set out to document the breadth, scope, and characteristics of the literature reporting on validation of assessments within medical education. METHOD:Searches in Medline (Ovid), Web of Science, ERIC, EMBASE (Ovid), and PsycINFO (Ovid) identified articles reporting on assessment of learners in medical education published since 1999. Included articles were coded for geographic origin, journal, journal category, targeted assessment, and authors. A map of collaborations between prolific authors was generated. RESULTS:A total of 2,863 articles were included. The majority of articles were from the United States, with Canada producing the most articles per medical school. Most articles were published in journals with medical categorizations (73.1% of articles), but Medical Education was the most represented journal (7.4% of articles). Articles reported on a variety of assessment tools and approaches, and 89 prolific authors were identified, with a total of 228 collaborative links. DISCUSSION:Literature reporting on validation of assessments in medical education is heterogeneous. Literature is produced by a broad array of authors and collaborative networks, reported to a broad audience, and is primarily generated in North American and European contexts. Our findings speak to the heterogeneity of the medical education literature on assessment validation, and suggest that this heterogeneity may stem, at least in part, from differences in constructs measured, assessment purposes, or conceptualizations of validity.
| S-EPMC6002290 | biostudies-literature