Dataset Information

Students tell us what good written feedback looks like.

ABSTRACT: Feedback can be an important element of learning, but only if students engage with it. Students are only likely to engage with feedback that they find useful. This study aimed to identify characteristics of written feedback perceived by students as effective. We used a mixed-method approach, integrating quantitative and qualitative data that were collected through the analysis of feedback that was identified by students as good, a student questionnaire, as well as interviews and a focus group exploring students' views on what good feedback looks like. Although the results show that length and composition of 'good' feedback can be extremely variable, some common characteristics could be identified, leading to a set of recommendations for staff marking written assessments. According to students, good feedback should be detailed and specific, and it should tell students how they can improve. Students also find it important that feedback is honest and constructive. In addition, positive reinforcement was identified as important by the focus group, although few examples of good written feedback on the assignment contained any direct praise. Surprisingly, feedforward which might help students in other modules did not feature highly in students' perceptions of good feedback, possibly indicating a focus by students on improving the current assignment rather than on future assignments.

SUBMITTER: Voelkel S

PROVIDER: S-EPMC7193163 | biostudies-literature | 2020 May

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Students tell us what good written feedback looks like.

Voelkel Susanne S Varga-Atkins Tunde T Mello Luciane V LV

FEBS open bio 20200330 5

Feedback can be an important element of learning, but only if students engage with it. Students are only likely to engage with feedback that they find useful. This study aimed to identify characteristics of written feedback perceived by students as effective. We used a mixed-method approach, integrating quantitative and qualitative data that were collected through the analysis of feedback that was identified by students as good, a student questionnaire, as well as interviews and a focus group ex ...[more]

PMID: 32176832

Similar Datasets

Project description:BackgroundThe Objective Structured Clinical Exam (OSCE) is a useful means of generating meaningful feedback. OSCE feedback may be in various forms (written, face to face and audio or video recordings). Studies on OSCE feedback are uncommon, especially involving Asian medical students.MethodsWe compared two methods of OSCE feedback delivered to fourth year medical students in Malaysia: (i) Face to face (FTF) immediate feedback (semester one) (ii) Individualised enhanced written (EW) feedback containing detailed scores in each domain, examiners' free text comments and the marking rubric (semester two). Both methods were evaluated by students and staff examiners, and students' responses were compared against their OSCE performance.ResultsOf the 116 students who sat for both formative OSCEs, 82.8% (n=96) and 86.2% (n=100) responded to the first and second survey respectively. Most students were comfortable to receive feedback (91.3% in FTF, 96% in EW) with EW feedback associated with higher comfort levels (p=0.022). Distress affected a small number with no differences between either method (13.5% in FTF, 10% in EW, p=0.316). Most students perceived both types of feedback improved their performance (89.6% in FTF, 95% in EW); this perception was significantly stronger for EW feedback (p=0.008). Students who preferred EW feedback had lower OSCE scores compared to those preferring FTF feedback (mean scores ± SD: 43.8 ± 5.3 in EW, 47.2 ± 6.5 in FTF, p=0.049). Students ranked the "marking rubric" to be the most valuable aspect of the EW feedback. Tutors felt both methods of feedback were equally beneficial. Few examiners felt they needed training (21.4% in FTF, 15% in EW) but students perceived this need for tutors' training differently (53.1% in FTF, 46% in EW) CONCLUSION: Whilst both methods of OSCE feedback were highly valued, students preferred to receive EW feedback and felt it was more beneficial. Learning cultures of Malaysian students may have influenced this view. Information provided in EW feedback should be tailored accordingly to provide meaningful feedback in OSCE exams.

Project description:PurposeDeveloping medical students' clinical reasoning requires a structured longitudinal curriculum with frequent targeted assessment and feedback. Performance-based assessments, which have the strongest validity evidence, are currently not feasible for this purpose because they are time-intensive to score. This study explored the potential of using machine learning technologies to score one such assessment-the diagnostic justification essay.MethodFrom May to September 2018, machine scoring algorithms were trained to score a sample of 700 diagnostic justification essays written by 414 third-year medical students from the Southern Illinois University School of Medicine classes of 2012-2017. The algorithms applied semantically based natural language processing metrics (e.g., coherence, readability) to assess essay quality on 4 criteria (differential diagnosis, recognition and use of findings, workup, and thought process); the scores for these criteria were summed to create overall scores. Three sources of validity evidence (response process, internal structure, and association with other variables) were examined.ResultsMachine scores correlated more strongly with faculty ratings than faculty ratings did with each other (machine: .28-.53, faculty: .13-.33) and were less case-specific. Machine scores and faculty ratings were similarly correlated with medical knowledge, clinical cognition, and prior diagnostic justification. Machine scores were more strongly associated with clinical communication than were faculty ratings (.43 vs .31).ConclusionsMachine learning technologies may be useful for assessing medical students' long-form written clinical reasoning. Semantically based machine scoring may capture the communicative aspects of clinical reasoning better than faculty ratings, offering the potential for automated assessment that generalizes to the workplace. These results underscore the potential of machine scoring to capture an aspect of clinical reasoning performance that is difficult to assess with traditional analytic scoring methods. Additional research should investigate machine scoring generalizability and examine its acceptability to trainees and educators.

Project description:ProblemWritten feedback is often overly positive, nonspecific, and difficult to interpret. Learner satisfaction with written feedback is low and obtaining written feedback that encourages self-reflection is challenging. Improving feedback quality is laborious and only modestly effective.ApproachThe authors developed the LEAF (Learner-Engaged Analysis of Feedback) method to improve learner satisfaction with, and reflection on, existing written feedback. The method pairs a learner and coach to methodically identify themes in the learner's written feedback. Themes occurring more frequently or less frequently than typical offer areas for reflection, as they may identify learners' relative strengths or weaknesses. The method was introduced at the Massachusetts General Hospital in 2017 during program director (PD) meetings with anesthesiology residents. In 2018, resident satisfaction was measured (1 to 5 Likert-type questions, 1 = "not at all satisfied," 5 = "extremely satisfied") for 4 feedback sources, 2 related to the LEAF method (PD meetings, written feedback) and 2 unrelated (verbal feedback, mentor feedback). Residents' comments were qualitatively assessed to explore the impact on self-reflection.OutcomesResidents who had participated in a LEAF session (n = 54), compared with those who had not (n = 11), reported higher satisfaction with written feedback (mean 3.1 versus 2.5, d = 0.53, P = .03) and PD meeting feedback (mean 3.8 versus 2.8, d = 0.80, P = .03). There were no significant differences between groups for satisfaction with feedback unrelated to the LEAF method. Qualitative analysis of comments suggested that residents found the method useful for providing holistic self-assessment, facilitating goal setting, uncovering blind spots, and improving feedback interpretation.Next stepsNext steps should include studies determining if the association between increased learner satisfaction with written feedback and the LEAF method is causal, and whether this feedback process changes learners' subsequent behaviors.

Dataset Information

Students tell us what good written feedback looks like.

Publications

Students tell us what good written feedback looks like.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets