Dataset Information

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students.

ABSTRACT: BACKGROUND: Clinical reasoning (CR) is a core capability for health practitioners. Assessing CR requires a suite of tools to encompass a wide scope of contexts and cognitive abilities. The aim of this project was to develop an oral examination and grading rubric for the assessment of CR in osteopathy, trial it with senior students in three accredited university programs in Australia and New Zealand, and to evaluate its content and face validity. METHODS: Experienced osteopathic academics developed 20 cases and a grading rubric. Thirty senior students were recruited, 10 from each university. Twelve fourth year and 18 fifth year students participated. Three members of the research team were trained and examined students at an institution different from their own. Two cases were presented to each student participant in a series of vignettes. The rubric was constructed to follow a set of examiner questions that related to each attribute of CR. Data were analysed to explore differences in examiner marking, as well as relationships between cases, institutions, and different year levels. A non-examining member of the research team acted as an observer at each location. RESULTS: No statistical difference was found between the total and single question scores, nor for the total scores between examiners. Significant differences were found between 4th and 5th students on total score and a number of single questions. The rubric was found to be internally consistent. CONCLUSIONS: A viva examination of clinical reasoning, trialled with senior osteopathy students, showed face and content validity. Results suggested that the viva exam may also differentiate between 4th and 5th year students' capabilities in CR. Further work is required to establish the reliability of assessment, to further refine the rubric, and to train examiners before it is implemented as a high-stakes assessment in accredited osteopathy programs.

SUBMITTER: Orrock P

PROVIDER: S-EPMC4179819 | biostudies-other | 2014

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students.

Orrock Paul P Grace Sandra S Vaughan Brett B Coutts Rosanne R

BMC medical education 20140919

<h4>Background</h4>Clinical reasoning (CR) is a core capability for health practitioners. Assessing CR requires a suite of tools to encompass a wide scope of contexts and cognitive abilities. The aim of this project was to develop an oral examination and grading rubric for the assessment of CR in osteopathy, trial it with senior students in three accredited university programs in Australia and New Zealand, and to evaluate its content and face validity.<h4>Methods</h4>Experienced osteopathic acad ...[more]

PMID: 25238784

Similar Datasets

Project description:PurposeGraduating medical students need broad clinical diagnostic reasoning skills that integrate learning across clinical specialties to deal with undifferentiated patient problems. The opportunity to acquire these skills may be limited during clinical placements on increasingly specialized hospital wards. We developed an intervention of regular general practitioner (GP) facilitated teaching in hospital placements to enable students to develop broad clinical diagnostic reasoning. The intervention was piloted, refined and delivered to a whole cohort of medical students at the start of their third year. This paper examines whether students perceived opportunities to improve their broad diagnostic clinical reasoning through our intervention.MethodsGP-facilitated teaching sessions were delivered weekly in hospital placements to small groups of 6–8 students for 90 mins over 6 weeks. Students practiced clinical reasoning with real patient cases that they encountered on their placements. Evaluation of learning outcomes was conducted through a student questionnaire using Likert scales with free-text boxes for additional explanation. Focus groups were conducted to gain a more in-depth understanding of student perspectives.ResultsAs high as 87% of students agreed that their broad clinical diagnostic reasoning ability had improved. Thematic analysis of the qualitative data revealed four factors supporting this improvement: practicing the hypothetico-deductive method, using real patient cases, composing student groups from different speciality placements and the breadth of the facilitators’ knowledge. Students additionally reported enhanced person-centredness in terms of understanding the patient’s perspective and journey. Students perceived that the added value of general practitioner facilitators lay in their broad knowledge base and knowledge of patient needs in the community.ConclusionOur results suggest that medical students can develop broad clinical diagnostic reasoning skills in hospital settings through regular GP-facilitated teaching. Our approach has the advantage of working within the established curricular format of hospital placements and being deliverable at scale to whole student cohorts.

Project description:BackgroundClinical reasoning plays a major role in the ability of doctors to make a diagnosis and reach treatment decisions. This paper describes the use of four clinical reasoning tests in the second National Medical Science Olympiad in Iran: key features (KF), script concordance (SCT), clinical reasoning problems (CRP) and comprehensive integrative puzzles (CIP). The purpose of the study was to design a multi instrument for multiple roles approach in clinical reasoning field based on the theoretical framework, KF was used to measure data gathering, CRP was used to measure hypothesis formation, SCT and CIP were used to measure hypothesis evaluation and investigating the combined use of these tests in the Olympiad. A bank of clinical reasoning test items was developed for emergency medicine by a scientific expert committee representing all the medical schools in the country. These items were pretested by a reference group and the results were analyzed to select items that could be omitted. Then 135 top-ranked medical students from 45 medical universities in Iran participated in the clinical domain of the Olympiad. The reliability of each test was calculated by Cronbach's alpha. Item difficulty and the correlation between each item and the total score were measured. The correlation between the students' final grade and each of the clinical reasoning tests was calculated, as was the correlation between final grades and another measure of knowledge, i.e., the students' grade point average.ResultsThe combined reliability for all four clinical reasoning tests was 0.91. Of the four clinical reasoning tests we compared, reliability was highest for CIP (0.91). The reliability was 0.83 for KF, 0.78 for SCT and 0.71 for CRP. Most of the tests had an acceptable item difficulty level between 0.2 and 0.8. The correlation between the score for each item and the total test score for each of the four tests was positive. The correlations between scores for each test and total score were highest for KF and CIP. The correlation between scores for each test and grade point average was low to intermediate for all four of the tests.ConclusionThe combination of these four clinical reasoning tests is a reliable evaluation tool that can be implemented to assess clinical reasoning skills in talented undergraduate medical students, however these data may not generalizable to whole medical students population. The CIP and KF tests showed the greatest potential to measure clinical reasoning skills. Grade point averages did not necessarily predict performance in the clinical domain of the national competitive examination for medical school students.

Project description:BACKGROUND:The clinical reasoning process, which requires biomedical knowledge, knowledge about problem-solving strategies, and knowledge about reasons for diagnostic procedures, is a key element of physicians' daily practice but difficult to assess. The aim of this study was to empirically develop a Clinical Reasoning Indicators-History Taking-Scale (CRI-HT-S) and to assess the clinical reasoning ability of advanced medical students during a simulation involving history taking. METHODS:The Clinical Reasoning Indictors-History Taking-Scale (CRI-HT-S) including a 5-point Likert scale for assessment was designed from clinical reasoning indicators identified in a qualitative study in 2017. To assess indicators of clinical reasoning ability, 65 advanced medical students (semester 10, n =?25 versus final year, n =?40) from three medical schools participated in a 360-degree competence assessment in the role of beginning residents during a simulated first workday in hospital. This assessment included a consultation hour with five simulated patients which was videotaped. Videos of 325 patient consultations were assessed using the CRI-HT-S. A factor analysis was conducted and the students' results were compared according to their advancement in undergraduate medical training. RESULTS:The clinical reasoning indicators of the CRI-HT-S loaded on three factors relevant for clinical reasoning: 1) focusing questions, 2) creating context, and 3) securing information. Students reached significantly different scores (p <?.001) for the three factors (factor 1: 4.07?±?.47, factor 2: 3.72?±?.43, factor 3: 2.79?±?.83). Students in semester 10 reached significantly lower scores for factor 3 than students in their final year (p <?.05). CONCLUSIONS:The newly developed CRI-HT-S worked well for quantitative assessment of clinical reasoning indicators during history taking. Its three-factored structure helped to explore different aspects of clinical reasoning. Whether the CRI-HT-S has the potential to be used as a scale in objective structured clinical examinations (OCSEs) or in workplace-based assessments of clinical reasoning has to be investigated in further studies with larger student cohorts.

Project description:PurposeDeveloping medical students' clinical reasoning requires a structured longitudinal curriculum with frequent targeted assessment and feedback. Performance-based assessments, which have the strongest validity evidence, are currently not feasible for this purpose because they are time-intensive to score. This study explored the potential of using machine learning technologies to score one such assessment-the diagnostic justification essay.MethodFrom May to September 2018, machine scoring algorithms were trained to score a sample of 700 diagnostic justification essays written by 414 third-year medical students from the Southern Illinois University School of Medicine classes of 2012-2017. The algorithms applied semantically based natural language processing metrics (e.g., coherence, readability) to assess essay quality on 4 criteria (differential diagnosis, recognition and use of findings, workup, and thought process); the scores for these criteria were summed to create overall scores. Three sources of validity evidence (response process, internal structure, and association with other variables) were examined.ResultsMachine scores correlated more strongly with faculty ratings than faculty ratings did with each other (machine: .28-.53, faculty: .13-.33) and were less case-specific. Machine scores and faculty ratings were similarly correlated with medical knowledge, clinical cognition, and prior diagnostic justification. Machine scores were more strongly associated with clinical communication than were faculty ratings (.43 vs .31).ConclusionsMachine learning technologies may be useful for assessing medical students' long-form written clinical reasoning. Semantically based machine scoring may capture the communicative aspects of clinical reasoning better than faculty ratings, offering the potential for automated assessment that generalizes to the workplace. These results underscore the potential of machine scoring to capture an aspect of clinical reasoning performance that is difficult to assess with traditional analytic scoring methods. Additional research should investigate machine scoring generalizability and examine its acceptability to trainees and educators.

Project description:BackgroundIn Australia, the profession of pharmacy has undergone many changes to adapt to the needs of the community. In recent years, concerns have been raised with evidence emerging of workforce saturation in traditional pharmacy practice sectors. It is not known how current final year pharmacy students' perceive the different pharmacy career paths in this changing environment. Hence investigating students' current experiences with their pharmacy course, interaction with the profession and developing an understanding of their career intentions would be an important step, as these students would make up a large proportion of future pharmacy workforce.ObjectiveThe objective of this study was thus to investigate final year students' career perspectives and the reasons for choosing pharmacy, satisfaction with this choice of pharmacy as a tertiary course and a possible future career, factors affecting satisfaction and intention of future career paths.MethodsA quantitative cross sectional survey of final year students from 3 Australian universities followed by a qualitative semi-structured interview of a convenience sample of final year students from the University of Sydney.Results'Interest in health and medicine' was the most important reason for choosing pharmacy (n=238). The majority of students were 'somewhat satisfied' with the choice of pharmacy (35.7%) as a course and possible future career. Positive associations were found between satisfaction and reasons for joining pharmacy such as 'felt pharmacy is a good profession' (p=0.003) while negative associations included 'joined pharmacy as a gateway to medicine or dentistry' (p=0.001). Quantitate and qualitative results showed the most frequent perception of community pharmacy was 'changing' while hospital and pharmaceutical industry was described as 'competitive' and 'research' respectively. The highest career intention was community followed by hospital pharmacy.ConclusionComplex factors including university experiences are involved in shaping students' satisfaction and perception of career. This may relate to challenges in the community pharmacy sector, job opportunities in hospital and limited understanding of the pharmaceutical industry. The results offer insight for the profession in terms of entry into various roles and also to pharmacy educators for their roles in shaping curricula and placement experiences that attract future graduates to defined career pathways in pharmacy.

Dataset Information

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students.

Publications

Developing a viva exam to assess clinical reasoning in pre-registration osteopathy students.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets