Dataset Information

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands.

ABSTRACT: PURPOSE:It is assumed that case-based questions require higher order cognitive processing, whereas questions that are not case-based require lower order cognitive processing. In this study, we investigated to what extent case-based questions and questions that are not case-based, relate to Bloom's taxonomy. METHODS:In this article, 4800 questions of the Progress Test were classified whether it was a case-based question and the level of Bloom's taxonomy. Lower-order questions require students to remember or/and basically understand the knowledge. Higher-order questions require students to apply, analyze, or/and evaluate. A phi-coefficient was calculated to investigate the relations between the presence of case-based questions and the required level of cognitive processing. RESULTS:Our results demonstrated that case-based questions were measuring higher levels of cognitive processing in 98.1% of the questions. Of the non-case-based questions, 33.7% required a higher level of cognitive processing. The phi-coefficient demonstrated a significant moderate correlation between the presence of a patient case in a question and its required level of cognitive processing (phi-coefficient = 0.55, p<0.001). CONCLUSION:Medical teachers should be aware of the association between item formats (case-based versus non-case-based) and the cognitive processes they elicit in order to meet a certain balance in a test, taking the learning objectives as well as the test difficulty into account.

SUBMITTER: Cecilio-Fernandes D

PROVIDER: S-EPMC6340837 | biostudies-literature | 2018

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands.

Cecilio-Fernandes Dario D Kerdijk Wouter W Bremers Andreas Johannes AJ Aalders Wytze W Tio René Anton RA

Journal of educational evaluation for health professions 20181212

<h4>Purpose</h4>It is assumed that case-based questions require higher order cognitive processing, whereas questions that are not case-based require lower order cognitive processing. In this study, we investigated to what extent case-based questions and questions that are not case-based, relate to Bloom's taxonomy.<h4>Methods</h4>In this article, 4800 questions of the Progress Test were classified whether it was a case-based question and the level of Bloom's taxonomy. Lower-order questions requi ...[more]

PMID: 30541188

Similar Datasets

Project description:PurposeThe purpose of this study was to improve the quality of items on the Korean Nursing Licensing Examination by developing and evaluating case-based items that reflect integrated nursing knowledge.MethodsWe conducted a cross-sectional observational study to develop new case-based items. The methods for developing test items included expert workshops, brainstorming, and verification of content validity. After a mock examination of undergraduate nursing students using the newly developed case-based items, we evaluated the appropriateness of the items through classical test theory and item response theory.ResultsA total of 50 case-based items were developed for the mock examination, and content validity was evaluated. The question items integrated 34 discrete elements of integrated nursing knowledge. The mock examination was taken by 741 baccalaureate students in their fourth year of study at 13 universities. Their average score on the mock examination was 57.4, and the examination showed a reliability of 0.40. According to classical test theory, the average level of item difficulty of the items was 57.4% (80%-100% for 12 items; 60%-80% for 13 items; and less than 60% for 25 items). The mean discrimination index was 0.19, and was above 0.30 for 11 items and 0.20 to 0.29 for 15 items. According to item response theory, the item discrimination parameter (in the logistic model) was none for 10 items (0.00), very low for 20 items (0.01 to 0.34), low for 12 items (0.35 to 0.64), moderate for 6 items (0.65 to 1.34), high for 1 item (1.35 to 1.69), and very high for 1 item (above 1.70). The item difficulty was very easy for 24 items (below -2.0), easy for 8 items (-2.0 to -0.5), medium for 6 items (-0.5 to 0.5), hard for 3 items (0.5 to 2.0), and very hard for 9 items (2.0 or above). The goodness-of-fit test in terms of the 2-parameter item response model between the range of 2.0 to 0.5 revealed that 12 items had an ideal correct answer rate.ConclusionWe surmised that the low reliability of the mock examination was influenced by the timing of the test for the examinees and the inappropriate difficulty of the items. Our study suggested a methodology for the development of future case-based items for the Korean Nursing Licensing Examination.

Project description:Background: General practitioners and general internists occupy a key position in German and Austrian healthcare systems. They provide primary care and act as gatekeepers between medical disciplines and sectors of care. Their explicit medical knowledge levels, however, can be quite disparate. Objective: This study analyses whether general practitioners' performances on a standardised knowledge test changes with four relevant socio-demographic variables. Design: The survey was based on the Progress Test Medicine (PTM), a standardised 200 item knowledge test on graduate level. After formal blueprinting and item analysis, 60 items of PTM were selected ("PTM-GP") for our study. Participants: PTM-GP was presented ad hoc to general practitioners and internists from Germany and Austria at a number of professional meetings in 2011. 161 volunteers completed the survey. Main measures: For evaluation, correlation analysis (Spearman), Kruskal Wallis-tests for non-parametric data and an analysis of covariance (ANCOVA) were calculated. Results: Overall, four indicators turned out to be slightly significant for the performance on the PTM-GP, namely: time passed since graduation, the grade received in the licensing examination, the type of institution for postgraduate training, and the medical specialisation. Conclusions: Recent graduates performed better in the PTM-GP; a doctor's licensing examination grade as well as training at a university hospital correlated positively with PTM-GP performance. A general doctor's knowledge level is moderately influenced by exam grades, time since graduation, the institutional affiliation of postgraduate training and medical specialisation. Individual changes in knowledge over time have to be deliberately considered in lifelong learning. In consequence, the on-going teaching of medical knowledge should be integrated mandatory and verifiable into general doctors' everyday practices, e.g. through repetitive knowledge tests with individual feedback and recommendations for further continuing medical education (CME).

Dataset Information

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands.

Publications

Comparison of level of cognitive process between case-based items and non-case-based items of the interuniversity progress test of medicine in the Netherlands.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets