Dataset Information

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

ABSTRACT:

Objective

The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.

Method

Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming class was compared with strategies of residents graduating a year later. MEDLINE search strategies formulated by faculty who had been identified as having search expertise were used as a gold standard comparison. Participants were presented with a clinical scenario and asked to identify the search question and conduct a MEDLINE search. Two librarians rated the blinded search strategies.

Results

Search strategy scores were significantly higher for residents who received training than the comparison group with no training. There was no significant difference in search strategy scores between senior residents who received training and faculty experts.

Conclusion

The results provide evidence for the validity of the instrument to evaluate MEDLINE search strategies. This assessment tool can measure improvements in information-seeking skills and provide data to fulfill Accreditation Council for Graduate Medical Education competencies.

SUBMITTER: Rana GK

PROVIDER: S-EPMC3016654 | biostudies-literature | 2011 Jan

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

Rana Gurpreet K GK Bradley Doreen R DR Hamstra Stanley J SJ Ross Paula T PT Schumacher Robert E RE Frohna John G JG Haftel Hilary M HM Lypson Monica L ML

Journal of the Medical Library Association : JMLA 20110101 1

<h4>Objective</h4>The objective of this study was to validate an assessment instrument for MEDLINE search strategies at an academic medical center.<h4>Method</h4>Two approaches were used to investigate if the search assessment tool could capture performance differences in search strategy construction. First, data from an evaluation of MEDLINE searches from a pediatric resident's longitudinal assessment were investigated. Second, a cross-section of search strategies from residents in one incoming ...[more]

PMID: 21243059

Similar Datasets

Project description:BackgroundThe principle of workplace based assessment (WBA) is to assess trainees at work with feedback integrated into the program simultaneously. A student driven WBA model was introduced and perception evaluation of this teaching method was done subsequently by taking feedback from the faculty as well as the postgraduate trainees (PGs) of a residency program.MethodsDescriptive multimethod study was conducted. A WBA program was designed for PGs in Chemical Pathology on Moodle and forms utilized were case-based discussion (CBD), direct observation of practical skills (DOPS) and evaluation of clinical events (ECE). Consented assessors and PGs were trained on WBA through a workshop. Pretest and posttest to assess PGs knowledge before and after WBA were conducted. Every time a WBA form was filled, perception of PGs and assessors towards WBA, time taken to conduct single WBA and feedback were recorded. Faculty and PGs qualitative feedback on perception of WBA was taken via interviews. WBA tools data and qualitative feedback were used to evaluate the acceptability and feasibility of the new tools.ResultsSix eligible PGs and seventeen assessors participated in this study. A total of 79 CBDs (assessors n = 7 and PGs n = 6), 12 ECEs (assessors n = 6 and PGs n = 5), and 20 DOPS (assessors n = 6 and PGs n = 6) were documented. PGs average pretest score was 55.6%, which was improved to 96.4% in posttest; p value< 0.05. Scores of annual assessment before and after implementation of WBA also showed significant improvement, p value 0.039, Overall mean time taken to evaluate PG's was 12.6 ± 9.9 min and feedback time 9.2 ± 7.4 min. Mean WBA process satisfaction of assessors and PGs on Likert scale of 1 to 10 was 8 ± 1 and 8.3 ± 0.8 respectively.ConclusionBoth assessors and fellows were satisfied with introduction and implementation of WBA. It gave the fellows opportunity to interact with assessors more often and learn from their rich experience. Gain in knowledge of PGs was identified from the statistically significant improvement in PGs' assessment scores after WBA implementation.

Project description:BACKGROUND: Few studies have systematically and rigorously examined the quality of care provided in educational practice sites. OBJECTIVE: The objectives of this study were to (1) describe the patient population cared for by trainees in internal medicine residency clinics; (2) assess the quality of preventive cardiology care provided to these patients; (3) characterize the practice-based systems that currently exist in internal medicine residency clinics; and (4) examine the relationships between quality, practice-based systems, and features of the program: size, type of program, and presence of an electronic medical record. DESIGN: This is a cross-sectional observational study. SETTING: This study was conducted in 15 Internal Medicine residency programs (23 sites) throughout the USA. PARTICIPANTS: The participants included site champions at residency programs and 709 residents. MEASUREMENTS: Abstracted charts provided data about patient demographics, coronary heart disease risk factors, processes of care, and clinical outcomes. Patients completed surveys regarding satisfaction. Site teams completed a practice systems survey. RESULTS: Chart abstraction of 4,783 patients showed substantial variability across sites. On average, patients had between 3 and 4 of the 9 potential risk factors for coronary heart disease, and approximately 21% had at least 1 important barrier of care. Patients received an average of 57% (range, 30-77%) of the appropriate interventions. Reported satisfaction with care was high. Sites with an electronic medical record showed better overall information management (81% vs 27%) and better modes of communication (79% vs 43%). CONCLUSIONS: This study has provided insight into the current state of practice in residency sites including aspects of the practice environment and quality of preventive cardiology care delivered. Substantial heterogeneity among the training sites exists. Continuous measurement of the quality of care provided and a better understanding of the training environment in which this care is delivered are important goals for delivering high quality patient care.

Dataset Information

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

Objective

Method

Results

Conclusion

Publications

A validated search assessment tool: assessing practice-based learning and improvement in a residency program.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure