Dataset Information

Validation of the Fresno test of competence in evidence based medicine.

ABSTRACT:

Objective

To describe the development and validation of a test of knowledge and skills in evidence based medicine.

Design

Cross sectional study.

Setting

Family practice residency programme in California; a list server for those who teach evidence based medicine; and an evidence based medicine seminar series.

Participants

Family practice residents and faculty members (n=43); volunteers self identified as experts in evidence based medicine (n=53); family practice teachers (19) beginning a seminar series on evidence based medicine.

Intervention

The Fresno test is a performance based measure for use in medical education that assesses a wide range of evidence based medicine skills. Open ended questions are scored with standardised grading rubrics. Calculation skills are assessed by fill in the blank questions.

Main outcome measures

Inter-rater reliability, internal reliability, item analyses, and construct validity.

Results

Inter-rater correlations ranged from 0.76 to 0.98 for individual items. Cronbach's alpha was 0.88. Item difficulties ranged from moderate to difficult, all with positive and strong ability to discriminate between candidates. Experts scored consistently higher than novices. On the 212 point test, the novice mean was 95.6 and the expert mean was 147.5 (P<0.001). On individual items, a higher proportion of experts than novices earned passing scores on 15 of the 17 items.

Conclusion

The Fresno test is a reliable and valid test for detecting the effect of instruction in evidence based medicine. Its use in other settings requires further exploration.

SUBMITTER: Ramos KD

PROVIDER: S-EPMC143529 | biostudies-literature | 2003 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of the Fresno test of competence in evidence based medicine.

Ramos Kathleen D KD Schafer Sean S Tracz Susan M SM

BMJ (Clinical research ed.) 20030201 7384

<h4>Objective</h4>To describe the development and validation of a test of knowledge and skills in evidence based medicine.<h4>Design</h4>Cross sectional study.<h4>Setting</h4>Family practice residency programme in California; a list server for those who teach evidence based medicine; and an evidence based medicine seminar series.<h4>Participants</h4>Family practice residents and faculty members (n=43); volunteers self identified as experts in evidence based medicine (n=53); family practice teach ...[more]

PMID: 12574047

Similar Datasets

Project description:BackgroundHealth care educators need valid and reliable tools to assess evidence based practice (EBP) knowledge and skills. Such instruments have yet to be developed for use among physical therapists. The Fresno Test (FT) has been validated only among general practitioners and occupational therapists and does not assess integration of research evidence with patient perspectives and clinical expertise. The purpose of this study was to develop and validate a modified FT to assess EBP knowledge and skills relevant to physical therapist (PT) practice.MethodsThe FT was modified to include PT-specific content and two new questions to assess integration of patient perspectives and clinical expertise with research evidence. An expert panel reviewed the test for content validity. A cross-sectional cohort representing three training levels (EBP-novice students, EBP-trained students, EBP-expert faculty) completed the test. Two blinded raters, not involved in test development, independently scored each test. Construct validity was assessed through analysis of variance for linear trends among known groups. Inter and intra-rater reliability, internal consistency, item discrimination index, item total correlation, and difficulty were analyzed.ResultsAmong 108 participants (31 EBP-novice students, 50 EBP-trained students, and 27 EBP-expert faculty), there was a statistically significant (p < 0.0001) difference in total score corresponding to training level. Total score reliability and psychometric properties of items modified for discipline-specific content were excellent [inter-rater (ICC (2,1)] = 0.91); intra-rater (ICC (2,1)] = 0.95, 0.96)]. Cronbach's alpha was 0.78. Of the two new items, only one had strong psychometric properties.ConclusionsThe 13-item modified FT presented here is a valid, reliable assessment of physical therapists' EBP knowledge and skills. One new item assesses integration of patient perspective as part of the EBP model. Educators and researchers may use the 13-item modified FT to evaluate PT EBP curricula and physical therapists' EBP knowledge and skills.

Project description:BackgroundThe Modified Fresno Test has been used to evaluate the use of the Evidence-Based Physiotherapy (EBP). So far, none of the versions of the Fresno Test were subjected to analysis of the factorial structure. The objective of the study was to describe the exploratory and confirmatory factor structure of the Modified Fresno Test adapted to the Portuguese-Brazilian and analyze the statistical feasibility for the elaboration of a short version.MethodsThe questionnaire was applied with a convenience sample of 57 physiotherapists, being 36 professionals (13 of these also professors) and 21 students from the last semester of the physiotherapy course. Exploratory Factor Analysis (EFA) was performed by the method of principal components. Confirmatory Factor Analysis (CFA) was performed by the method of maximum likelihood. The total score of the answers in the test and retest was evaluated, totalling 228 observations. Reliability was assessed by means of internal consistency, using Cronbach's alpha coefficient.ResultsReliability was satisfactory (α 0.81) for all questions of the instrument. The coefficient α calculated for the corrected item-total showed values higher than 0.20 except for item 9. Preliminary tests for Exploratory Factor Analysis showed acceptable values with Kaiser-Meyer-Olkin (KMO = 0.80) and Bartlett's test of sphericity [chi-square (78) = 1149.615, p < 0.001], indicating that the correlations were sufficient for analysis. The analysis revealed the presence of 3 factors (eigenvalues> 1), which explains 60.9% of the instrument's total variance. In Confirmatory Factor Analysis, none of the indices came close to an acceptable level (≥ 0.90), however, the second model which tested a three-factor structure provided a better fit to the data. From the results of this study the Modified Fresno Test short version was drawn.ConclusionThe analysis showed good factor validity and adequate internal consistency for the use of the instrument consisting of 13 questions and 3 factors. This model proved to be better than the original model. The short version consisting of 9 questions may be an appropriate alternative for use in the population of interest.

Project description:ObjectiveNational and international medical organizations and boards have recognized the importance of Evidence Based Medicine (EBM) and emphasized that EBM training should be included in medical education programs. Although some Turkish medical schools have developed and implemented EBM training programs, no validated Turkish language assessment tool has been available to compare the effectiveness of these training programs to national or international standards. The aim of this study is to cross-culturally adapt the Fresno Test, which is a validated English language tool utilized worldwide in the assessment of EBM training.MethodsThis study is a cross-sectional validation study, which was performed in two stages: Cross-cultural adaptation of the Fresno Test into Turkish; and evaluation of the psychometric properties, validity, reliability and responsiveness, of the Turkish version of the Fresno Test.ResultsThe content validity of the test was evaluated by experienced physicians in the field of Evidence-Based Medicine, and the content validity index was 1.00. The Cronbach α coefficient was 0.78 on the post-test results. The intraclass correlation (ICC) coefficient and the kappa analysis were calculated to evaluate inter-rater reliability. The ICC coefficients ranged from 0.66 to 0.97 for pre- and post-test results. The Kappa coefficients were 1.00 for all pre-test and post-test questions except one post-test question which was 0.89. The change score of the Fresno Test was used to evaluate responsiveness. The students' score of the Turkish Fresno Test was 49.9 ±18.2 pre-training and 118.9 ±26.3 post-training with a change of 69 points (95% CI, 63.9-74.2). The Cohen's effect size was 3.04 (95% CI, 2.6-3.5) indicating a very large change in scores.ConclusionsThe Turkish adapted Fresno Test used to evaluate students' success and program effectiveness is a valid and reliable measurement tool. It will be of great benefit for the comparison of the effectiveness of Turkish education programs nationally and cross-culturally.

Project description:Teachers' social-emotional competence is considered important in order to master the social and emotional challenges inherent in their profession and to build positive teacher-student relationships. In turn, this is key to both teachers' occupational well-being and positive student development. Nonetheless, an instrument assessing the profession-specific knowledge and skills that teachers need to master the social and emotional demands in the classroom is still lacking. Therefore, we developed the Test of Regulation in and Understanding of Social Situations in Teaching (TRUST), which is a theory-based situational judgment test measuring teachers' knowledge about strategies for emotion regulation and relationship management in emotionally and socially challenging situations with students. Results from three studies (N = 166 in-service teachers, N = 73 in-service teachers, N = 107 pre-service teachers) showed satisfactory internal consistency for both the emotion regulation and relationship management subtests. Furthermore, confirmatory factor analyses supported the differentiation between the two facets of social-emotional competence. Regarding convergent validity, results from Study 3 revealed a positive association between the profession-specific TRUST and pre-service teachers' general emotional intelligence. Furthermore, small to moderate correlations with the Big Five personality traits provided evidence for the discriminant validity of TRUST. In Studies 1 and 2, we found evidence for a correlation with external criteria, that is, teachers with higher test scores reported providing more emotional support for students and having better teacher-student relationships. For teachers' occupational well-being, we found a link with symptoms of depersonalization and job satisfaction, but none for emotional exhaustion. We will discuss the use of TRUST in research, for the evaluation of interventions, in teacher education, and professional development and will illustrate ideas for enhancing the tool.

Dataset Information

Validation of the Fresno test of competence in evidence based medicine.

Objective

Design

Setting

Participants

Intervention

Main outcome measures

Results

Conclusion

Publications

Validation of the Fresno test of competence in evidence based medicine.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets