Dataset Information

Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals.

ABSTRACT:

Background

Health educators need rigorously developed instruments to evaluate cognitive skills relating to evidence based practice (EBP). Previous EBP evaluation instruments have focused on the acquisition and appraisal of the evidence and are largely based in the medical profession. The aim of this study was to develop and validate an EBP evaluation instrument to assess EBP cognitive skills for entry-level health professional disciplines.

Methods

The Fresno test of competence in evidence based medicine was considered in the development of the 'Knowledge of Research Evidence Competencies' instrument (K-REC). The K-REC was reviewed for content validity. Two cohorts of entry-level students were recruited for the pilot study, those who had been exposed to EBP training (physiotherapy students, n = 24), and who had not been exposed to EBP training (human movement students, n = 76). The K-REC was administered to one cohort of students (n = 24) on two testing occasions to evaluate test-retest reliability. Two raters independently scored the first test occasion (n = 24) to evaluate the inter-rater reliability of the marking guidelines. Construct validity was assessed by comparison of the two groups, 'exposed' and 'non-exposed', and the percentage of students achieving a 'pass' score in each of these groups. Item difficulty was established.

Results

Among the 100 participants (24 EBP 'exposed', and 76 EBP 'non-exposed' students), there was a statistically significant (p < 0.0001) difference in the total K-REC scores. The test-retest and inter-rater reliability of the individual items and total scores ranged from moderate to excellent (measured by Cohen's Kappa and ICC, range: 0.62 to perfect agreement).

Conclusions

The K-REC instrument is a valid and reliable evaluation instrument of cognitive skills of EBP in entry-level student health professionals. The instrument is quick to disseminate and easy to score, making it a suitable instrument for health educators to employ to evaluate students' knowledge of EBP or in the evaluation of entry-level EBP training.

SUBMITTER: Lewis LK

PROVIDER: S-EPMC3196731 | biostudies-literature | 2011 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals.

Lewis Lucy K LK Williams Marie T MT Olds Timothy S TS

BMC medical education 20111003

<h4>Background</h4>Health educators need rigorously developed instruments to evaluate cognitive skills relating to evidence based practice (EBP). Previous EBP evaluation instruments have focused on the acquisition and appraisal of the evidence and are largely based in the medical profession. The aim of this study was to develop and validate an EBP evaluation instrument to assess EBP cognitive skills for entry-level health professional disciplines.<h4>Methods</h4>The Fresno test of competence in ...[more]

PMID: 21967728

Similar Datasets

Project description:BackgroundValid and reliable instruments are required to measure the effect of educational interventions to improve evidence-based practice (EBP) knowledge and skills in occupational therapy. The aims of this paper are to: 1) describe amendments to the Adapted Fresno Test of Competence in EBP (AFT), and 2) report the psychometric properties of the modified instrument when used with South African occupational therapists.MethodsThe clinical utility of the AFT was evaluated for use with South African occupational therapists and modifications made. The modified AFT was used in two studies to assess its reliability and validity. In Study 1 a convenience sample of 26 occupational therapists in private practice or government-funded health facilities in a South African province were recruited to complete the modified AFT on two occasions 1 week apart. Completed questionnaires were scored independently by two raters. Inter-rater, test-retest reliability and internal consistency were determined. Study 2 was a pragmatic randomised controlled trial involving occupational therapists in four Western Cape Department of Health district municipalities (n = 58). Therapists were randomised in matched pairs to one of two educational interventions (interactive or didactic), and completed the modified AFT at baseline and 12 weeks after the intervention. An intention-to-treat analysis was performed. Data were not normally distributed, thus non-parametric statistics were used.ResultsIn Study 1, 21 of 26 participants completed the questionnaire twice. Test-retest (ICC = 0.95, 95 % CI = 0.88-0.98) and inter-rater reliability (Time 1: ICC = 0.995, 95 % CI = 0.99-0.998; Time 2: ICC = 0.99, 95 % CI = 0.97-0.995) were excellent for total scores. Internal consistency based on time 1 scores was satisfactory (α = 0.70). In Study 2, 28 participants received an interactive educational intervention and completed the modified AFT at baseline and 12 weeks later. Median total SAFT scores increased significantly from baseline to 12-weeks (Z = -4.078, p < 0.001) with a moderate effect size (r = 0.55).ConclusionThe modified AFT has demonstrated validity for detecting differences in EBP knowledge between two groups. It also has excellent test-retest and inter-rater reliability. The instrument is recommended for contexts where EBP is an emerging approach and time is at a premium.Trial registrationPan African Controlled Trials Register PACTR201201000346141 . Registered 31 January 2012. Clinical Trials NCT01512823 . Registered 1 February 2012. South African National Clinical Trial Register DOH2710093067 . Registered 27 October 2009.

Project description:BackgroundIn Brazil, as in most countries nowadays, there is a pursuit for healthcare quality improvement and sustainability in public and private systems. Healthcare professionals' perceptions, knowledge and attitudes determine evidence-based practice (EBP), which remain uncertain among Brazilian practitioners. A standardised national instrument whose wide use will identify gaps and flaws in establishing an EBP could contribute to an effective resources allocation from health professionals willing to use an EBP.ObjectivesTo present a study protocol on the development and validation of an instrument to measure Brazilian healthcare professionals' behaviour, skills, self-efficacy, knowledge and attitudes towards EBP.MethodsThis is a validation study with Brazilian healthcare professionals to develop a valid and reliable questionnaire, including selection of domains and formulation of questions. Construct and content validity will be assess by a panel of experts, with data collection and analysis following a Delphi-like methodology. Further, a pilot survey will be accomplished with a representative sample of different healthcare professionals from all main Brazilian regions. An exploratory factor analysis and a confirmatory factor analysis will be conducted afterwards. The ratio of χ2 and df (χ2/df), comparative fit index, goodness of fit index and root mean square error of approximation will be used for assessing the model fit. In addition, the reliability of the instrument will be estimated by test-retest reproducibility and Cronbach's alpha coefficient (α).Ethics and disseminationThis study has received ethical approval from the Pharmaceutical Sciences Faculty of the São Paulo State University (1.425.808). The use among a wide national sample is expected to promote an extensive view of evidence-based decision-making, identifying the knowledge gaps in this area. Study findings will be circulated to healthcare professionals and scientists in the field through the publication in peer-reviewed journals and conference presentations.

Project description:BackgroundCultural competence of healthcare professionals (HCPs) is recognized as a strategy to reduce cultural disparities in healthcare. However, standardised, valid and reliable instruments to assess HCPs' cultural competence are notably lacking. The present study aims to 1) identify the core components of cultural competence from a healthcare perspective, 2) to develop a self-report instrument to assess cultural competence of HCPs and 3) to evaluate the psychometric properties of the new instrument.MethodsThe conceptual model and initial item pool, which were applied to the cross-cultural competence instrument for the healthcare profession (CCCHP), were derived from an expert survey (n = 23), interviews with HCPs (n = 12), and a broad narrative review on assessment instruments and conceptual models of cultural competence. The item pool was reduced systematically, which resulted in a 59-item instrument. A sample of 336 psychologists, in advanced psychotherapeutic training, and 409 medical students participated, in order to evaluate the construct validity and reliability of the CCCHP.ResultsConstruct validity was supported by principal component analysis, which led to a 32-item six-component solution with 50% of the total variance explained. The different dimensions of HCPs' cultural competence are: Cross-Cultural Motivation/Curiosity, Cross-Cultural Attitudes, Cross-Cultural Skills, Cross-Cultural Knowledge/Awareness and Cross-Cultural Emotions/Empathy. For the total instrument, the internal consistency reliability was .87 and the dimension's Cronbach's α ranged from .54 to .84. The discriminating power of the CCCHP was indicated by statistically significant mean differences in CCCHP subscale scores between predefined groups.ConclusionsThe 32-item CCCHP exhibits acceptable psychometric properties, particularly content and construct validity to examine HCPs' cultural competence. The CCCHP with its five dimensions offers a comprehensive assessment of HCPs' cultural competence, and has the ability to distinguish between groups that are expected to differ in cultural competence. This instrument can foster professional development through systematic self-assessment and thus contributes to improve the quality of patient care.

Project description:Background: Communicating interprofessionally using the telephone is an essential skill within pharmacy practice. Student pharmacists' ability to perform this task effectively and efficiently may be hindered by generational changes, social anxiety, and very few opportunities to practice these skills. Objective: The purpose of this study was to develop and implement a simulation allowing students to practice interprofessional communication and assess the simulation's impact on students' confidence in providing pharmacy-related interventions to another health care professional via telephone. Methods: Faculty developed a simulation focused on interprofessional telephone communication. Baseline student information was collected to quantify pharmacy work experience in terms of practice setting, duration of employment, and skills. Presimulation and postsimulation surveys evaluated self-assessed telephone-related skills, attitudes, and confidence. Quantitative data were analyzed with descriptive statistics. Qualitative data were evaluated through a thematic analysis of students' reflective responses to 2 open-ended questions. Results: Of the 53 pharmacy students that participated in the simulation, 44 (83%) and 43 (81%) completed the anonymous presimulation and postsimulation surveys. Students significantly improved as reflected in the following response: "I have confidence in my ability to provide pharmacy-related interventions to another health care professional in a logical and concise manner via telephone call." Significant improvement also occurred in the ability to work independently, communicate an order change to another health care professional, justify recommendations, answer a drug information question, and discuss recommendations in a logical and concise manner. Conclusion: The simulation discussed in this article provided students an opportunity to practice interprofessional telephone communication in a low-risk environment and resulted in significant growth in confidence and skills.

Dataset Information

Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals.

Background

Methods

Results

Conclusions

Publications

Development and psychometric testing of an instrument to evaluate cognitive skills of evidence based practice in student health professionals.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets