Dataset Information

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool.

ABSTRACT: The importance of effective clinical teaching skills is well established in the literature. However, reliable tools with validity evidence that are able to measure the development of these skills and can effectively be used by nonphysician raters do not exist.Our initiative had 2 aims: (1) to develop a teaching development assessment tool (TDAT) that allows skill assessment along a continuum, and (2) to determine if trained nonphysicians can assess clinical teachers with this tool.We describe the development of the TDAT, including identification of 6 global teaching domains and observable teaching behaviors along a 3-level continuum (novice/beginner, competent/proficient, expert) and an iterative revision process involving local and national content experts. The TDAT was studied with attending physicians during inpatient rounds with trained physician and nonphysician observers over 6 months.The TDAT showed emerging evidence of content, construct, and viable validity (the degree to which an assessment tool is practical, affordable, suitable, evaluable, and helpful in the real world) for the evaluation of attending physicians on inpatient rounds. Moderate to near perfect interrater reliability was seen between physician and nonphysician raters for the domains of promotion of clinical reasoning, control of the learning environment, ability to teach to multiple levels of learners, and provision of feedback.The TDAT holds potential as a valid and reliable assessment tool for clinical teachers to track the development of each individual's teaching skills along the continuum from early development to mastery.

SUBMITTER: O'Toole JK

PROVIDER: S-EPMC4676415 | biostudies-literature | 2015 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool.

O'Toole Jennifer K JK Klein Melissa D MD McLinden Daniel D Sucharew Heidi H DeWitt Thomas G TG

Journal of graduate medical education 20151201 4

<h4>Background</h4>The importance of effective clinical teaching skills is well established in the literature. However, reliable tools with validity evidence that are able to measure the development of these skills and can effectively be used by nonphysician raters do not exist.<h4>Objective</h4>Our initiative had 2 aims: (1) to develop a teaching development assessment tool (TDAT) that allows skill assessment along a continuum, and (2) to determine if trained nonphysicians can assess clinical t ...[more]

PMID: 26692978

Similar Datasets

Project description:BackgroundImplementation science comprises a large set of theories suggesting interacting factors at different organisational levels. Development of literature syntheses and frameworks for implementation have contributed to comprehensive descriptions of implementation. However, corresponding instruments for measuring these comprehensive descriptions are currently lacking. The present study aimed to develop an instrument measuring care providers' perceptions of an implementation effort, and to explore the instrument's psychometric properties.MethodsBased on existing implementation literature, a questionnaire was designed with items on individual and team factors and on stages of change in an implementation process. The instrument was tested in a Norwegian study on implementation of evidence based practices for psychosis. Item analysis, factor structure, and internal consistency at baseline were examined.ResultsThe 27-item Implementation Process Assessment Tool (IPAT) revealed large variation between mean score of the items. The total scale scores were widely dispersed across respondents. Internal consistency for the total scale was high (Cronbach's alpha: .962), and all but one item contributed positively to the construct. The results indicated four underlying constructs: individual stages for behavioural change, individual activities and perceived support, collective readiness and support, and individual perceptions of the intervention.ConclusionsThe IPAT appears to be a feasible instrument for investigating the implementation process from the perspective of those making the change. It can enable examination of the relative importance of factors thought to be essential for implementation outcomes. It may also provide ongoing feedback for leaders tailoring support for teams to improve implementation. However, further research is needed to detect the instrument's properties later in the implementation process and in different contexts.Trial registrationClinicalTrials.gov code NCT03271242 (retrospective registered September 5, 2017).

Project description:The "replication crisis" is a methodological problem in which many scientific research findings have been difficult or impossible to replicate. Because the reproducibility of empirical results is an essential aspect of the scientific method, such failures endanger the credibility of theories based on them and possibly significant portions of scientific knowledge. An instance of the replication crisis, analytic replication, pertains to reproducing published results through computational reanalysis of the authors' original data. However, direct replications are costly, time-consuming, and unrewarded in today's publishing standards. We propose that bioinformatics and computational biology students replicate recent discoveries as part of their curriculum. Considering the above, we performed a pilot study in one of the graduate-level courses we developed and taught at our University. The course is entitled Intro to R Programming and is meant for students in our Master's and PhD programs who have little to no programming skills. As the course emphasized real-world data analysis, we thought it would be an appropriate setting to carry out this study. The primary objective was to expose the students to real biological data analysis problems. These include locating and downloading the needed datasets, understanding any underlying conventions and annotations, understanding the analytical methods, and regenerating multiple graphs from their assigned article. The secondary goal was to determine whether the assigned articles contained sufficient information for a graduate-level student to replicate its figures. Overall, the students successfully reproduced 39% of the figures. The main obstacles were the need for more advanced programming skills and the incomplete documentation of the applied methods. Students were engaged, enthusiastic, and focused throughout the semester. We believe that this teaching approach will allow students to make fundamental scientific contributions under appropriate supervision. It will teach them about the scientific process, the importance of reporting standards, and the importance of openness.

Project description:BackgroundAlthough 80% of children with disabilities live in developing countries, there are few culturally appropriate developmental assessment tools available for these settings. Often tools from the West provide misleading findings in different cultural settings, where some items are unfamiliar and reference values are different from those of Western populations.Methods and findingsFollowing preliminary and qualitative studies, we produced a draft developmental assessment tool with 162 items in four domains of development. After face and content validity testing and piloting, we expanded the draft tool to 185 items. We then assessed 1,426 normal rural children aged 0-6 y from rural Malawi and derived age-standardized norms for all items. We examined performance of items using logistic regression and reliability using kappa statistics. We then considered all items at a consensus meeting and removed those performing badly and those that were unnecessary or difficult to administer, leaving 136 items in the final Malawi Developmental Assessment Tool (MDAT). We validated the tool by comparing age-matched normal children with those with malnutrition (120) and neurodisabilities (80). Reliability was good for items remaining with 94%-100% of items scoring kappas >0.4 for interobserver immediate, delayed, and intra-observer testing. We demonstrated significant differences in overall mean scores (and individual domain scores) for children with neurodisabilities (35 versus 99 [p<0.001]) when compared to normal children. Using a pass/fail technique similar to the Denver II, 3% of children with neurodisabilities passed in comparison to 82% of normal children, demonstrating good sensitivity (97%) and specificity (82%). Overall mean scores of children with malnutrition (weight for height <80%) were also significantly different from scores of normal controls (62.5 versus 77.4 [p<0.001]); scores in the separate domains, excluding social development, also differed between malnourished children and controls. In terms of pass/fail, 28% of malnourished children versus 94% of controls passed the test overall.ConclusionsA culturally relevant developmental assessment tool, the MDAT, has been created for use in African settings and shows good reliability, validity, and sensitivity for identification of children with neurodisabilities.

Dataset Information

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool.

Publications

A Pilot Study of the Creation and Implementation of a Teaching Development Assessment Tool.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets