Dataset Information

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career.

ABSTRACT: Anonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures of professors' teaching effectiveness. However, previous research has generally used inappropriate parametric statistics and effect sizes to examine and to evaluate the significance of TEIFs on personnel decisions. Accordingly, we examined the influence of quantitative vs. non-quantitative courses on SET ratings and SET based personnel decisions using 14,872 publicly posted class evaluations where each evaluation represents a summary of SET ratings provided by individual students responding in each class. In total, 325,538 individual student evaluations from a US mid-size university contributed to theses class evaluations. The results demonstrate that class subject (math vs. English) is strongly associated with SET ratings, has a substantial impact on professors being labeled satisfactory vs. unsatisfactory and excellent vs. non-excellent, and the impact varies substantially depending on the criteria used to classify professors as satisfactory vs. unsatisfactory. Professors teaching quantitative courses are far more likely not to receive tenure, promotion, and/or merit pay when their performance is evaluated against common standards.

SUBMITTER: Uttl B

PROVIDER: S-EPMC5426349 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career.

Uttl Bob B Smibert Dylan D

PeerJ 20170509

Anonymous student evaluations of teaching (SETs) are used by colleges and universities to measure teaching effectiveness and to make decisions about faculty hiring, firing, re-appointment, promotion, tenure, and merit pay. Although numerous studies have found that SETs correlate with various teaching effectiveness irrelevant factors (TEIFs) such as subject, class size, and grading standards, it has been argued that such correlations are small and do not undermine the validity of SETs as measures ...[more]

PMID: 28503380

Similar Datasets

Project description:Student evaluations of teaching (SETs) have become a widely used tool for assessing teaching in higher education. However, numerous investigations have shown that SETs are subject to multiple biases, one of which is particularly relevant, namely, the area of knowledge to which the subject belongs. This article aims to replicate the article by Uttl & Smibert (2017, https://doi.org/10.7717/peerj.3299) in a different educational context to verify whether the negative bias toward instructors who teach quantitative courses found by the authors in the US also appears in the Spanish university system. The study was conducted at the Business and Law School of the Universidad Pontificia Comillas, a private Spanish university, using two different samples. First, we analyzed undergraduate courses using a sample of 80,667 SETs in which 2,885 classes (defined as a single semester-long course taught by an individual instructor to a specific group of students), 488 instructors, and 322 different courses were evaluated over a time period of four academic years (2016/2017-2019/2020). Second, in the same period, 16,083 SETs corresponding to master's degree courses were analyzed, which involved the study of 871 classes, 275 instructors, and 155 different courses. All the data included in the analysis were obtained from official university surveys developed by a team of professionals specialized in teaching quality responsible for ensuring the reliability of the information. At the degree level, the results show that despite the considerable cultural and temporal difference between the samples, the results are very similar to those obtained by Uttl & Smibert (2017, https://doi.org/10.7717/peerj.3299); i.e., professors teaching quantitative courses are far more likely to obtain worse SETs than instructors in other areas. There are hardly any differences at the master's degree level, regardless of whether nearly 75% of master's degree instructors also teach at the undergraduate level. This leads us to three different conclusions. (1) Evidence suggests that the reason for these differences is not due to faculty teaching quantitative courses being less effective than faculty teaching in some other fields. Our results indicate that the same instructor is evaluated very differently depending on whether he or she teaches at the undergraduate or master's level. (2) It is essential to avoid comparisons of SETs between different areas of knowledge, at least at the undergraduate level. (3) A significant change in the use and interpretation of SETs is imperative, or its replacement by other evaluation mechanisms should be considered. If this does not occur, it is possible that in the future, there will be an adverse selection effect among professors of quantitative methods; i.e., only the worst professionals in quantitative methods will opt for teaching since the good professionals will prefer other jobs.

Project description:Just-In-Time Teaching (JiTT) active learning pedagogy is utilized by various disciplines, but its value in a professional pharmacy curriculum has not yet been demonstrated. The purpose of our research study is to implement and evaluate JiTT in a Doctor of Pharmacy (PharmD) program. The impetus in implementing JiTT into a PharmD curriculum was to provide students with an out-of-classroom learning opportunity to enhance knowledge-based skills. The current study summarizes the implementation of JiTT in four distinct instances: two iterations of the required courses "Integrated Microbiology and Virology" (Fall 2016 and Fall 2017) and "Integrated Immunology" (Winter 2016-2017 and Winter 2017-2018). JiTT included knowledge-based questions in multiple-choice format, integrated case studies, and student responses prior to the actual lecture session. After the conclusion of each course, students were asked to provide feedback on the utilization of JiTT by way of an anonymous survey. Following the Fall 2016 iteration of the Microbiology & Virology course, students found the integrated case studies to be beneficial (mean = 3.27 out of a maximum of 4, SD = 0.62), and their overall endorsement of JiTT was high (mean = 3.61 out of 4, SD = 0.50). For the other three courses included in this study, the primary dependent variable was the student's average rating of JiTT, rated on a five-point scale. Aggregating the scores from the Fall 2017 iteration of the Integrated Microbiology & Virology course and both instances of the Immunology course, students rated JiTT very favorably (mean = 4.17 out of a maximum of 5, SD = 0.77). Students' performances in JiTT-based courses were compared against non-JiTT-based courses. Analysis of assessment data for student's performance on knowledge-based questions showed JiTT was helpful for student learning and JiTT-based courses had more consistent exam scores compared to non-JiTT-based courses. The current results are a promising initial step in validating the usefulness of JiTT in a pharmacy program and lays the foundation for future studies aimed at a direct comparison between a traditional lecture style and JiTT pedagogy implemented into PharmD curricula.

Project description:BACKGROUND:In a whole-of-system approach to evaluation of teaching across any degree, multiple sources of information can help develop an educators' understanding of their teaching quality. In the health professions, student evaluations of clinical teaching are commonplace. However, self-evaluation of teaching is less common, and exploration of clinical educators' self-efficacy even less so. The aim of the study was to evaluate how a clinical educator's self-evaluation of teaching intersects with their self-efficacy, to ascertain if that matches student evaluation of their teaching. This information may assist in facilitating targeted professional development to improve teaching quality. METHODS:Clinical educators in the osteopathy program at Victoria University (VU) were invited to complete: a) self-evaluation version of the Osteopathy Clinical Teaching Questionnaire (OCTQ); and b) the Self-Efficacy in Clinical Teaching (SECT) questionnaire. Students in the VU program completed the OCTQ for each of the clinical educators they worked with during semester 2, 2017. RESULTS:Completed OCTQ and SECT were received from 37 clinical educators. These were matched with 308 student evaluations (mean of 6 student ratings per educator). Three possible educator cohorts were identified: a) high clinical eductor self-OCTQ with low student evaluation; b) low clinical educator self-evaluation and high student evaluations; and, c) no difference between self- and student evaulations. Clinical educators in the first cohort demonstrated significantly higher SECT subscale scores (effect size >?0.42) than their colleagues. Age, gender, teaching qualification, and years practicing or years as a clinical educator were not associated with clinical educator OCTQ scores or the SECT subscales. CONCLUSIONS:Targeted professional development directed towards fostering self-efficacy may provide an avenue for engaging those clinical educators whose self-efficacy is low and/or those who did not receive high student evaluations. Given there is no gold standard measure of clinical teaching quality, educators should engage with multiple sources of feedback to benchmark their current performance level, and identify opportunities to improve. Student and self-evaluations using the OCTQ and evaluation of self-efficacy using the SECT, are useful tools for inclusion in a whole-of-system approach to evaluation of the clinical learning environment.

Dataset Information

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career.

Publications

Student evaluations of teaching: teaching quantitative courses can be hazardous to one's career.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets