Dataset Information

Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ).

ABSTRACT: In higher education, student ratings are often used to evaluate and improve the quality of courses and professors' instructional skills. Unfortunately, student-rating questionnaires rarely generate specific feedback for professors to improve their instructional skills. The impact of student ratings on professors' instructional skills has proven to be low. This study concerns the psychometric properties of the Instructional Skills Questionnaire (ISQ), a new theory-based student-rating-of-teaching questionnaire with specific questions concerning lecturing skills. The ISQ is administered after a single lecture. This way, it serves as a formative feedback instrument for university professors during courses to assist them to improve and (re-) evaluate their skills if necessary. The ISQ contains seven dimensions of professors' instructional skills and three student (self perceived) learning outcomes. In this study, Dutch students in 75 courses rated three 90-minute lectures (T1, T2 and T3) of their respective professors using the ISQ. In total, 14,298 ISQ-forms were used to rate 225 lectures. The teacher level reliabilities of the seven dimensions were found to be good at each measurement occasion. In addition, confirmatory multilevel factor analysis confirmed a seven dimensional factor structure at the teacher level at each measurement occasion. Furthermore, specific teacher level factors significantly predicted students' (self-assessed) learning outcomes. These results partly supported the proposed theoretical framework on the relationship between the ISQ teaching dimensions and the student learning process, and provided evidence for the construct validity of the instrument. In sum, the ISQ is found to be a reliable and valid instrument, which can be used by professors and faculty development centers to assess and improve university teaching.

SUBMITTER: Knol MH

PROVIDER: S-EPMC4769345 | biostudies-literature | 2016

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ).

Knol Mariska H MH Dolan Conor V CV Mellenbergh Gideon J GJ van der Maas Han L J HL

PloS one 20160226 2

In higher education, student ratings are often used to evaluate and improve the quality of courses and professors' instructional skills. Unfortunately, student-rating questionnaires rarely generate specific feedback for professors to improve their instructional skills. The impact of student ratings on professors' instructional skills has proven to be low. This study concerns the psychometric properties of the Instructional Skills Questionnaire (ISQ), a new theory-based student-rating-of-teaching ...[more]

PMID: 26918646

Similar Datasets

Project description:BackgroundImproving the coordination of cancer care is a priority area for service improvement. However, quality improvement initiatives are hindered by the lack of accurate and reliable measures of this aspect of cancer care. This study was conducted to develop a questionnaire to measures patients' experience of cancer care coordination and to assess the psychometric properties of this instrument.MethodsQuestionnaire items were developed on the basis of literature review and qualitative research involving focus groups and interviews with cancer patients, carers and clinicians. The draft instrument was completed 686 patients who had been recently treated for a newly diagnosed cancer, including patients from metropolitan, regional and rural areas of New South Wales, Australia. To assess test-retest reliability, 119 patients completed the questionnaire twice. Unreliable items those with limited variability or high levels of missing data were eliminated. Exploratory factor analysis was conducted to define the underlying factor structure of the remaining items and subscales were constructed. Correlations between these and global measures of the experience of care coordination and the quality of care were assessed.ResultsOf 40 items included in the draft questionnaire, 20 were eliminated due to poor test-retest reliability (n = 4), limited response distributions (n = 8), failure to load onto a factor (n = 7) or detrimental effect on the internal consistency of the scale (n = 1). The remaining 20 items loaded onto two factors named 'Communication' and 'Navigation', which explained 91% of the common variance. Internal consistency was with high for the instrument (Cronbach's alpha 0.88) and each subscale (Cronbach's alpha 0.87 and 0.73 respectively). There was no apparent 'floor' or 'ceiling' effect for the total score or the Communication subscale, but evidence of a ceiling effect for the Navigation subscale with 21% of respondents achieving the highest possible score. There were moderate positive associations between the total score and global measures of care coordination (r = 0.57) and quality of care (r = 0.53).ConclusionsThe instrument developed in this study demonstrated consistency and robust psychometric properties. It may provide a useful tool to measure patients' experience of cancer care coordination in future surveys and intervention studies.

Project description:BackgroundThe quality of life among children with idiopathic scoliosis during their adolescence has been reported to be affected by the brace itself. However, a controversy exists whether brace treated scoliotics experience a poor quality of life, thus there is a need for the development of a brace-oriented instrument, as the now-existing questionnaires that are commonly used, such as the SRS -22, take into consideration the effects of both the conservative and the surgical treatment on quality of life of scoliotic children. The aim of the present study is to assess the validity and reliability of Brace Questionnaire (BrQ), a new instrument for measuring quality of life of scoliotic adolescents who are treated conservatively with a brace.Material-methodMethodology of development involved literature review, patient and health care professionals' in-depth interviews and content validity analysis on patients. A validation study was performed on 28 brace treated scoliotic children aged between 9 and 18 years old. BrQ was assessed for the following psychometric properties: item convergent validity, floor and ceiling effects, internal consistency reliability, clinical validity and responsiveness to change.ResultsBrQ is self administrated and developmentally appropriate for ages 9 to 18 years old and is consisted of 34 Likert-scale items associated with eight domains: general health perception, physical functioning, emotional functioning, self esteem and aesthetics, vitality, school activity, bodily pain and social functioning of scoliotic children treated conservatively with a brace. The subscales of these eight dimensions can be combined to produce a total score. Higher scores mean a better quality of life. An item convergent validity >/= 0.40 was satisfied by all items in the present study. A satisfactory internal consistency reliability for the BrQ was recorded (Cronbach's alpha coefficient was 0.82). There were no floor or ceiling effects. The correlation between BrQ overall scores and mild and moderate scoliosis was statistically significant (p < 0.001), revealing high clinical validity. An increase in effect sizes for the patient with improved scoliotic curves indicates that the BrQ is responsive to change in health status.ConclusionBrQ is reliable, valid and responsive to change in adolescents with IS who are treated conservatively with a brace.

Project description:This study describes the development of a new questionnaire to measure health-related quality of life (HRQoL) in patients with type 1 diabetes (the ViDa1 questionnaire) and provides information on its psychometric properties. For its development, open interviews with patients took place and topics relevant to patients' HRQoL were identified and items were generated. Qualitative analysis of items, expert review, and refinement of the questionnaire followed. A pilot study (N = 150) was conducted to explore the underlying structure of the 40-item ViDa1 questionnaire. A Principal Component Analysis (PCA) was performed and six of the items that did not load on any of the factors were eliminated. The results supported a four-dimensional structure for ViDa1, the dimensions being Interference of diabetes in everyday life, Self-care, Well-being, and Worry about the disease. Subsequently, the PCA was repeated in a larger sample (N = 578) with the reduced 34-item version of the questionnaire, and a Confirmatory Factor Analysis (CFA) was performed (N = 428). Overall fit indices obtained presented adequate values which supported the four-factor model initially proposed [([Formula: see text] 2601.93) (p < 0.001); Root Mean Square Error of Approximation = 0.060 (CI = 0.056 -0.064)]. As regards reliability, the four dimensions of the ViDa1 demonstrated good internal consistency, with Cronbach's alphas ranging between 0.71 and 0.86. Evidence of convergent-discriminant validity in the form of high correlations with another specific HRQoL questionnaire for diabetes and low correlations with other constructs such as self-efficacy, anxiety, and depression were presented. The ViDa1 also discriminated between different aspects of clinical interest such as type of insulin treatment, presence of chronic complications, and glycemic control, temporal stability, and sensitivity to change after an intervention. In conclusion, the ViDa1 questionnaire presents adequate psychometric properties and may represent a good alternative for the evaluation of HRQoL in type 1 diabetes.

Project description:The aim of this study was to develop a questionnaire to conveniently assess the diurnal preferences of physical activity (PA) in Japanese university students. A total of 219 subjects completed our novel Morningness-eveningness Exercise Preference Questionnaire (MEEPQ). The MEEPQ consisted of 30 items (15 items for the morning and the same 15 items for the evening) rated on a 5-point Likert scale concerning their preference for participating in PA in the morning and evening. The morning score (MS) and evening score (ES) were determined by summing each of the respective 15 items. The internal consistency and construct validity were assessed, and a factor analysis was conducted. To examine the external validity of the MEEPQ, participants wore an accelerometer for seven consecutive days to measure their PA levels objectively. Finally, the test-retest reliability was evaluated at a one-month interval. The MEEPQ showed excellent internal consistency (Cronbach's alpha = 0.896) and construct validity (morning KMO = 0.913, evening KMO = 0.875). A factor analysis showed a three-factor structure involving Physical Wellness (MEEPQ-W), Psychological Well-Being (MEEPQ-P) and Exercise Barrier (MEEPQ-B). The percent of variance was largest for MEEPQ-W in the morning (45.2%) and MEEPQ-P in the evening (40.8%). Test-retest showed that MEEPQ scores had fair repeatability. Significant and positive associations between scores and objectively measured PA levels were found in the MS and 6-9 AM PA and in the ES and 6-9 PM and 9 PM- 0 AM PA (all p<0.05). In summary, the novel MEEPQ showed relatively good agreement and thus can be used for Japanese university student samples. In the MEEPQ, three factors (the physical wellness, psychological well-being and exercise barrier) contributed to a morning or evening PA preference. The summed scores were significantly associated with the objectively measured PA levels in both the morning and evening. Therefore the MEEPQ appears to be a suitable tool for assessing diurnal PA preferences.

Project description:BackgroundWeb-based health behavior change interventions may be more effective if they offer autonomy-supportive communication facilitating the internalization of motivation for health behavior change. Yet, at this moment no validated tools exist to assess user-perceived autonomy-support of such interventions.ObjectiveThe aim of this study was to develop and validate the virtual climate care questionnaire (VCCQ), a measure of perceived autonomy-support in a virtual care setting.MethodsItems were developed based on existing questionnaires and expert consultation and were pretested among experts and target populations. The virtual climate care questionnaire was administered in relation to Web-based interventions aimed at reducing consumption of alcohol (Study 1; N=230) or cannabis (Study 2; N=228). Item properties, structural validity, and reliability were examined with item-response and classical test theory methods, and convergent and divergent validity via correlations with relevant concepts.ResultsIn Study 1, 20 of 23 items formed a one-dimensional scale (alpha=.97; omega=.97; H=.66; mean 4.9 [SD 1.0]; range 1-7) that met the assumptions of monotonicity and invariant item ordering. In Study 2, 16 items fitted these criteria (alpha=.92; H=.45; omega=.93; mean 4.2 [SD 1.1]; range 1-7). Only 15 items remained in the questionnaire in both studies, thus we proceeded to the analyses of the questionnaire's reliability and construct validity with a 15-item version of the virtual climate care questionnaire. Convergent validity of the resulting 15-item virtual climate care questionnaire was confirmed by positive associations with autonomous motivation (Study 1: r=.66, P<.001; Study 2: r=.37, P<.001) and perceived competence for reducing alcohol intake (Study 1: r=.52, P<.001). Divergent validity could only be confirmed by the nonsignificant association with perceived competence for learning (Study 2: r=.05, P=.48).ConclusionsThe virtual climate care questionnaire accurately assessed participants' perceived autonomy-support offered by two Web-based health behavior change interventions. Overall, the scale showed the expected properties and relationships with relevant concepts, and the studies presented suggest this first version of the virtual climate care questionnaire to be reasonably valid and reliable. As a result, the current version may cautiously be used in future research and practice to measure perceived support for autonomy within a virtual care climate. Future research efforts are required that focus on further investigating the virtual climate care questionnaire's divergent validity, on determining the virtual climate care questionnaire's validity and reliability when used in the context of Web-based interventions aimed at improving nonaddictive or other health behaviors, and on developing and validating a short form virtual climate care questionnaire.

Dataset Information

Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ).

Publications

Measuring the Quality of University Lectures: Development and Validation of the Instructional Skills Questionnaire (ISQ).

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets