Dataset Information

Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning analytics application.

ABSTRACT: For decades, self-report measures based on questionnaires have been widely used in educational research to study implicit and complex constructs such as motivation, emotion, cognitive and metacognitive learning strategies. However, the existence of potential biases in such self-report instruments might cast doubts on the validity of the measured constructs. The emergence of trace data from digital learning environments has sparked a controversial debate on how we measure learning. On the one hand, trace data might be perceived as "objective" measures that are independent of any biases. On the other hand, there is mixed evidence of how trace data are compatible with existing learning constructs, which have traditionally been measured with self-reports. This study investigates the strengths and weaknesses of different types of data when designing predictive models of academic performance based on computer-generated trace data and survey data. We investigate two types of bias in self-report surveys: response styles (i.e., a tendency to use the rating scale in a certain systematic way that is unrelated to the content of the items) and overconfidence (i.e., the differences in predicted performance based on surveys' responses and a prior knowledge test). We found that the response style bias accounts for a modest to a substantial amount of variation in the outcomes of the several self-report instruments, as well as in the course performance data. It is only the trace data, notably that of process type, that stand out in being independent of these response style patterns. The effect of overconfidence bias is limited. Given that empirical models in education typically aim to explain the outcomes of learning processes or the relationships between antecedents of these learning outcomes, our analyses suggest that the bias present in surveys adds predictive power in the explanation of performance data and other questionnaire data.

SUBMITTER: Tempelaar D

PROVIDER: S-EPMC7292385 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning analytics application.

Tempelaar Dirk D Rienties Bart B Nguyen Quan Q

PloS one 20200612 6

For decades, self-report measures based on questionnaires have been widely used in educational research to study implicit and complex constructs such as motivation, emotion, cognitive and metacognitive learning strategies. However, the existence of potential biases in such self-report instruments might cast doubts on the validity of the measured constructs. The emergence of trace data from digital learning environments has sparked a controversial debate on how we measure learning. On the one han ...[more]

PMID: 32530954

Similar Datasets

Project description:BackgroundAcute kidney injury (AKI) is a common complication after surgery that is associated with increased morbidity and mortality. The majority of existing perioperative AKI risk prediction models are limited in their generalizability and do not fully utilize intraoperative physiological time-series data. Thus, there is a need for intelligent, accurate, and robust systems to leverage new information as it becomes available to predict the risk of developing postoperative AKI.MethodsA retrospective single-center cohort of 2,911 adults who underwent surgery at the University of Florida Health between 2000 and 2010 was utilized for this study. Machine learning and statistical analysis techniques were used to develop perioperative models to predict the risk of developing AKI during the first three days after surgery, first seven days after surgery, and overall (after surgery during the index hospitalization). The improvement in risk prediction was examined by incorporating intraoperative physiological time-series variables. Our proposed model enriched a preoperative model that produced a probabilistic AKI risk score by integrating intraoperative statistical features through a machine learning stacking approach inside a random forest classifier. Model performance was evaluated using the area under the receiver operating characteristic curve (AUC), accuracy, and Net Reclassification Improvement (NRI).ResultsThe predictive performance of the proposed model is better than the preoperative data only model. The proposed model had an AUC of 0.86 (accuracy of 0.78) for the seven-day AKI outcome, while the preoperative model had an AUC of 0.84 (accuracy of 0.76). Furthermore, by integrating intraoperative features, the algorithm was able to reclassify 40% of the false negative patients from the preoperative model. The NRI for each outcome was AKI at three days (8%), seven days (7%), and overall (4%).ConclusionsPostoperative AKI prediction was improved with high sensitivity and specificity through a machine learning approach that dynamically incorporated intraoperative data.

Dataset Information

Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning analytics application.

Publications

Subjective data, objective data and the role of bias in predictive modelling: Lessons from a dispositional learning analytics application.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets