Dataset Information

Visualizing learner engagement, performance, and trajectories to evaluate and optimize online course design.

ABSTRACT: Learning analytics and visualizations make it possible to examine and communicate learners' engagement, performance, and trajectories in online courses to evaluate and optimize course design for learners. This is particularly valuable for workforce training involving employees who need to acquire new knowledge in the most effective manner. This paper introduces a set of metrics and visualizations that aim to capture key dynamical aspects of learner engagement, performance, and course trajectories. The metrics are applied to identify prototypical behavior and learning pathways through and interactions with course content, activities, and assessments. The approach is exemplified and empirically validated using more than 30 million separate logged events that capture activities of 1,608 Boeing engineers taking the MITxPro Course, "Architecture of Complex Systems," delivered in Fall 2016. Visualization results show course structure and patterns of learner interactions with course material, activities, and assessments. Tree visualizations are used to represent course hierarchical structures and explicit sequence of content modules. Learner trajectory networks represent pathways and interactions of individual learners through course modules, revealing patterns of learner engagement, content access strategies, and performance. Results provide evidence for instructors and course designers for evaluating the usage and effectiveness of course materials and intervention strategies.

SUBMITTER: Ginda M

PROVIDER: S-EPMC6502341 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Visualizing learner engagement, performance, and trajectories to evaluate and optimize online course design.

Ginda Michael M Richey Michael C MC Cousino Mark M Börner Katy K

PloS one 20190506 5

Learning analytics and visualizations make it possible to examine and communicate learners' engagement, performance, and trajectories in online courses to evaluate and optimize course design for learners. This is particularly valuable for workforce training involving employees who need to acquire new knowledge in the most effective manner. This paper introduces a set of metrics and visualizations that aim to capture key dynamical aspects of learner engagement, performance, and course trajectorie ...[more]

PMID: 31059546

Similar Datasets

Project description:BACKGROUND:The digital revolution has led to a boom in the number of available online health care resources. To navigate these resources successfully, digital literacy education is required. Learners who can evaluate the reliability and validity of online health care information are likely to be more effective at avoiding potentially dangerous misinformation. In addition to providing health care education, massive open online courses (MOOCs) are well positioned to play a role in providing digital literacy education in this context. OBJECTIVE:This study focused on learners enrolled in a MOOC on cancer genomics. The aim of this study was to evaluate the efficacy of a series of digital literacy-related activities within this course. This was an iterative study, with changes made to digital literacy-related activities in 4 of the 8 runs of the course. METHODS:This mixed methods study focused on learner engagement with the digital literacy-related activities, including the final course written assignment. Quantitative data including the number of references listed in each written assignment were compared between successive runs. Qualitative data in the form of learner comments on discussion forums for digital literacy-related tasks were evaluated to determine the impact of these educational activities. RESULTS:Using the number of references included for each final course assignment as an indicator of digital literacy skills, the digital literacy-related activities in the final 2 runs were judged to be the most successful. We found a statistically significant increase in the number of references cited by learners in their final written assignments. The average number of references cited in Run 8 was significantly higher (3.5) than in Run 1 (1.8) of the MOOC (P=.001). Learner comments in Runs 7 and 8 showed that a poll in which learners were asked to select which of 4 online resources was reliable was effective in stimulating learner discussion about how to evaluate resource reliability. CONCLUSIONS:Similar to many health care MOOCs, the course studied here had a heterogeneous group of learners, including patients (and their families), the public, health care students, and practitioners. Carefully designing a range of digital literacy-related activities that would be beneficial to this heterogenous group of learners enabled learners to become more effective at evaluating and citing appropriate online resources within their written assignments.

Project description:With a motivation to immerse students in engineering design, graphics communication, and computer aided design (CAD) skills early-on in the biomedical engineering curriculum, we launched a new 2-unit laboratory course on "Graphics Design in BME" in the Spring 2020 quarter for UC Davis sophomores. Due to the COVID-19 pandemic, the course met with the significant challenge of conversion to an online mode of teaching, instead of planned face-to-face instruction. Providing formative feedback was thought to be an important step to help students succeed in their final CAD project of the course. In the process of designing feedback, we found that the concept of feedback is still fragile in an online learning environment because online learning settings provide distinct pedagogical demands as compared to face-to-face settings. The situation is especially delicate in the context of contemporary higher education imparting engineering skills, where students attend large classes, with diminished opportunities to interact with the teaching staff. The challenge we faced was to provide meaningful dialogic feedback in an online environment, especially while teaching engineering graphics design to a large class. Here we addressed this challenge by focusing on the process of structuring meaningful feedback. We designed a project assignment to be submitted as multiple deliverables to be submitted in two-stages. Then, we characterized its feedback with multiple notions, such as dialogic iterative cycles, personalized, goal-directed, immediate, in written format, and having a peer assessment component. The process of providing formative feedback online through the structure mentioned in this paper resulted in students' improved scores on the final project elements. It also helped us identify the common issues students are faltering in a graphics design class, and provide customized guidance, ideal examples of expected work, and more resources to motivate each student group to achieve mastery of course content.Supplementary informationThe online version contains supplementary material available at 10.1007/s43683-021-00046-z.

Project description:BACKGROUND:Mobile health (mHealth) apps for pediatric chronic conditions are growing in availability and challenge investigators to conduct rigorous evaluations that keep pace with mHealth innovation. Traditional research methods are poorly suited to operationalize the agile, iterative trials required to evidence and optimize these digitally mediated interventions. OBJECTIVE:We sought to contribute a resource to support the quantification, analysis, and visualization of analytic indicators of effective engagement with mHealth apps for chronic conditions. METHODS:We applied user-centered design methods to design and develop an Analytics Platform to Evaluate Effective Engagement (APEEE) with consumer mHealth apps for chronic conditions and implemented the platform to analyze both retrospective and prospective data generated from a smartphone-based pain self-management app called iCanCope for young people with chronic pain. RESULTS:Through APEEE, we were able to automate the process of defining, operationalizing, and evaluating effective engagement with iCanCope. Configuring the platform to integrate with the app was feasible and provided investigators with a resource to consolidate, analyze, and visualize engagement data generated by participants in real time. Preliminary efforts to evaluate APEEE showed that investigators perceived the platform to be an acceptable evaluative resource and were satisfied with its design, functionality, and performance. Investigators saw potential in APEEE to accelerate and augment evidence generation and expressed enthusiasm for adopting the platform to support their evaluative practice once fully implemented. CONCLUSIONS:Dynamic, real-time analytic platforms may provide investigators with a powerful means to characterize the breadth and depth of mHealth app engagement required to achieve intended health outcomes. Successful implementation of APEEE into evaluative practice may contribute to the realization of effective and evidence-based mHealth care.

Dataset Information

Visualizing learner engagement, performance, and trajectories to evaluate and optimize online course design.

Publications

Visualizing learner engagement, performance, and trajectories to evaluate and optimize online course design.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets