Dataset Information

Predicting Participant Compliance With Fitness Tracker Wearing and Ecological Momentary Assessment Protocols in Information Workers: Observational Study.

ABSTRACT:

Background

Studies that use ecological momentary assessments (EMAs) or wearable sensors to track numerous attributes, such as physical activity, sleep, and heart rate, can benefit from reductions in missing data. Maximizing compliance is one method of reducing missing data to increase the return on the heavy investment of time and money into large-scale studies.

Objective

This paper aims to identify the extent to which compliance can be prospectively predicted from individual attributes and initial compliance.

Methods

We instrumented 757 information workers with fitness trackers for 1 year and conducted EMAs in the first 56 days of study participation as part of an observational study. Their compliance with the EMA and fitness tracker wearing protocols was analyzed. Overall, 31 individual characteristics (eg, demographics and personalities) and behavioral variables (eg, early compliance and study portal use) were considered, and 14 variables were selected to create beta regression models for predicting compliance with EMAs 56 days out and wearable compliance 1 year out. We surveyed study participation and correlated the results with compliance.

Results

Our modeling indicates that 16% and 25% of the variance in EMA compliance and wearable compliance, respectively, could be explained through a survey of demographics and personality in a held-out sample. The likelihood of higher EMA and wearable compliance was associated with being older (EMA: odds ratio [OR] 1.02, 95% CI 1.00-1.03; wearable: OR 1.02, 95% CI 1.01-1.04), speaking English as a first language (EMA: OR 1.38, 95% CI 1.05-1.80; wearable: OR 1.39, 95% CI 1.05-1.85), having had a wearable before joining the study (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.50, 95% CI 1.23-1.83), and exhibiting conscientiousness (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.34, 95% CI 1.14-1.58). Compliance was negatively associated with exhibiting extraversion (EMA: OR 0.74, 95% CI 0.64-0.85; wearable: OR 0.67, 95% CI 0.57-0.78) and having a supervisory role (EMA: OR 0.65, 95% CI 0.54-0.79; wearable: OR 0.66, 95% CI 0.54-0.81). Furthermore, higher wearable compliance was negatively associated with agreeableness (OR 0.68, 95% CI 0.56-0.83) and neuroticism (OR 0.85, 95% CI 0.73-0.98). Compliance in the second week of the study could help explain more variance; 62% and 66% of the variance in EMA compliance and wearable compliance, respectively, was explained. Finally, compliance correlated with participants' self-reflection on the ease of participation, usefulness of our compliance portal, timely resolution of issues, and compensation adequacy, suggesting that these are avenues for improving compliance.

Conclusions

We recommend conducting an initial 2-week pilot to measure trait-like compliance and identify participants at risk of long-term noncompliance, performing oversampling based on participants' individual characteristics to avoid introducing bias in the sample when excluding data based on noncompliance, using an issue tracking portal, and providing special care in troubleshooting to help participants maintain compliance.

SUBMITTER: Martinez GJ

PROVIDER: S-EPMC8663716 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMobile device-based ecological momentary assessment (mobile-EMA) is increasingly used to collect participants' data in real-time and in context. Although EMA offers methodological advantages, these advantages can be diminished by participant noncompliance. However, evidence on how well participants comply with mobile-EMA protocols and how study design factors associated with participant compliance is limited, especially in the youth literature.ObjectiveTo systematically and meta-analytically examine youth's compliance to mobile-EMA protocols and moderators of participant compliance in clinical and nonclinical settings.MethodsStudies using mobile devices to collect EMA data among youth (age ≤18 years old) were identified. A systematic review was conducted to describe the characteristics of mobile-EMA protocols and author-reported factors associated with compliance. Random effects meta-analyses were conducted to estimate the overall compliance across studies and to explore factors associated with differences in youths' compliance.ResultsThis review included 42 unique studies that assessed behaviors, subjective experiences, and contextual information. Mobile phones were used as the primary mode of EMA data collection in 48% (20/42) of the reviewed studies. In total, 12% (5/42) of the studies used wearable devices in addition to the EMA data collection platforms. About half of the studies (62%, 24/42) recruited youth from nonclinical settings. Most (98%, 41/42) studies used a time-based sampling protocol. Among these studies, most (95%, 39/41) prompted youth 2-9 times daily, for a study length ranging from 2-42 days. Sampling frequency and study length did not differ between studies with participants from clinical versus nonclinical settings. Most (88%, 36/41) studies with a time-based sampling protocol defined compliance as the proportion of prompts to which participants responded. In these studies, the weighted average compliance rate was 78.3%. The average compliance rates were not different between studies with clinical (76.9%) and nonclinical (79.2%; P=.29) and studies that used only a mobile-EMA platform (77.4%) and mobile platform plus additional wearable devices (73.0%, P=.36). Among clinical studies, the mean compliance rate was significantly lower in studies that prompted participants 2-3 times (73.5%) or 4-5 times (66.9%) compared with studies with a higher sampling frequency (6+ times: 89.3%). Among nonclinical studies, a higher average compliance rate was observed in studies that prompted participants 2-3 times daily (91.7%) compared with those that prompted participants more frequently (4-5 times: 77.4%; 6+ times: 75.0%). The reported compliance rates did not differ by duration of EMA period among studies from either clinical or nonclinical settings.ConclusionsThe compliance rate among mobile-EMA studies in youth is moderate but suboptimal. Study design may affect protocol compliance differently between clinical and nonclinical participants; including additional wearable devices did not affect participant compliance. A more consistent compliance-related result reporting practices can facilitate understanding and improvement of participant compliance with EMA data collection among youth.

Project description:BackgroundEcological momentary assessment (EMA) has the potential to minimize recall bias by having people report on their experiences in the moment (momentary model) or over short periods (coverage model). This potential hinges on the assumption that participants provide their ratings based on the reporting time frame instructions prescribed in the EMA items. However, it is unclear what time frames participants actually use when answering the EMA questions and whether participant training improves participants' adherence to the reporting instructions.ObjectiveThis study aims to investigate the reporting time frames participants used when answering EMA questions and whether participant training improves participants' adherence to the EMA reporting timeframe instructions.MethodsTelephone-based cognitive interviews were used to investigate the research questions. In a 2×2 factorial design, participants (n=100) were assigned to receive either basic or enhanced EMA training and randomized to rate their experiences using a momentary (at the moment you were called) or a coverage (since the last phone call) model. Participants received five calls over the course of a day to provide ratings; after each rating, participants were immediately interviewed about the time frame they used to answer the EMA questions. A total of 2 raters independently coded the momentary interview responses into time frame categories (Cohen κ=0.64, 95% CI 0.55-0.73).ResultsThe results from the momentary conditions showed that most of the calls referred to the period during the call (57/199, 28.6%) or just before the call (98/199, 49.2%) to provide ratings; the remainder were from longer reporting periods. Multinomial logistic regression results indicated a significant training effect (χ21=16.6; P<.001) in which the enhanced training condition yielded more reports within the intended reporting time frames for momentary EMA reports. Cognitive interview data from the coverage model did not lend themselves to reliable coding and were not analyzed.ConclusionsThe results of this study provide the first evidence about adherence to EMA instructions to reporting periods and that enhanced participant training improves adherence to the time frame specified in momentary EMA studies.

Project description:BackgroundEcological momentary assessment (EMA) is increasingly used to evaluate behavioral health processes over extended time periods. The validity of EMA for providing representative, real-world data with high temporal precision is threatened to the extent that EMA compliance drops over time.ObjectiveThis research builds on prior short-term studies by evaluating the time course of EMA compliance over 9 weeks and examines predictors of weekly compliance rates among cigarette-using adults.MethodsA total of 257 daily cigarette-using adults participating in a randomized controlled trial for smoking cessation completed daily smartphone EMA assessments, including 1 scheduled morning assessment and 4 random assessments per day. Weekly EMA compliance was calculated and multilevel modeling assessed the rate of change in compliance over the 9-week assessment period. Participant and study characteristics were examined as predictors of overall compliance and changes in compliance rates over time.ResultsCompliance was higher for scheduled morning assessments (86%) than for random assessments (58%) at the beginning of the EMA period (P<.001). EMA compliance declined linearly across weeks, and the rate of decline was greater for morning assessments (2% per week) than for random assessments (1% per week; P<.001). Declines in compliance were stronger for younger participants (P<.001), participants who were employed full-time (P=.03), and participants who subsequently dropped out of the study (P<.001). Overall compliance was higher among White participants compared to Black or African American participants (P=.001).ConclusionsThis study suggests that EMA compliance declines linearly but modestly across lengthy EMA protocols. In general, these data support the validity of EMA for tracking health behavior and hypothesized treatment mechanisms over the course of several months. Future work should target improving compliance among subgroups of participants and investigate the extent to which rapid declines in EMA compliance might prove useful for triggering interventions to prevent study dropout.Trial registrationClinicalTrials.gov NCT03262662; https://clinicaltrials.gov/ct2/show/NCT03262662.

Project description:BackgroundBorderline personality disorder (BPD) is characterized by frequent and intense moment-to-moment changes in affect, behavior, identity, and interpersonal relationships, which typically result in significant and negative deterioration of the person's overall functioning and well-being. Measuring and characterizing the rapidly changing patterns of instability in BPD dysfunction as they occur in a person's daily life can be challenging. Ecological momentary assessment (EMA) is a method that can capture highly dynamic processes in psychopathology research and, thus, is well suited to study intense variability patterns across areas of dysfunction in BPD. EMA studies are characterized by frequent repeated assessments that are delivered to participants in real-life, real-time settings using handheld devices capable of registering responses to short self-report questions in daily life. Compliance in EMA research is defined as the proportion of prompts answered by the participant, considering all planned prompts sent. Low compliance with prompt schedules can compromise the relative advantages of using this method. Despite the growing EMA literature on BPD in recent years, findings regarding study design features that affect compliance with EMA protocols have not been compiled, aggregated, and estimated.ObjectiveThis systematic meta-analytic review aimed to investigate the relationship between study design features and participant compliance in EMA research of BPD.MethodsA systematic review was conducted on November 12, 2021, following the PRISMA (Preferred Reporting Items for Systematic Reviews and Meta-Analyses) and MOOSE (Meta-analyses of Observational Studies in Epidemiology) guidelines to search for articles featuring EMA studies of BPD that reported compliance rates and included sufficient data to extract relevant design features. For studies with complete data, random-effect models were used to estimate the overall compliance rate and explore its association with design features.ResultsIn total, 28 peer-reviewed EMA studies comprising 2052 participants were included in the study. Design features (sampling strategy, average prompting frequency, number of items, response window, sampling device, financial incentive, and dropout rate) showed a large variability across studies, and many studies did not report design features. The meta-analytic synthesis was restricted to 64% (18/28) of articles and revealed a pooled compliance rate of 79% across studies. We did not find any significant relationship between design features and compliance rates.ConclusionsOur results show wide variability in the design and reporting of EMA studies assessing BPD. Compliance rates appear to be stable across varying setups, and it is likely that standard design features are not directly responsible for improving or diminishing compliance. We discuss possible nonspecific factors of study design that may have an impact on compliance. Given the promise of EMA research in BPD, we also discuss the importance of unifying standards for EMA reporting so that data stemming from this rich literature can be aggregated and interpreted jointly.