Dataset Information

Measuring Criterion Validity of Microinteraction Ecological Momentary Assessment (Micro-EMA): Exploratory Pilot Study With Physical Activity Measurement.

ABSTRACT:

Background

Ecological momentary assessment (EMA) is an in situ method of gathering self-report on behaviors using mobile devices. In typical phone-based EMAs, participants are prompted repeatedly with multiple-choice questions, often causing participation burden. Alternatively, microinteraction EMA (micro-EMA or μEMA) is a type of EMA where all the self-report prompts are single-question surveys that can be answered using a 1-tap glanceable microinteraction conveniently on a smartwatch. Prior work suggests that μEMA may permit a substantially higher prompting rate than EMA, yielding higher response rates and lower participation burden. This is achieved by ensuring μEMA prompt questions are quick and cognitively simple to answer. However, the validity of participant responses from μEMA self-report has not yet been formally assessed.

Objective

In this pilot study, we explored the criterion validity of μEMA self-report on a smartwatch, using physical activity (PA) assessment as an example behavior of interest.

Methods

A total of 17 participants answered 72 μEMA prompts each day for 1 week using a custom-built μEMA smartwatch app. At each prompt, they self-reported whether they were doing sedentary, light/standing, moderate/walking, or vigorous activities by tapping on the smartwatch screen. Responses were compared with a research-grade activity monitor worn on the dominant ankle simultaneously (and continuously) measuring PA.

Results

Participants had an 87.01% (5226/6006) μEMA completion rate and a 74.00% (5226/7062) compliance rate taking an average of only 5.4 (SD 1.5) seconds to answer a prompt. When comparing μEMA responses with the activity monitor, we observed significantly higher (P<.001) momentary PA levels on the activity monitor when participants self-reported engaging in moderate+vigorous activities compared with sedentary or light/standing activities. The same comparison did not yield any significant differences in momentary PA levels as recorded by the activity monitor when the μEMA responses were randomly generated (ie, simulating careless taps on the smartwatch).

Conclusions

For PA measurement, high-frequency μEMA self-report could be used to capture information that appears consistent with that of a research-grade continuous sensor for sedentary, light, and moderate+vigorous activity, suggesting criterion validity. The preliminary results show that participants were not carelessly answering μEMA prompts by randomly tapping on the smartwatch but were reporting their true behavior at that moment. However, more research is needed to examine the criterion validity of μEMA when measuring vigorous activities.

SUBMITTER: Ponnada A

PROVIDER: S-EPMC7991987 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.

Project description:ObjectivesBecause audiometric evaluation, symptom histories, questionnaires, and similar standard assessment tools may not adequately sample the effects of chronic tinnitus on day-to-day activities, there is a need for alternative methodological approaches to study the impact of tinnitus on day-to-day life. An innovative methodological approach that has shown great promise in the study of chronic health problems characterized by reported temporal and/or situational variability in symptoms and distress is known as ecological momentary assessment (EMA). EMA involves the real-time measurement of states, situational factors, and symptoms by individuals as they go about their day-to-day activities. The objective of this pilot investigation was to explore the feasibility of using EMA methods to examine within- and between-day effects of tinnitus.DesignThis study was conducted in three phases: (1) design and development of an EMA methodology that could be used to assess effects of tinnitus; (2) refinement of the methodology through the use of two focus groups; and (3) field-test the methodology with individuals who experienced bothersome tinnitus. For Phase 3, each of the 24 participants wore, throughout their waking hours for 2 weeks, a personal digital assistant that produced alerts four times a day. The alerts prompted participants to respond to 19 questions, including 9 relating to situational and mood factors and 10 comprising the Tinnitus Handicap Inventory-Screening version (THI-S). To evaluate for potential reactive effects of performing the EMA protocol, each participant completed the paper-and-pencil version of the full 25-item THI before and after the 2-week EMA period.ResultsParticipants responded to the alerts with a 90% compliance rate, providing a total of 1210 completed surveys. At the time of their response, participants indicated that they were in their house or apartment (67.7%), alone (50.2%), happy (50%), and calm (54.5%). Across most responses, participants could hear their tinnitus (97%), and the loudness of their tinnitus averaged 4.7 on a 7-point increasing-loudness scale. The mean THI-S index score (out of a possible maximum 40 points for greatest tinnitus severity) was 17.0 (moderate self-perceived tinnitus handicap). Repeated THI-S index scores varied considerably both within and between participants. Mean 25-item THI scores were not significantly different before and after the EMA period, suggesting little reactivity of the EMA.ConclusionsThe high compliance rate, positive feedback from participants, lack of reactivity as a result of performing the EMA protocol, and data collected indicate that EMA methodology is feasible with patients who have tinnitus. Outcome data obtained with this methodology cannot be obtained any other way because retrospective questionnaires cannot capture the day-to-day reactions. This methodology has the potential to provide more in-depth and accurate assessments of patients receiving therapy for tinnitus.