Dataset Information

A scalable approach to measuring the impact of nonignorable nonresponse with an EMA application.

ABSTRACT: There is often a need to assess the dependence of standard analyses on the strong untestable assumption of ignorable missingness. To tackle this problem, past research developed simple sensitivity index measures assuming a linear impact of nonignorability and missingness in outcomes only. These restrictions limit their applicability for studies with missingness in both outcome and covariates. Nonignorable missingness in this setting poses significant new analytic challenges and calls for more general and flexible methods that are also computationally tractable even for large datasets. In this paper, we relax the restrictions of extant linear sensitivity index methods and develop nonlinear sensitivity indices that maintain computational simplicity and perform equally well when the impact of nonignorability is locally linear. On the other hand, they can substantially improve the effectiveness of local sensitivity analysis when regression outcomes and covariates are subject to concurrent missingness. In this situation, the local linear sensitivity analysis fails to detect the impact of nonignorability while the proposed nonlinear sensitivity measures can. Because the new sensitivity indices avoid fitting complicated nonignorable models, they are computationally tractable (i.e., scalable) for use in large datasets. We develop general formula for nonlinear sensitivity index measures, and evaluate the new measures in simulated data and a real dataset collected using the ecological momentary assessment method. Copyright © 2016 John Wiley & Sons, Ltd.

SUBMITTER: Gao W

PROVIDER: S-EPMC5135668 | biostudies-literature | 2016 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

A scalable approach to measuring the impact of nonignorable nonresponse with an EMA application.

Gao Weihua W Hedeker Donald D Mermelstein Robin R Xie Hui H

Statistics in medicine 20160818 30

There is often a need to assess the dependence of standard analyses on the strong untestable assumption of ignorable missingness. To tackle this problem, past research developed simple sensitivity index measures assuming a linear impact of nonignorability and missingness in outcomes only. These restrictions limit their applicability for studies with missingness in both outcome and covariates. Nonignorable missingness in this setting poses significant new analytic challenges and calls for more ge ...[more]

PMID: 27538504

Similar Datasets

Project description:Digital interventions often suffer from low usage, which may reflect insufficient attention to user experience. Moreover, the existing evaluation methods have limited applicability in the remote study of user experience of complex interventions that have expansive content and that are used over an extensive period of time. To alleviate these challenges, we describe here a novel qualitative Ecological Momentary Assessment (EMA) method: the CORTO method (Contextual, One-item, Repeated, Timely, Open-ended). We used it to gather digital intervention user experience data from Finnish adults (n = 184) who lived with interview-confirmed major depressive disorder (MDD) and took part in a randomized controlled trial (RCT) that studied the efficacy of a novel 12-week game-based digital intervention for depression. A second dataset on user experience was gathered with retrospective interviews (n = 22). We inductively coded the CORTO method and retrospective interview data, which led to four user experience categories: (1) contextual use, (2) interaction-elicited emotional experience, (3) usability, and (4) technical issues. Then, we used the created user experience categories and Template Analysis to analyze both datasets together, and reported the results qualitatively. Finally, we compared the two datasets with each other. We found that the data generated with the CORTO method offered more insights into usability and technical categories than the interview data that particularly illustrated the contextual use. The emotional valence of the interview data was more positive compared with the CORTO data. Both the CORTO and interview data detected 55 % of the micro-level categories; 20 % of micro-level categories were only detected by the CORTO data and 25 % only by the interview data. We found that the during-intervention user experience measurement with the CORTO method can provide intervention-specific insights, and thereby further the iterative user-centered intervention development. Overall, these findings highlight the impact of evaluation methods on the categories and qualities of insights acquired in intervention research.

Project description:BackgroundEcological momentary assessment (EMA) is an in situ method of gathering self-report on behaviors using mobile devices. In typical phone-based EMAs, participants are prompted repeatedly with multiple-choice questions, often causing participation burden. Alternatively, microinteraction EMA (micro-EMA or μEMA) is a type of EMA where all the self-report prompts are single-question surveys that can be answered using a 1-tap glanceable microinteraction conveniently on a smartwatch. Prior work suggests that μEMA may permit a substantially higher prompting rate than EMA, yielding higher response rates and lower participation burden. This is achieved by ensuring μEMA prompt questions are quick and cognitively simple to answer. However, the validity of participant responses from μEMA self-report has not yet been formally assessed.ObjectiveIn this pilot study, we explored the criterion validity of μEMA self-report on a smartwatch, using physical activity (PA) assessment as an example behavior of interest.MethodsA total of 17 participants answered 72 μEMA prompts each day for 1 week using a custom-built μEMA smartwatch app. At each prompt, they self-reported whether they were doing sedentary, light/standing, moderate/walking, or vigorous activities by tapping on the smartwatch screen. Responses were compared with a research-grade activity monitor worn on the dominant ankle simultaneously (and continuously) measuring PA.ResultsParticipants had an 87.01% (5226/6006) μEMA completion rate and a 74.00% (5226/7062) compliance rate taking an average of only 5.4 (SD 1.5) seconds to answer a prompt. When comparing μEMA responses with the activity monitor, we observed significantly higher (P<.001) momentary PA levels on the activity monitor when participants self-reported engaging in moderate+vigorous activities compared with sedentary or light/standing activities. The same comparison did not yield any significant differences in momentary PA levels as recorded by the activity monitor when the μEMA responses were randomly generated (ie, simulating careless taps on the smartwatch).ConclusionsFor PA measurement, high-frequency μEMA self-report could be used to capture information that appears consistent with that of a research-grade continuous sensor for sedentary, light, and moderate+vigorous activity, suggesting criterion validity. The preliminary results show that participants were not carelessly answering μEMA prompts by randomly tapping on the smartwatch but were reporting their true behavior at that moment. However, more research is needed to examine the criterion validity of μEMA when measuring vigorous activities.

Dataset Information

A scalable approach to measuring the impact of nonignorable nonresponse with an EMA application.

Publications

A scalable approach to measuring the impact of nonignorable nonresponse with an EMA application.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets