Dataset Information

Interactive model building for Q-learning.

ABSTRACT: Evidence-based rules for optimal treatment allocation are key components in the quest for efficient, effective health care delivery. Q-learning, an approximate dynamic programming algorithm, is a popular method for estimating optimal sequential decision rules from data. Q-learning requires the modeling of nonsmooth, nonmonotone transformations of the data, complicating the search for adequately expressive, yet parsimonious, statistical models. The default Q-learning working model is multiple linear regression, which is not only provably misspecified under most data-generating models, but also results in nonregular regression estimators, complicating inference. We propose an alternative strategy for estimating optimal sequential decision rules for which the requisite statistical modeling does not depend on nonsmooth, nonmonotone transformed data, does not result in nonregular regression estimators, is consistent under a broader array of data-generation models than Q-learning, results in estimated sequential decision rules that have better sampling properties, and is amenable to established statistical approaches for exploratory data analysis, model building, and validation. We derive the new method, IQ-learning, via an interchange in the order of certain steps in Q-learning. In simulated experiments IQ-learning improves on Q-learning in terms of integrated mean squared error and power. The method is illustrated using data from a study of major depressive disorder.

SUBMITTER: Laber EB

PROVIDER: S-EPMC4274394 | biostudies-literature | 2014 Oct

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Interactive model building for <i>Q</i>-learning.

Laber Eric B EB Linn Kristin A KA Stefanski Leonard A LA

Biometrika 20141001 4

Evidence-based rules for optimal treatment allocation are key components in the quest for efficient, effective health care delivery. Q-learning, an approximate dynamic programming algorithm, is a popular method for estimating optimal sequential decision rules from data. Q-learning requires the modeling of nonsmooth, nonmonotone transformations of the data, complicating the search for adequately expressive, yet parsimonious, statistical models. The default Q-learning working model is multiple lin ...[more]

PMID: 25541562

Similar Datasets

Project description:BackgroundResilience is a person's mental ability to deal with challenging situations adaptively and is a crucial stress management skill. Psychological resilience and finding ways to cope in crises is a highly relevant topic considering the COVID-19 pandemic, which enforced quarantine, social distancing measures, and school closures worldwide. Parents and children are currently living with increased stress due to COVID-19. We need to respond with immediate ways to strengthen children's resilience. Internet-based cognitive behavioral therapy interventions for children's stress management overcome accessibility issues such as the inability to visit mental health experts owing to COVID-19 movement restrictions. An interactive learning environment was created, based on the preventive program "Friends," to overcome accessibility issues associated with delivering cognitive behavioral therapy-based interventions in formal and informal education settings.ObjectiveThis study aimed to examine the effectiveness of a web-based learning environment on resilience in (1) reducing anxiety symptoms and (2) increasing emotion recognition and recognition of stress management techniques among 9-10-year-old children. We also aimed to evaluate the learning environment's usability.MethodsA quasi-experimental pretest-posttest control group design was used. In total, 20 fourth graders in the experimental group interacted with the learning environment over 6 weekly 80-minute sessions. Further, 21 fourth graders constituted the control group. The main data sources were (1) a psychometric tool to measure children's anxiety symptoms, namely the Greek translation of the original Spence Children's Anxiety Scale, (2) 3 open-ended questions assessing emotion recognition and recognition of stress management techniques, and (3) the System Usability Scale to measure the usability of the learning environment.ResultsIn both groups, there was a small but nonsignificant postintervention reduction in reported anxiety symptoms, except for obsessive-compulsive disorder symptoms in the experimental group. A paired samples t test revealed that students' reported symptom scores of obsessive-compulsive disorder significantly decreased from 1.06 (SD 0.68) to 0.76 (SD 0.61) (t19= 5.16; P=.01). The experimental group revealed a significant increase in emotion recognition (t19=-6.99; P<.001), identification of somatic symptoms of stress (t19=-7.31; P<.001), and identification of stress management techniques (t19=-6.85; P<.001). The learning environment received a satisfactory usability score. The raw average system usability score was 76.75 (SD 8.28), which is in the 80th percentile rank and corresponds to grade B.ConclusionsThis study shows that interactive learning environments might deliver resilience interventions in an accessible and cost-effective manner in formal education, potentially even in distance-learning conditions owing to the COVID-19 pandemic. Interactive learning environments on resilience are also valuable tools for parents who can use them with their children at home, for informal learning, using mobile devices. As such, they could be a promising first-step, low-intensity intervention that children and the youth can easily access.

Project description:BackgroundHepatocellular carcinoma (HCC) is the leading liver cancer with special immune microenvironment, which played vital roles in tumor relapse and poor drug responses. In this study, we aimed to explore the prognostic immune signatures in HCC and tried to construct an immune-risk model for patient evaluation.MethodsRNA sequencing profiles of HCC patients were collected from the cancer genome Atlas (TCGA), international cancer genome consortium (ICGC), and gene expression omnibus (GEO) databases (GSE14520). Differentially expressed immune genes, derived from ImmPort database and MSigDB signaling pathway lists, between tumor and normal tissues were analyzed with Limma package in R environment. Univariate Cox regression was performed to find survival-related immune genes in TCGA dataset, and in further random forest algorithm analysis, significantly changed immune genes were used to generate a multivariate Cox model to calculate the corresponding immune-risk score. The model was examined in the other two datasets with recipient operation curve (ROC) and survival analysis. Risk effects of immune-risk score and clinical characteristics of patients were individually evaluated, and significant factors were then used to generate a nomogram.ResultsThere were 52 downregulated and 259 upregulated immune genes between tumor and relatively normal tissues, and the final immune-risk model (based on SPP1, BRD8, NDRG1, KITLG, HSPA4, TRAF3, ITGAV and MAP4K2) can better differentiate patients into high and low immune-risk subpopulations, in which high score patients showed worse outcomes after resection (p < 0.05). The differentially enriched pathways between the two groups were mainly about cell proliferation and cytokine production, and calculated immune-risk score was also highly correlated with immune infiltration levels. The nomogram, constructed with immune-risk score and tumor stages, showed high accuracy and clinical benefits in prediction of 1-, 3- and 5-year overall survival, which is useful in clinical practice.ConclusionThe immune-risk model, based on expression of SPP1, BRD8, NDRG1, KITLG, HSPA4, TRAF3, ITGAV, and MAP4K2, can better differentiate patients into high and low immune-risk groups. Combined nomogram, using immune-risk score and tumor stages, could make accurate prediction of 1-, 3- and 5-year survival in HCC patients.

Dataset Information

Interactive model building for Q-learning.

Publications

Interactive model building for <i>Q</i>-learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets