Dataset Information

When does reward maximization lead to matching law?

ABSTRACT: What kind of strategies subjects follow in various behavioral circumstances has been a central issue in decision making. In particular, which behavioral strategy, maximizing or matching, is more fundamental to animal's decision behavior has been a matter of debate. Here, we prove that any algorithm to achieve the stationary condition for maximizing the average reward should lead to matching when it ignores the dependence of the expected outcome on subject's past choices. We may term this strategy of partial reward maximization "matching strategy". Then, this strategy is applied to the case where the subject's decision system updates the information for making a decision. Such information includes subject's past actions or sensory stimuli, and the internal storage of this information is often called "state variables". We demonstrate that the matching strategy provides an easy way to maximize reward when combined with the exploration of the state variables that correctly represent the crucial information for reward maximization. Our results reveal for the first time how a strategy to achieve matching behavior is beneficial to reward maximization, achieving a novel insight into the relationship between maximizing and matching.

SUBMITTER: Sakai Y

PROVIDER: S-EPMC2582656 | biostudies-other | 2008

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

When does reward maximization lead to matching law?

Sakai Yutaka Y Fukai Tomoki T

PloS one 20081124 11

What kind of strategies subjects follow in various behavioral circumstances has been a central issue in decision making. In particular, which behavioral strategy, maximizing or matching, is more fundamental to animal's decision behavior has been a matter of debate. Here, we prove that any algorithm to achieve the stationary condition for maximizing the average reward should lead to matching when it ignores the dependence of the expected outcome on subject's past choices. We may term this strateg ...[more]

PMID: 19030101

Similar Datasets

Project description:Choice behavior requires animals to evaluate both short- and long-term advantages and disadvantages of all potential alternatives. Impulsive choice is traditionally measured in laboratory tasks by utilizing delay discounting (DD), a paradigm that offers a choice between a smaller immediate reward, or a larger more delayed reward. This study tested a large sample of Heterogeneous Stock (HS) male (n = 896) and female (n = 898) rats, part of a larger genetic study, to investigate whether measures of reward maximization overlapped with traditional models of delay discounting via the patch depletion model using a Sequential Patch Depletion procedure. In this task, rats were offered a concurrent choice between two water "patches" and could elect to "stay" in the current patch or "leave" for an alternative patch. Staying in the current patch resulted in decreasing subsequent reward magnitudes, whereas the choice to leave a patch was followed by a delay and a resetting to the maximum reward magnitude. Based on the delay in a given session, different visit durations were necessary to obtain the maximum number of rewards. Visit duration may be analogous to an indifference point in traditional DD tasks. Males and females did not significantly differ on traditional measures of DD (e.g. delay gradient; AUC). When examining measures of patch utilization, females made fewer patch changes at all delays and spent more time in the patch before leaving for the alternative patch compared to males. Consistent with this, there was some evidence that females deviated from reward maximization more than males. However, when controlling for body weight, females had a higher normalized rate of reinforcement than males. Measures of reward maximization were only weakly associated with traditional DD measures and may represent distinctive underlying processes. Taken together, females performance differed from males with regard to reward maximization that were not observed utilizing traditional measures of DD, suggesting that the patch depletion model was more sensitive to modest sex differences when compared to traditional DD measures in a large sample of HS rats.

Project description:Choice behavior requires animals to evaluate both short- and long-term advantages and disadvantages of all potential alternatives. Impulsive choice is traditionally measured in laboratory tasks by utilizing delay discounting (DD), a paradigm that offers a choice between a smaller immediate reward, or a larger more delayed reward. This study tested a large sample of Heterogeneous Stock (HS) male (n = 896) and female (n = 898) rats, part of a larger genetic study, to investigate whether measures of reward maximization overlapped with traditional models of delay discounting via the patch depletion model using a Sequential Patch Depletion procedure. In this task, rats were offered a concurrent choice between two water "patches" and could elect to "stay" in the current patch or "leave" for an alternative patch. Staying in the current patch resulted in decreasing subsequent reward magnitudes, whereas the choice to leave a patch was followed by a delay and a resetting to the maximum reward magnitude. Based on the delay in a given session, different visit durations were necessary to obtain the maximum number of rewards. Visit duration may be analogous to an indifference point in traditional DD tasks. While differences in traditional DD measures (e.g., delay gradient) have been detected between males and females, these effects were small and inconsistent. However, when examining measures of reward maximization, females made fewer patch changes at all delays and spent more time in the patch before leaving for the alternative patch compared to males. This pattern of choice resulted in males having a higher rate of reinforcement than females. Consistent with this, there was some evidence that females deviated from the optimal more, leading to less reward. Measures of reward maximization were only weakly associated with traditional DD measures and may represent distinctive underlying processes. Taken together, females performance differed from males with regard to reward maximization that were not observed utilizing traditional measures of DD, suggesting that the patch depletion model was more sensitive to modest sex differences when compared to traditional DD measures in a large sample of HS rats.

Project description:OBJECTIVE:This study aims to validate the 12-lead electrocardiogram (ECG) as a biometric modality based on two straightforward binary QRS template matching characteristics. Different perspectives of the human verification problem are considered, regarding the optimal lead selection and stability over sample size, gender, age, heart rate (HR). METHODS:A clinical 12-lead resting ECG database, including a population of 460 subjects with two-session recordings (>1 year apart) is used. Cost-effective strategies for extraction of personalized QRS patterns (100ms) and binary template matching estimate similarity in the time scale (matching time) and dissimilarity in the amplitude scale (mismatch area). The two-class person verification task, taking the decision to validate or to reject the subject identity is managed by linear discriminant analysis (LDA). Non-redundant LDA models for different lead configurations (I,II,III,aVF,aVL,aVF,V1-V6) are trained on the first half of 230 subjects by stepwise feature selection until maximization of the area under the receiver operating characteristic curve (ROC AUC). The operating point on the training ROC at equal error rate (EER) is tested on the independent dataset (second half of 230 subjects) to report unbiased validation of test-ROC AUC and true verification rate (TVR = 100-EER). The test results are further evaluated in groups by sample size, gender, age, HR. RESULTS AND DISCUSSION:The optimal QRS pattern projection for single-lead ECG biometric modality is found in the frontal plane sector (60°-0°) with best (Test-AUC/TVR) for lead II (0.941/86.8%) and slight accuracy drop for -aVR (-0.017/-1.4%), I (-0.01/-1.5%). Chest ECG leads have degrading accuracy from V1 (0.885/80.6%) to V6 (0.799/71.8%). The multi-lead ECG improves verification: 6-chest (0.97/90.9%), 6-limb (0.986/94.3%), 12-leads (0.995/97.5%). The QRS pattern matching model shows stable performance for verification of 10 to 230 individuals; insignificant degradation of TVR in women by (1.2-3.6%), adults ?70 years (3.7%), younger <40 years (1.9%), HR<60bpm (1.2%), HR>90bpm (3.9%), no degradation for HR change (0 to >20bpm).

Dataset Information

When does reward maximization lead to matching law?

Publications

When does reward maximization lead to matching law?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets