Unknown

Dataset Information

0

Complexity control by gradient descent in deep networks.

ABSTRACT: Overparametrized deep networks predict well, despite the lack of an explicit complexity control during training, such as an explicit regularization term. For exponential-type loss functions, we solve this puzzle by showing an effective regularization effect of gradient descent in terms of the normalized weights that are relevant for classification.

SUBMITTER: Poggio T 

PROVIDER: S-EPMC7039878 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

Json Xml
altmetric image

Publications

Complexity control by gradient descent in deep networks.

Poggio Tomaso T   Liao Qianli Q   Banburski Andrzej A  

Nature communications 20200224 1

Overparametrized deep networks predict well, despite the lack of an explicit complexity control during training, such as an explicit regularization term. For exponential-type loss functions, we solve this puzzle by showing an effective regularization effect of gradient descent in terms of the normalized weights that are relevant for classification. ...[more]