Fractional ridge regression: a fast, interpretable reparameterization of ridge regression.

ABSTRACT: Background: Ridge regression is a regularization technique that penalizes the L2-norm of the coefficients in linear regression. One of the challenges of using ridge regression is the need to set a hyperparameter (?) that controls the amount of regularization. Cross-validation is typically used to select the best ? from a set of candidates. However, efficient and appropriate selection of ? can be challenging. This becomes prohibitive when large amounts of data are analyzed. Because the selected ? depends on the scale of the data and correlations across predictors, it is also not straightforwardly interpretable.

Results: The present work addresses these challenges through a novel approach to ridge regression. We propose to reparameterize ridge regression in terms of the ratio ? between the L2-norms of the regularized and unregularized coefficients. We provide an algorithm that efficiently implements this approach, called fractional ridge regression, as well as open-source software implementations in Python and matlab (https://github.com/nrdg/fracridge). We show that the proposed method is fast and scalable for large-scale data problems. In brain imaging data, we demonstrate that this approach delivers results that are straightforward to interpret and compare across models and datasets.

Conclusion: Fractional ridge regression has several benefits: the solutions obtained for different ? are guaranteed to vary, guarding against wasted calculations; and automatically span the relevant range of regularization, avoiding the need for arduous manual exploration. These properties make fractional ridge regression particularly suitable for analysis of large complex datasets.

SUBMITTER: Rokem A 

PROVIDER: S-EPMC7702219 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature