On robust regression with high-dimensional predictors.

ABSTRACT: We study regression M-estimates in the setting where p, the number of covariates, and n, the number of observations, are both large, but p ? n. We find an exact stochastic representation for the distribution of ? = argmin(???(p)) ?(i=1)(n) ?(Y(i) - X(i')?) at fixed p and n under various assumptions on the objective function ? and our statistical model. A scalar random variable whose deterministic limit r?(?) can be studied when p/n ? ? > 0 plays a central role in this representation. We discover a nonlinear system of two deterministic equations that characterizes r?(?). Interestingly, the system shows that r?(?) depends on ? through proximal mappings of ? as well as various aspects of the statistical model underlying our study. Several surprising results emerge. In particular, we show that, when p/n is large enough, least squares becomes preferable to least absolute deviations for double-exponential errors.

SUBMITTER: El Karoui N 

PROVIDER: S-EPMC3767561 | biostudies-literature | 2013 Sep

REPOSITORIES: biostudies-literature