Groupwisely Sparse Penalty for Highly Correlated Covariates

Abstract

This paper considers a problem of model selection and estimation in sparse, high-dimensional regression models where covariates are grouped. We propose a new regularization method which can reflect a correlation structure between groups. We propose a combination of group MCP penalty and groupwise quadratic penalty called groupwise weighted ridge penalty. The former ensures groupwise sparsity and the later promotes simultaneous selection of highly correlated groups. We show that this estimator satisfes the group selection consistency. We drive the optimization algorithm for the proposed method. Numerical studies show that this estimator appears to perform well in comparisons with other existing methods.