Weighted Mnet Penalty for Twice Differentiable Convex Losses on High Dimensions

Abstract

In many regression problems, covariates can be naturally correlated. Kim and Jeon [2014] proposed weighted Mnet penalty which is defined combination of weighted minimax concave penalty(MCP) and weighted ridge penalty. They showed that the weighted Mnet penalty is useful to squared loss when the covariates of correlations are highly correlated. They also point out that the weighted l2 penalty is equivalent to the Laplacian penalty with certain weights and the weighted Mnet estimator has the oracle property to the squared loss under regular conditions. We extend the weighted Mnet estimator to twice differentiable convex losses. We showed that the weighted l2 penalty to twice differentiable convex losses also can be equivalent to the Laplacian penalty with certain weights and the weighted Mnet has an oracle property on high dimensional model in the sense that it is equal to the oracle ridge estimator with high probability. By simulations and real data analysis, we show that the weighted Mnet penalty is a useful to the other competitors including the elastic net, the Ment and the sparse Laplacian penalty.