Positive-definite correction of covariance matrix estimators via linear shrinkage

Abstract

In this paper, we study the positive definiteness (PDness) problem in covariance matrix estimation. For high dimensional data, the most common sample covariance matrix performs poorly in estimating the true matrix. Recently, as an alternative to the sample covariance matrix, many regularized estimators are proposed under structural assumptions on the true including sparsity. They are shown to be asymptotically consistent and rate-optimal in estimating the true covariance matrix and its structure. However, many of them do not take into account the PDness of the estimator and produce a non-PD estimate. Otherwise, additional regularizations (or constraints) are required on eigenvalues which make both the asymptotic analysis and computation much harder. To achieve the PDness, we propose a simple one-step procedure to update the regularized covariance matrix estimator which is not necessarily PD in finite sample. We revisit the idea of linear shrinkage (Stein, 1956

Ledoit and Wolf, 2004) and propose to take a convex combination between the first stage covariance matrix estimator (the regularized covariance matrix without PDness) and a given form of diagonal matrix. The proposed one-step correction, which we denote as LSPD (linear shrinkage for positive definiteness) estimator, is shown to preserve the asymptotic properties of the first stage estimator if the shrinkage parameters are carefully selected. In addition, it has a closed form expression and its computation is optimization-free, unlike existing sparse PD estimators (Rothman, 2012

Xue et al., 2012). The LSPD estimator is numerically compared with other sparse PD estimators to understand its finite sample properties as well as its computational gain. Finally, it is applied to two multivariate procedures relying on the covariance matrix estimator - the linear minimax classification problem posed by Lanckriet et al. (2002) and the well-known mean-variance portfolio optimization problem - and is shown to substantially improve the performance of both procedures.