Covariance Structure Regularization via Entropy Loss Function

Lin, Lijing and Higham, Nicholas J. and Pan, Jianxin (2014) Covariance Structure Regularization via Entropy Loss Function. Computational Statistics & Data Analysis, 72. pp. 315-327. ISSN 0167-9473

PDF covariance-regularization-R2_elsarticle.pdf Download (276kB)

Abstract

The need to estimate structured covariance matrices arises in a variety of applications and the problem is widely studied in statistics. A new method is proposed for regularizing the covariance structure of a given covariance matrix whose underlying structure has been blurred by random noise, particularly when the dimension of the covariance matrix is high. The regularization is made by choosing an optimal structure from an available class of covariance structures in terms of minimizing the discrepancy, defined via the entropy loss function, between the given matrix and the class. A range of potential candidate structures comprising tridiagonal Toeplitz, compound symmetry, AR(1), and banded Toeplitz is considered. It is shown that for the first three structures local or global minimizers of the discrepancy can be computed by one-dimensional optimization, while for the fourth structure Newton's method enables efficient computation of the global minimizer. Simulation studies are conducted, showing that the proposed new approach provides a reliable way to regularize covariance structures. The approach is also applied to real data analysis, demonstrating the usefulness of the proposed approach in practice.

Item Type:	Article
Uncontrolled Keywords:	Covariance estimation; Covariance structure; Entropy loss function; Kullback-Leibler divergence; Regularization.
Subjects:	MSC 2010, the AMS's Mathematics Subject Classification > 62 Statistics MSC 2010, the AMS's Mathematics Subject Classification > 65 Numerical analysis
Depositing User:	Dr Lijing Lin
Date Deposited:	07 Feb 2014
Last Modified:	20 Oct 2017 14:13
URI:	https://eprints.maths.manchester.ac.uk/id/eprint/2101

Available Versions of this Item

• Covariance Structure Regularization via Entropy Loss Function. (deposited 11 Jun 2012)
  • Covariance Structure Regularization via Entropy Loss Function. (deposited 10 Sep 2013)
    • Covariance Structure Regularization via Entropy Loss Function. (deposited 08 Oct 2013)
      • Covariance Structure Regularization via Entropy Loss Function. (deposited 07 Feb 2014) [Currently Displayed]

Actions (login required)

View Item