Efficient Estimation of Covariance and (Partial) Correlation

Implements a James-Stein-type shrinkage estimator for the covariance matrix, with separate shrinkage for variances and correlations. The details of the method are explained in Schafer and Strimmer (2005) and Opgen-Rhein and Strimmer (2007) . The approach is both computationally as well as statistically very efficient, it is applicable to "small n, large p" data, and always returns a positive definite and well-conditioned covariance matrix. In addition to inferring the covariance matrix the package also provides shrinkage estimators for partial correlations and partial variances. The inverse of the covariance and correlation matrix can be efficiently computed, as well as any arbitrary power of the shrinkage correlation matrix. Furthermore, functions are available for fast singular value decomposition, for computing the pseudoinverse, and for checking the rank and positive definiteness of a matrix.


News

       RELEASE HISTORY OF THE "corpcor" PACKAGE

Version 1.6.9

  • fixed a bug that made it possible that in very rare cases the shrinkage intensity can become negative
  • DOIs for all references added

Version 1.6.8

  • import statements added required by R-devel

Version 1.6.7

  • change of maintainer email address

Version 1.6.6

  • is.positive.definite() and make.positive.definite() now use eigen() with option symmetric=TRUE, so that it works under R 3.0.0 (thanks to Roberto Ugoccioni for reporting this issue)

Version 1.6.5

  • License file removed
  • Dependencies updated

Version 1.6.4

  • estimate.lambda() now also works if the input data matrix has two columns (thanks to Manuela Hummel for reporting this bug)

Version 1.6.3

  • substantially faster estimate.lambda() for small sample size and large dimensions (new algorithm suggested by Miika Ahdesm"aki)

Version 1.6.2

  • new public functions estimate.lambda() and estimate.lambda.var() returning the shrinkage intensities for correlation and variance shrinkage
  • some internal code reorganization

Version 1.6.1

  • NAMESPACE file added
  • small changes in de documentation

Version 1.6.0

  • collapse option removed from: powcor.shrink, cor.shrink, invcor.shrink, cov.shrink, invcov.shrink
  • default minimum R version changed to 2.10.0
  • new function crossprod.powcor.shrink() to efficiently compute R_shrink^alpha y without the need to explicitly compute R_shrink^alpha

Version 1.5.7

  • the function mvr.shrink() has been removed (for a related function see the "care" R package).
  • mpower() now keeps row and column names.

Version 1.5.6

  • cor.shrink() now also works if the input data matrix only has a single column.

Version 1.5.5

  • a numerical problem with symmetry check in invcor.shrink() and related functions has been resolved (thanks to Gad Abraham).
  • following an idea by Miika Ahdesm"aki the procedures for estimating the shrinkage intensities have been rewritten in pure R without employing any C code any more (w/o speed penalty!).

Version 1.5.4

  • mpower() now has an option to exclude nonzero eigenvalues and also checks whether the input matrix is symmetric.

Version 1.5.3

  • small corrections to pass checks for R version 2.10.
  • reference to Zuber and Strimmer (2009) added.

Version 1.5.2

  • small corrections in the help pages, to pass the more stringent checks on .Rd files introduced in R in January 2009.

Version 1.5.1

  • powcor.shrink() now collapses the identity matrix if alpha=0 and collapse=TRUE.
  • help page for powcor.shrink was revised.
  • package description was also revised.

Version 1.5.0

  • new function powcor.shrink() computes (very efficiently!) an arbitrary power of the correlation shrinkage matrix (i.e. R_shrink^alpha).
  • invcor.shrink() is now a special case powcor.shrink() with alpha=-1.
  • invcov.shrink() now also uses powcor.shrink().
  • new mpower() utility function to estimate the matrix power of a real symmetric matrix.

Version 1.4.8

  • new "collapse" option in cor.shrink, cov.shrink, invcor.shrink, invov.shrink to allow memory savings when lambda equals 1.
  • to simplify code base two rarely used options were removed: "protect" and "scale.by" (wt.scale is now always done using "sd").
  • package depends now on R 2.7.0.
  • documentation was polished.

Version 1.4.7

  • change of license from "GPL 2 or later", to "GPL 3 or later".

Version 1.4.6

  • following a suggestion (and a patch) by Nicola Soranzo internal big objects are now explicitly removed when they are not needed anymore.
    As a result, the package now needs less computer memory, and larger (partial) correlation matrices can be computed.

Version 1.4.5

  • when partial correlations are computed using pcor.shrink() the returned matrix now has the standardized partial variances (i.e. PVAR/VAR) attached under the attribute "spv".
  • updated references in the help pages

Version 1.4.4

  • the function wt.scale() is now much faster, especially for large p, due to using colSums() rather than apply() .. note that this indirectly speeds up most other functions in the corpcor package!
  • typos in the documtation were corrected and references updated

Version 1.4.3

  • the shrinkage target for the variance is now the median (previously, variances were shrunken towards the mean).
  • var.shrink now also has a"protect" argument.
  • limited translation shrinkage is now turned off by default (i.e. protect has value zero).

Version 1.4.2

  • limited translation estimator implemented for the shrinkage estimate of the correlation matrix. This prevents excessive component risk.
  • new functions for decomposing the covariance matrix and its inverse: decompose.cov(), decompose.invcov(), rebuild.invcov()
  • new function pvar.shrink() to estimate partial variance.
  • in the documentation the definition of partial variance and partial covariance are corrected (following Whittaker 1990)
  • the functions cov2pcov(), pcov2cov(), pcov.shrink() have been removed.
  • functions sm2vec(), vec2sm(), sm.index() back (from GeneTS)
  • is.positive.definite() checks for complex eigenvalues.

Version 1.4.1:

  • fast.svd() now doesn't use any more the LAPACK routines DGESVD to compute the singular value decomposition (this routine is deprecated from R version 2.3.0)
  • weighted.var(), weighted.moments(), weighted.scale() are now called wt.var(), wt.moments(), wt.scale()

Version 1.4.0:

  • New functions mvr.shrink() and mvr.predict() for multivariate shrinkage regression.
  • All shrinkage estimate now carry the class attribute "shrinkage. This allows for a more informative output via print.shrinkage()
  • Removed functions: sm2vec(), vec2sm(), sm.indexes()

Version 1.3.1:

  • This versions fixes a bug present in "corpcor" version 1.3.0 and 1.2.0 but not in earlier versions. This bug leads to a (probably negligible) small bias in the computation of the optimal shrinkage intensity.
  • The functions cov.bagged(), cor.bagged(), and pcor.bagged() have been removed.
  • Typographical errors in the documentation were corrected.

Version 1.3.0:

  • New function "var.shrink" to compute shrinkage estimates of variances (target: average empirical variances.
  • cov.shrink() and pcov.shrink() are now also based on shrunken variances.
  • Estimation of shrinkage intensities are now done in C. This greatly decreases the computational costs.
  • Options "check.eigenvalues" and "exact.inversion" have been removed in cor2pcor() and pcor2cor()
  • The functions have been modified so that data sets with zero-variance variables may also be analyzed (these will be in effect ignored in estimating correlation but taken into account when estimating variances).

Version 1.2.0:

  • Greatly reduced memory and faster computations.
  • New code on fast inversion using Woodbury identity.
  • Consequently, pcor.shrink() is now much faster .
  • New functions for computation of weighted variances, weighted moments, and weighted rescaling.
  • All covariance etc. estimators now also have the option to supply data weights".
  • varcov() function removed (not necessary any more).
  • Several parts of documentation updated.
  • Juliane's Web address updated.

Version 1.1.2:

  • Minor typos in documentation corrected.
  • From this version is.positive.definite() works with arbitrary matrix (previously it required symmetric matrix).

Version 1.1.1:

  • Reference to shrinkage covariance paper is updated.

Version 1.1:

  • cor.shrink() is now the central estimator, and cov.shrink is derived.

Version 1.0:

  • First stand-alone release (20 August 2005). This package contains various functions shrinkage estimation of (partial) correlation and covariance. Prior to release in this package the functions were part of the GeneTS package.

Reference manual

It appears you don't have a PDF plugin for this browser. You can click here to download the reference manual.

install.packages("corpcor")

1.6.9 by Korbinian Strimmer, 4 years ago


http://strimmerlab.org/software/corpcor/


Browse source code at https://github.com/cran/corpcor


Authors: Juliane Schafer , Rainer Opgen-Rhein , Verena Zuber , Miika Ahdesmaki , A. Pedro Duarte Silva , and Korbinian Strimmer.


Documentation:   PDF Manual  


Task views: Multivariate Statistics


GPL (>= 3) license


Imports stats


Imported by BNrich, BinOrdNonNor, CopulaCenR, ERP, FADA, FoReco, GGMnonreg, GWpcor, HSPOR, IRCcheck, MCMCglmm, MetaIntegration, MultiVarMI, NetOrigin, REndo, RMTL, RNGforGPD, RPANDA, SAMtool, SCCS, SLBDD, SimTimeVar, StabilizedRegression, VARshrink, biospear, bootnet, boral, cape, carSurv, cate, clustrd, corHMM, covequal, deepgmm, dnapath, esaBcv, fspe, gwpcormapper, iTOP, idm, jackstraw, knockoff, list, lvm4net, lvnet, lymphclon, mlVAR, netassoc, nowcasting, parma, pcalg, pcev, penalizedSVM, perARMA, pfica, pivmet, psychonetrics, qgraph, r4ss, ratematrix, relaimpo, rmgarch, seedCCA, segMGarch, semPlot, sparsevar, spduration, subselect, ttScreening, vampyr, varTestnlme, wux, xLLiM.

Depended on by BinNonNor, BinNor, COMBAT, EDISON, FPCA2D, FRCC, GLSME, GeneNet, HPbayes, Hotelling, MAVTgsa, MultiOrd, OUwie, OrdNor, PoisBinNonNor, PoisBinOrd, PoisBinOrdNonNor, PoisBinOrdNor, PoisNonNor, PoisNor, SMFI5, care, kcirt, leapp, longitudinal, metavcov, mgpd, miRtest, mvMORPH, qtlhot, sda, sideChannelAttack, splitFeas, st, whitening.

Suggested by BLCOP, EGAnet, NMF, PortfolioAnalytics, picante.


See at CRAN