Efficient Estimation of Covariance and (Partial) Correlation
Implements a James-Stein-type shrinkage estimator for
the covariance matrix, with separate shrinkage for variances and correlations.
The details of the method are explained in Schafer and Strimmer (2005)
<10.2202> and Opgen-Rhein and Strimmer (2007)
<10.2202>. The approach is both computationally as well
as statistically very efficient, it is applicable to "small n, large p" data,
and always returns a positive definite and well-conditioned covariance matrix.
In addition to inferring the covariance matrix the package also provides
shrinkage estimators for partial correlations and partial variances.
The inverse of the covariance and correlation matrix
can be efficiently computed, as well as any arbitrary power of the
shrinkage correlation matrix. Furthermore, functions are available for fast
singular value decomposition, for computing the pseudoinverse, and for
checking the rank and positive definiteness of a matrix.10.2202>10.2202>
RELEASE HISTORY OF THE "corpcor" PACKAGE
- fixed a bug that made it possible that in very rare cases the shrinkage
intensity can become negative
- DOIs for all references added
- import statements added required by R-devel
- change of maintainer email address
- is.positive.definite() and make.positive.definite() now use eigen()
with option symmetric=TRUE, so that it works under R 3.0.0
(thanks to Roberto Ugoccioni for reporting this issue)
- License file removed
- Dependencies updated
- estimate.lambda() now also works if the input data matrix has
two columns (thanks to Manuela Hummel for reporting this bug)
- substantially faster estimate.lambda() for small sample size
and large dimensions (new algorithm suggested by Miika Ahdesm"aki)
- new public functions estimate.lambda() and estimate.lambda.var()
returning the shrinkage intensities for correlation and variance shrinkage
- some internal code reorganization
- NAMESPACE file added
- small changes in de documentation
- collapse option removed from:
powcor.shrink, cor.shrink, invcor.shrink, cov.shrink, invcov.shrink
- default minimum R version changed to 2.10.0
- new function crossprod.powcor.shrink() to efficiently compute
R_shrink^alpha y without the need to explicitly compute R_shrink^alpha
- the function mvr.shrink() has been removed (for a related function see the
"care" R package).
- mpower() now keeps row and column names.
- cor.shrink() now also works if the input data matrix only has a
- a numerical problem with symmetry check in invcor.shrink() and related
functions has been resolved (thanks to Gad Abraham).
- following an idea by Miika Ahdesm"aki the procedures for estimating the
shrinkage intensities have been rewritten in pure R without employing
any C code any more (w/o speed penalty!).
- mpower() now has an option to exclude nonzero eigenvalues
and also checks whether the input matrix is symmetric.
- small corrections to pass checks for R version 2.10.
- reference to Zuber and Strimmer (2009) added.
- small corrections in the help pages, to pass the more stringent
checks on .Rd files introduced in R in January 2009.
- powcor.shrink() now collapses the identity matrix if alpha=0 and collapse=TRUE.
- help page for powcor.shrink was revised.
- package description was also revised.
- new function powcor.shrink() computes (very efficiently!) an arbitrary
power of the correlation shrinkage matrix (i.e. R_shrink^alpha).
- invcor.shrink() is now a special case powcor.shrink() with alpha=-1.
- invcov.shrink() now also uses powcor.shrink().
- new mpower() utility function to estimate the matrix power
of a real symmetric matrix.
- new "collapse" option in cor.shrink, cov.shrink, invcor.shrink, invov.shrink
to allow memory savings when lambda equals 1.
- to simplify code base two rarely used options were removed:
"protect" and "scale.by" (wt.scale is now always done using "sd").
- package depends now on R 2.7.0.
- documentation was polished.
- change of license from "GPL 2 or later", to "GPL 3 or later".
- following a suggestion (and a patch) by Nicola Soranzo internal big
objects are now explicitly removed when they are not needed anymore.
As a result, the package now needs less computer memory, and
larger (partial) correlation matrices can be computed.
- when partial correlations are computed using pcor.shrink() the
returned matrix now has the standardized partial variances (i.e.
PVAR/VAR) attached under the attribute "spv".
- updated references in the help pages
- the function wt.scale() is now much faster, especially for large p,
due to using colSums() rather than apply() .. note that this indirectly
speeds up most other functions in the corpcor package!
- typos in the documtation were corrected and references updated
- the shrinkage target for the variance is now the median (previously,
variances were shrunken towards the mean).
- var.shrink now also has a"protect" argument.
- limited translation shrinkage is now turned off by default
(i.e. protect has value zero).
- limited translation estimator implemented for the shrinkage estimate
of the correlation matrix. This prevents excessive component risk.
- new functions for decomposing the covariance matrix and
its inverse: decompose.cov(), decompose.invcov(), rebuild.invcov()
- new function pvar.shrink() to estimate partial variance.
- in the documentation the definition of partial variance and
partial covariance are corrected (following Whittaker 1990)
- the functions cov2pcov(), pcov2cov(), pcov.shrink() have been removed.
- functions sm2vec(), vec2sm(), sm.index() back (from GeneTS)
- is.positive.definite() checks for complex eigenvalues.
- fast.svd() now doesn't use any more the LAPACK routines DGESVD
to compute the singular value decomposition (this routine is deprecated
from R version 2.3.0)
- weighted.var(), weighted.moments(), weighted.scale() are
now called wt.var(), wt.moments(), wt.scale()
- New functions mvr.shrink() and mvr.predict() for multivariate
- All shrinkage estimate now carry the class attribute "shrinkage.
This allows for a more informative output via print.shrinkage()
- Removed functions: sm2vec(), vec2sm(), sm.indexes()
- This versions fixes a bug present in "corpcor" version 1.3.0
and 1.2.0 but not in earlier versions. This bug leads to a
(probably negligible) small bias in the computation of the
optimal shrinkage intensity.
- The functions cov.bagged(), cor.bagged(), and pcor.bagged()
have been removed.
- Typographical errors in the documentation were corrected.
- New function "var.shrink" to compute shrinkage estimates of
variances (target: average empirical variances.
- cov.shrink() and pcov.shrink() are now also based on shrunken
- Estimation of shrinkage intensities are now done in C.
This greatly decreases the computational costs.
- Options "check.eigenvalues" and "exact.inversion" have been
removed in cor2pcor() and pcor2cor()
- The functions have been modified so that data sets with
zero-variance variables may also be analyzed (these will be in
effect ignored in estimating correlation but taken into account
when estimating variances).
- Greatly reduced memory and faster computations.
- New code on fast inversion using Woodbury identity.
- Consequently, pcor.shrink() is now much faster .
- New functions for computation of weighted variances,
weighted moments, and weighted rescaling.
- All covariance etc. estimators now also have the option to
supply data weights".
- varcov() function removed (not necessary any more).
- Several parts of documentation updated.
- Juliane's Web address updated.
- Minor typos in documentation corrected.
- From this version is.positive.definite() works with
arbitrary matrix (previously it required symmetric matrix).
- Reference to shrinkage covariance paper is updated.
- cor.shrink() is now the central estimator, and cov.shrink
- First stand-alone release (20 August 2005).
This package contains various functions shrinkage estimation
of (partial) correlation and covariance.
Prior to release in this package the functions were part
of the GeneTS package.