Gaussian Mixture Proxies for Densities and Characteristic Functions

Fits multivariate Gaussian-mixture proxies to target distributions on real Euclidean space, given by a density, by samples, or by a characteristic function. Four fitting regimes are unified under one verb: (i) closed-form moment matching for a single component, (ii) classical expectation-maximisation when independent samples are available, (iii) importance-sampled expectation-maximisation that minimises the Kullback-Leibler divergence when the target can be evaluated point-wise but not (cheaply) sampled, and (iv) a least-squares fit to the characteristic function when only that is known, with numerical bounds on the Kolmogorov distance along one-dimensional projections. Closed-form Gaussian-mixture operators (density, sampling, marginalisation, conditioning, divergence) round out the toolkit. The conditioning operator drives multiple imputation of data missing at random, covering the multimodal and heteroscedastic cases a single-Gaussian model cannot represent. Implements the regime hierarchy of van der Hoek and Elliott (2024) .

proxymix

proxymix fits multivariate Gaussian-mixture proxies to target distributions on $\mathbb{R}^p$. Three regimes from van der Hoek and Elliott (2024) fit a density or samples and are Kullback–Leibler optimal. A fourth fits a distribution known only through its characteristic function. All four are unified under one verb:

Regime (iii) is the reason the package exists. The sample-based mixture packages (mclust, mixtools, flexmix) all assume i.i.d. draws from the target; proxymix fits directly against an evaluable (possibly unnormalised) log-density. The nearest CRAN neighbour is AdMit, which adaptively fits a mixture of Student-t distributions to an evaluable kernel as an importance/proposal density; proxymix differs in fitting a Gaussian mixture that is Kullback–Leibler optimal, precisely so that the fitted object then supports the closed-form operator calculus (marginals, conditionals, Bayes updates, products, convolutions, filtering) and carries a fit-quality certificate through every operation.

Why not MCMC? If you can evaluate the unnormalised density you can always run a sampler and then fit a mixture to the draws. The regime-(iii) fit is the shortcut when what you want is the compact closed-form object: no chain tuning or convergence diagnostics, a deterministic pipeline given the seed, and a mixture whose marginals, conditionals, moments and samples are then available in closed form through the operator calculus. The trade-off is dimension: the importance sampling that drives regime (iii) loses effective sample size sharply beyond roughly $p = 5$–$10$, and every fit reports its effective sample size so that limit is visible rather than silent. For high-dimensional posteriors, sample with your favourite MCMC and use regime (ii) on the draws.

Related packages

• mclust fits Gaussian finite mixtures by EM for model-based clustering, classification and density estimation.
• mixtools analyses finite mixture models in parametric and semiparametric settings, including mixtures of regressions, with tools for choosing the number of components.
• AdMit fits an adaptive mixture of Student-t distributions to a target density through its kernel, for use as an importance density or a Metropolis-Hastings candidate.

mclust and mixtools fit to draws from the target. proxymix also fits to a density it can only evaluate, and its Gaussian components keep marginals, conditionals and Bayes updates in closed form.

Installation

From CRAN:

install.packages("proxymix")

The development version from GitHub:

# install.packages("remotes")
remotes::install_github("max578/proxymix")

or locally from the source tree:

R CMD build proxymix
R CMD INSTALL proxymix_*.tar.gz

Documentation site: https://max578.github.io/proxymix/.

Quick start

library(proxymix)

## A target you can evaluate but not sample from -- a 2D "banana".
banana <- banana_target()

## Fit a 3-component Gaussian mixture proxy via KLD-EM with importance sampling.
fit <- fit_proxymix(banana, N = 3L, regime = "kld",
                    proposal = proposal_mvt(n_dim = 2L, df = 5),
                    is_size = 2000L, max_iter = 60L, seed = 1L)

print(fit)
#> <gmm_fit>: regime = "kld", K = 3, p = 2
#>   target     : banana
#>   iterations : 37
#>   converged  : TRUE
#>   [1] w = 0.6456, |mu| = 0.3299, tr(Sigma) = 1.3601
#>   [2] w = 0.2473, |mu| = 1.2996, tr(Sigma) = 2.2312
#>   [3] w = 0.1071, |mu| = 1.8882, tr(Sigma) = 4.9044

## Closed-form operations on the fitted mixture.
gmm_marginalise(fit, keep = 1L)
#> <marginalise(kld_em[N=3] on banana)>: K = 3 components in p = 1 dimensions
#>   [1] w = 0.6456, |mu| = 0.1818, tr(Sigma) = 0.4620
#>   [2] w = 0.2473, |mu| = 1.1518, tr(Sigma) = 0.4846
#>   [3] w = 0.1071, |mu| = 1.5954, tr(Sigma) = 0.6321
gmm_conditionalise(fit, given = c(NA, 0.5))
#> <conditionalise(kld_em[N=3] on banana)>: K = 3 components in p = 1 dimensions
#>   [1] w = 0.6730, |mu| = 0.2620, tr(Sigma) = 0.4524
#>   [2] w = 0.2576, |mu| = 1.1133, tr(Sigma) = 0.2352
#>   [3] w = 0.0694, |mu| = 1.4188, tr(Sigma) = 0.1199

A distribution known only through its characteristic function

## The symmetric stable law of index 1.5 has no closed-form density
## and infinite variance, but its characteristic function is exp(-|t|^1.5).
cf_stable <- function(t) exp(-abs(t[, 1])^1.5)
fit_stable <- from_cf_elliptical(cf_stable, n_dim = 1L, seed = 1L)
fit_stable
#> <gmm_fit>: regime = "cf", K = 10, p = 1
#>   target     : <none>
#>   iterations : NA
#>   converged  : TRUE
#>   cf held-out RMS error : 0.000557
#>   Kolmogorov distance   : at least 0.000245, at most 0.00509 (1 direction)
#>   [1] w = 0.5595, |mu| = 0.0000, tr(Sigma) = 1.3211
#>   [2] w = 0.2645, |mu| = 0.0000, tr(Sigma) = 3.7563
#>   [3] w = 0.0618, |mu| = 0.0000, tr(Sigma) = 10.6806
#>   [4] w = 0.0500, |mu| = 0.0000, tr(Sigma) = 30.3689
#>   [5] w = 0.0478, |mu| = 0.0000, tr(Sigma) = 0.4646
#>   ... 5 more components

## Quantiles, tail probabilities and draws come from the mixture.
qgmm(c(0.95, 0.99), fit_stable)
#> [1] 3.048049 7.752634

The printed bounds say how far the distribution function of the mixture can be from that of the stable law. The vignette on characteristic functions checks them against an exact computation.

The unified fitting verb

fit_proxymix(
  target,
  N        = 1L,
  regime   = c("auto", "moment", "sample", "kld", "cf"),
  ...
)

• target is an S7 gmm_target produced by gmm_target() (from a log-density), gmm_target_from_samples() (from samples), cf_target() (from a characteristic function), or one of the built-in factories (banana_target(), donut_target(), mixture_target()).
• regime = "auto" picks the cheapest applicable regime from the target’s contents.

Closed-form mixture operators

dgmm(x, fit)              # density evaluation
rgmm(n, fit)              # exact sampling
gmm_marginalise(fit, keep)
gmm_conditionalise(fit, given)
gmm_kld(fit_p, fit_q)     # Monte Carlo + variational bounds

Mapping the optima of an objective

fit   <- from_objective(f, lower, upper, N = 10L)  # a mixture map of the optima
gmm_modes(fit)$modes                               # the distinct optima

from_objective() treats an objective f as the Gibbs measure exp(-f / T) – a regime-(iii) target you can evaluate but not sample – and returns a closed-form mixture over its low regions, so a multimodal f is recovered as a whole rather than one optimum at a time. gmm_modes() resolves the fitted map into the recovered optima.

Which function do I need?

Vignettes

The package ships with thirteen vignettes:

• quickstart – one-page tour.
• posterior_proxy – the flagship workflow: a real unnormalised Bayesian posterior compressed to a proxy, with the evidence, closed-form reads, and bootstrap error bars.
• three_regimes – a walk-through of regimes (i)–(iii) on toy 2-D targets, including the agreement of (i) and (iii) at $N = 1$.
• density_shapes – the regime-(iii) demonstration: banana, donut, three-mixture targets fit by importance-sampled KLD-EM.
• cf_targets – regime (iv): a stable law, a sum of independent variables, two-dimensional targets and a filtering step, each fitted from its characteristic function and checked against an independent computation.
• operator_calculus – closed-form pushforward, Bayesian update, aggregation and conditioning on a fitted mixture.
• from_kde – compressing a kernel density estimate into a Gaussian-mixture proxy.
• many_methods – one fitted mixture in place of regression, clustering, PCA and ridge regression.
• entropy – closed-form entropy, divergence and mutual-information diagnostics.
• calibration – mapping the optima of an objective via its Gibbs measure.
• missing_data – multiple imputation by conditioning the fitted mixture (missing at random).
• missing_data_mnar – imputation under value-dependent missingness and censoring, with sensitivity analysis.
• end_of_sample – testing whether the last few observations of a series are consistent with a fitted state-space model.

Interactive tutorial

A standalone, single-file HTML primer for proxymix is hosted at

https://max578.github.io/proxymix-tutorial/

The page is targeted at Adelaide University Mathematics and Statistics undergraduates (Guided depth, default) and postgraduate / PhD readers (Technical depth, via the in-page dial). It runs in the browser with no R install required, and works the same way offline by double-click. Source at max578/proxymix-tutorial.

Reference

Hoek, J. van der and Elliott, R. J. (2024). Mixtures of multivariate Gaussians. Stochastic Analysis and Applications. doi:10.1080/07362994.2024.2372605.

Contributing

Bug reports and suggestions are welcome through the GitHub issues.

Citation

citation("proxymix")

Licence

MIT © Max Moldovan.