Tools for (Un)Conditional Forecast Dominance

A unified toolkit for out-of-sample forecast dominance testing. Covers unconditional and conditional equal and superior predictive ability, encompassing, and nested-model comparison. Implements the Diebold-Mariano test with the Harvey, Leybourne, and Newbold (1997) small-sample correction; the Clark-West MSFE-adjusted statistic (Clark and West, 2007) ; the ENC-NEW encompassing test of Clark and McCracken (2001) ; the Giacomini-White conditional equal predictive ability test (Giacomini and White, 2006) ; Hansen's superior predictive ability test (Hansen, 2005) ; the conditional superior predictive ability test of Li, Liao, and Quaedvlieg (2022) ; and the uniform and average multi-horizon SPA tests of Quaedvlieg (2021) .

forecastdom

Author: Gabriel Cabrera
License: MIT + file LICENSE

Overview

forecastdom is an R toolkit for comparing the predictive ability of forecasting methods. It implements the four cells of the Li, Liao, and Quaedvlieg (2022) taxonomy (equal vs. superior, unconditional vs. conditional) plus tests for nested models, forecast encompassing, return predictability, and parameter instability.

Installation

# install.packages("devtools")
devtools::install_github("gabbocg/forecastdom")

Tests

Forecast Comparison

Predictive Regression & Parameter Instability

Usage

Pairwise forecast comparison

library(forecastdom)

# Diebold-Mariano test with HLN correction
e1 <- rnorm(200)
e2 <- rnorm(200, mean = 0.1)
dm_test(e1, e2)

Conditional Superior Predictive Ability

# Simulate data from LLQ (2022) DGP
sim <- do_sim(J = 3, n = 500, a = 1.5, c = 0, rho_u = 0.4)

# CSPA test
result <- cspa_test(sim$Y, sim$X, level = 0.05, trim = 2)
result

# Visualization
cspa_test_plot(result)

Confidence Set for the Most Superior

# Compare multiple methods symmetrically
csms(losses, X, level = 0.05, trim = 2, method_names = c("AR1", "HAR", "HARQ", "LASSO"))

IVX-Wald test for return predictability

ivx_wald(returns, predictors, K = 1, M_n = floor(T ^ (1 / 3)))

Taxonomy

The four cells in Li, Liao, and Quaedvlieg (2022):

Equal vs. superior asks whether forecasts have the same loss or whether one is strictly lower. Unconditional vs. conditional asks whether the comparison holds on average or holds at every value of a conditioning variable.

Performance

The CSPA test uses Rcpp / C++ for the two hot loops (Gaussian-process column maxima and the binary search over the p-value).

Getting help

If you encounter a bug, please file an issue with a minimal reproducible example on GitHub. For questions, email [email protected].

References

• Clark, T.E. and McCracken, M.W. (2001). Tests of Equal Forecast Accuracy and Encompassing for Nested Models. Journal of Econometrics, 105(1), 85-110.
• Clark, T.E. and West, K.D. (2007). Approximately Normal Tests for Equal Predictive Accuracy in Nested Models. Journal of Econometrics, 138(1), 291-311.
• Diebold, F.X. and Mariano, R.S. (1995). Comparing Predictive Accuracy. Journal of Business & Economic Statistics, 13(3), 253-263.
• Elliott, G. and Muller, U.K. (2006). Efficient Tests for General Persistent Time Variation in Regression Coefficients. Review of Economic Studies, 73(4), 907-940.
• Giacomini, R. and White, H. (2006). Tests of Conditional Predictive Ability. Econometrica, 74(6), 1545-1578.
• Hansen, P.R. (2005). A Test for Superior Predictive Ability. Journal of Business & Economic Statistics, 23(4), 365-380.
• Harvey, D., Leybourne, S., and Newbold, P. (1997). Testing the Equality of Prediction Mean Squared Errors. International Journal of Forecasting, 13(2), 281-291.
• Kostakis, A., Magdalinos, T., and Stamatogiannis, M.P. (2015). Robust Econometric Inference for Stock Return Predictability. Review of Financial Studies, 28(5), 1506-1553.
• Li, J., Liao, Z., and Quaedvlieg, R. (2022). Conditional Superior Predictive Ability. Review of Economic Studies, 89(2), 843-875.
• McCracken, M.W. (2007). Asymptotics for Out of Sample Tests of Granger Causality. Journal of Econometrics, 140(2), 719-752.
• Quaedvlieg, R. (2021). Multi-Horizon Forecast Comparison. Journal of Business & Economic Statistics, 39(1), 40-53.