statsmodels.tsa.stattools.clark_west_test#

statsmodels.tsa.stattools.clark_west_test(y, forecast_restricted, forecast_unrestricted, *, lags=None, horizon=1, kernel='bartlett')[source]#

Clark-West test of equal predictive accuracy of nested models.

Added in version 0.15.1.

Parameters:
yarray_like

The observed values of the forecast variable. The array is 1-D; a column with shape (nobs, 1) is also accepted.

forecast_restrictedarray_like

The forecasts of y from the restricted model, for example the historical mean or a random walk. The array is 1-D; a column with shape (nobs, 1) is also accepted.

forecast_unrestrictedarray_like

The forecasts of y from the unrestricted model, which nests the restricted model. The array is 1-D; a column with shape (nobs, 1) is also accepted.

lagsint, optional

The number of lags to include in the HAC variance estimator used for the adjusted loss differential. Must be non-negative if provided. If not provided, max(horizon - 1, ceil(nobs ** (1/3))) is used with the Bartlett kernel and horizon - 1 with the uniform kernel, as in diebold_mariano_test. Must be smaller than nobs, and smaller than nobs - 1 with the uniform kernel.

horizonint, optional

The forecast horizon, used to form the default number of lags. Must be a positive integer. Must be smaller than nobs if lags is not provided, and is otherwise not used. Default is 1.

kernel{“bartlett”, “uniform”}, optional

The kernel of the HAC estimator. “bartlett” (the default) is the Newey-West estimator. “uniform” weights all autocovariances up to lags equally. See diebold_mariano_test.

Returns:
ClarkWestResult

A result object containing the test statistic, its one-sided p-value and the mean squared prediction errors.

Raises:
MissingDataError

If the adjusted loss differential contains non-finite values.

ValueError

If horizon or lags is too large for the number of observations, if the adjusted loss differential is constant up to rounding error, for example when the two forecasts are identical, or if its estimated long-run variance is not positive, which can happen with the uniform kernel.

See also

diebold_mariano_test

Test of equal predictive accuracy of two forecasts.

Notes

If the restricted model is nested in the unrestricted model, the comparison of their mean squared prediction errors (MSPE), for example with diebold_mariano_test, is biased against the unrestricted model. Under the null hypothesis the additional parameters of the unrestricted model are zero, but their estimation adds noise to its forecasts, so its MSPE is expected to be larger. Clark and West (2007) adjust the MSPE of the unrestricted model for this noise. With the forecast errors \(e_{1,t}\) and \(e_{2,t}\) and the forecasts \(\hat{y}_{1,t}\) and \(\hat{y}_{2,t}\) of the restricted and the unrestricted model, the test is based on the adjusted loss differential

\[f_t = e_{1,t}^2 - \left[e_{2,t}^2 - (\hat{y}_{1,t} - \hat{y}_{2,t})^2\right] = 2 e_{1,t} (\hat{y}_{2,t} - \hat{y}_{1,t}).\]

The statistic is the t-statistic of the mean of \(f_t\), computed with a HAC estimate of its variance as in diebold_mariano_test. The null hypothesis is equal predictive accuracy and the alternative is that the unrestricted model is more accurate, so the test is one-sided and rejects for large values of the statistic. Because of the adjustment, forecast_restricted and forecast_unrestricted are not interchangeable: swapping them does not only change the sign of the statistic.

The statistic is not exactly normal under the null. Clark and West (2007) recommend the standard normal critical values 1.282 for a test at the 10% level and 1.645 at the 5% level, and find that these tests reject slightly less often than the nominal level. The p-value is computed from the standard normal distribution and is approximate.

The forecasts must be out-of-sample, for example from rolling or recursive estimation, and y[t] is the observation that forecast_restricted[t] and forecast_unrestricted[t] predict. The errors of forecasts h > 1 steps ahead are serially correlated, and Clark and West (2007) only state that an autocorrelation consistent standard error should be used. For one-step forecasts they use the least squares standard error, which corresponds to lags=0 and not to the default number of lags. With lags=0 the statistic is the least squares t-statistic of the mean of \(f_t\) multiplied by \(\sqrt{T / (T - 1)}\), since the HAC estimator divides by \(T\).

References

[1]

Clark, Todd E., and Kenneth D. West. “Approximately normal tests for equal predictive accuracy in nested models.” Journal of Econometrics 138, no. 1 (2007): 291-311.

Examples

Compare the historical mean with a predictive regression on x, where x[t] is known before y[t] is observed. Both forecasts use recursive estimation windows.

>>> import numpy as np
>>> from statsmodels.tsa.stattools import clark_west_test, diebold_mariano_test
>>> rng = np.random.default_rng(0)
>>> x = rng.standard_normal(400)
>>> y = 0.05 * x + rng.standard_normal(400)
>>> f_restricted, f_unrestricted = [], []
>>> for t in range(200, 400):
...     f_restricted.append(y[:t].mean())
...     slope, intercept = np.polyfit(x[:t], y[:t], 1)
...     f_unrestricted.append(intercept + slope * x[t])
>>> res = clark_west_test(y[200:], f_restricted, f_unrestricted)
>>> print(f"{res.statistic:.3f}, {res.pvalue:.4f}")
1.873, 0.0306

The one-sided Diebold-Mariano test of the same forecasts does not adjust for the noise from the estimated slope and does not reject at the 5% level.

>>> dm = diebold_mariano_test(
...     y[200:], f_restricted, f_unrestricted, alternative="larger"
... )
>>> print(f"{dm.statistic:.3f}, {dm.pvalue:.4f}")
1.251, 0.1056