statsmodels.tsa.stattools.clark_west_test#
- statsmodels.tsa.stattools.clark_west_test(y, forecast_restricted, forecast_unrestricted, *, lags=None, horizon=1, kernel='bartlett')[source]#
Clark-West test of equal predictive accuracy of nested models.
Added in version 0.15.1.
- Parameters:
- yarray_like
The observed values of the forecast variable. The array is 1-D; a column with shape (nobs, 1) is also accepted.
- forecast_restrictedarray_like
The forecasts of
yfrom the restricted model, for example the historical mean or a random walk. The array is 1-D; a column with shape (nobs, 1) is also accepted.- forecast_unrestrictedarray_like
The forecasts of
yfrom the unrestricted model, which nests the restricted model. The array is 1-D; a column with shape (nobs, 1) is also accepted.- lags
int,optional The number of lags to include in the HAC variance estimator used for the adjusted loss differential. Must be non-negative if provided. If not provided,
max(horizon - 1, ceil(nobs ** (1/3)))is used with the Bartlett kernel andhorizon - 1with the uniform kernel, as indiebold_mariano_test. Must be smaller thannobs, and smaller thannobs - 1with the uniform kernel.- horizon
int,optional The forecast horizon, used to form the default number of lags. Must be a positive integer. Must be smaller than
nobsiflagsis not provided, and is otherwise not used. Default is 1.- kernel{“bartlett”, “uniform”},
optional The kernel of the HAC estimator. “bartlett” (the default) is the Newey-West estimator. “uniform” weights all autocovariances up to
lagsequally. Seediebold_mariano_test.
- Returns:
ClarkWestResultA result object containing the test statistic, its one-sided p-value and the mean squared prediction errors.
- Raises:
MissingDataErrorIf the adjusted loss differential contains non-finite values.
ValueErrorIf
horizonorlagsis too large for the number of observations, if the adjusted loss differential is constant up to rounding error, for example when the two forecasts are identical, or if its estimated long-run variance is not positive, which can happen with the uniform kernel.
See also
diebold_mariano_testTest of equal predictive accuracy of two forecasts.
Notes
If the restricted model is nested in the unrestricted model, the comparison of their mean squared prediction errors (MSPE), for example with
diebold_mariano_test, is biased against the unrestricted model. Under the null hypothesis the additional parameters of the unrestricted model are zero, but their estimation adds noise to its forecasts, so its MSPE is expected to be larger. Clark and West (2007) adjust the MSPE of the unrestricted model for this noise. With the forecast errors \(e_{1,t}\) and \(e_{2,t}\) and the forecasts \(\hat{y}_{1,t}\) and \(\hat{y}_{2,t}\) of the restricted and the unrestricted model, the test is based on the adjusted loss differential\[f_t = e_{1,t}^2 - \left[e_{2,t}^2 - (\hat{y}_{1,t} - \hat{y}_{2,t})^2\right] = 2 e_{1,t} (\hat{y}_{2,t} - \hat{y}_{1,t}).\]The statistic is the t-statistic of the mean of \(f_t\), computed with a HAC estimate of its variance as in
diebold_mariano_test. The null hypothesis is equal predictive accuracy and the alternative is that the unrestricted model is more accurate, so the test is one-sided and rejects for large values of the statistic. Because of the adjustment,forecast_restrictedandforecast_unrestrictedare not interchangeable: swapping them does not only change the sign of the statistic.The statistic is not exactly normal under the null. Clark and West (2007) recommend the standard normal critical values 1.282 for a test at the 10% level and 1.645 at the 5% level, and find that these tests reject slightly less often than the nominal level. The p-value is computed from the standard normal distribution and is approximate.
The forecasts must be out-of-sample, for example from rolling or recursive estimation, and
y[t]is the observation thatforecast_restricted[t]andforecast_unrestricted[t]predict. The errors of forecastsh > 1steps ahead are serially correlated, and Clark and West (2007) only state that an autocorrelation consistent standard error should be used. For one-step forecasts they use the least squares standard error, which corresponds tolags=0and not to the default number of lags. Withlags=0the statistic is the least squares t-statistic of the mean of \(f_t\) multiplied by \(\sqrt{T / (T - 1)}\), since the HAC estimator divides by \(T\).References
[1]Clark, Todd E., and Kenneth D. West. “Approximately normal tests for equal predictive accuracy in nested models.” Journal of Econometrics 138, no. 1 (2007): 291-311.
Examples
Compare the historical mean with a predictive regression on
x, wherex[t]is known beforey[t]is observed. Both forecasts use recursive estimation windows.>>> import numpy as np >>> from statsmodels.tsa.stattools import clark_west_test, diebold_mariano_test >>> rng = np.random.default_rng(0) >>> x = rng.standard_normal(400) >>> y = 0.05 * x + rng.standard_normal(400) >>> f_restricted, f_unrestricted = [], [] >>> for t in range(200, 400): ... f_restricted.append(y[:t].mean()) ... slope, intercept = np.polyfit(x[:t], y[:t], 1) ... f_unrestricted.append(intercept + slope * x[t]) >>> res = clark_west_test(y[200:], f_restricted, f_unrestricted) >>> print(f"{res.statistic:.3f}, {res.pvalue:.4f}") 1.873, 0.0306
The one-sided Diebold-Mariano test of the same forecasts does not adjust for the noise from the estimated slope and does not reject at the 5% level.
>>> dm = diebold_mariano_test( ... y[200:], f_restricted, f_unrestricted, alternative="larger" ... ) >>> print(f"{dm.statistic:.3f}, {dm.pvalue:.4f}") 1.251, 0.1056