To test for the effects of data revisions, the regression results of each earlier vintage is tested against the regression produced by vintage 2008:Q2, while keeping the number of observations the same in each comparison. This amounts to testing for coincidence, which means that the estimated intercepts and slope coefficients are statistically equivalent at the 5% significance level.
Three different methodologies are used to test for coincidence. Lafléche and Armour’s (2006) model of Ordinary Least Squares (OLS) is used as a benchmark comparison against two versions of the kernel weighted least squares (KWLS) method of nonparametrics. The first nonparametric methodology involves using the average of all the local conditional nonparametric estimators, which is referred to as the global nonparametric model. It is offered as a measure of central tendency and is meant as a direct comparison against OLS. The second methodology involves using the local results of the nonparametric regression produced conditional on just the very last observation, i.e. the Tth observation of each comparison vintage.
In regards to using quarterly data, which this paper analyzes, a maximum of twelve observations of a given real-time data set has the potential of changing at any given time, aside from the benchmark revisions. This paper finds that an econometric model that is aggregate-driven such as OLS is unable to utilize the subtlety of the new information, while both versions of the KWLS nonparametric model is able to do so especially at the Tth local conditional level. The effects of data revisions are only detected in 7% of the regressions in the OLS model, 56% of the regressions for the global nonparametric model, and in 87% of the regressions involving the Tth local conditional nonparametric model
Thus, data revisions, which are subtle changes in magnitude, can be lost in aggregation or when outliers dominate such as in the parametric model. With the proper measuring tool such as the global nonparametric model and especially the Tth local conditional nonparametric model, which are flexible and able to provide local time-varying estimators, data revisions can be gleaned for used in policy analysis.