Skip Navigation

On January 31, 2001, the Congressional Budget Office (CBO) released The Budget and Economic Outlook: Fiscal Years 2002-2011, which presents CBO's latest projections of federal revenues and outlays for that period. Chapter 5 of that report discusses the uncertainties in CBO's baseline projection of the total budget surplus and includes a chart (reproduced here as Figure 1) illustrating how those uncertainties increase over six years. This supplement to that report discusses the data and methods used to construct the chart.
 

Figure 1. Uncertainty in CBO's Projections of the Total Budget Surplus Under Current Policies (By fiscal year)

SOURCE: Congressional Budget Office.

NOTES: The figure shows the estimated likelihood of alternative projections of the surplus under current policies. The calculations are based on CBO's past track record. CBO's baseline projection falls in the middle of the darkest area. Assuming that policies do not change, the probability is 10 percent that actual surpluses will fall in the darkest area and 90 percent that they will fall within the whole shaded area.

Actual surpluses will of course be affected by legislation enacted during the next 10 years, including decisions about discretionary spending. The effects of future legislation are not included in this figure.

Figure 1 presents CBO's baseline projection of the budget surplus as a fan of probabilities around the mean projection for fiscal years 2001 through 2006. The fan widens as the projection extends. The baseline projection falls in the middle of the highest probabilities--the darkest part of the figure. But the figure makes clear that nearby projections--other paths in the darkest part of the figure--have nearly the same probability as the baseline. Moreover, projections that are quite different from the baseline have a significant probability of being realized.⁽¹⁾

The uncertainties shown in Figure 1 are based on CBO's historical record of budget projections. That is, the estimates of uncertainty presume that in the future, CBO will make errors similar to those it made in the past, with about the same probability distribution of large and small errors.

Preparing Figure 1 involved two stages. In the first stage, CBO constructed measures of its past projection errors that take into account the effects of legislation and other factors. In the second stage, CBO constructed probability distributions for those errors at six time horizons, beginning with the current fiscal year (the year in which the projection was made) and covering the next five years. Those distributions are depicted in Figure 1.
 

Stage One: Constructing the Measures of Error

CBO made several adjustments to its previously published projections before subtracting them from actual values to determine past projection errors. The projections of revenues and outlays were adjusted to account for the estimated effects of legislation enacted after the projections were prepared. CBO made that adjustment because its baseline projections are intended to show the expected level of the budget deficit or surplus assuming that current tax and spending policies continue. Without that adjustment, the measure of projection error would include the effects of later legislation, which would run counter to the purpose of the baseline.

In calculating the projection errors, CBO excluded discretionary spending and net interest payments from both the baseline projections and actual outlays. Discretionary spending was omitted because all discrepancies between actual discretionary spending and baseline projections of discretionary spending were counted as resulting from legislation (since levels of discretionary spending are determined anew each year through appropriation legislation).⁽²⁾ Interest payments were excluded because those payments depend on the size of the government's publicly held debt--the cumulation of annual budget deficits--so errors in projecting interest payments depend on the cumulation of errors in projecting the deficit. In stage two, when CBO constructed its probability distributions, interest payments were brought back into the calculations.

Errors were calculated for each year covered by the winter baseline projections that CBO published from 1981 through 2000. In most years, those projections were published in January or February, although in 1996, publication was delayed until May. Also, CBO used its July 1981 projection in place of the one published in February 1982.⁽³⁾ The resulting sample size was small: only 19 for current-year projections, declining to 14 for five-year-ahead projections.⁽⁴⁾ (The sample size diminishes because projections made in the past five years can be compared with actual data only through 2000.)

The estimated effects of legislation were taken primarily from data published in CBO's twice-yearly reports on the budget and economic outlook. Most of those reports show the multiyear budgetary effects of legislation enacted since the previous projection. For cases in which estimates were not available (as will be discussed below), substitutes were constructed.

Revenues

The effects of tax legislation were originally estimated by either the Congress's Joint Committee on Taxation (JCT) or CBO at the time the legislation was being considered. Those estimates are not revised after their initial publication, even though later economic and technical information might permit better estimates (for example, knowledge of an actual tax base, such as wages or corporate profits, would improve estimates of how a change in tax law would affect revenues). Using unrevised data on the effects of legislation may overstate the true uncertainty of CBO's budget projections, other things held equal.

Those estimated effects of tax legislation were used to adjust each baseline revenue projection. For example, the baseline projection of revenues made in January 1994 for fiscal year 1999 was adjusted downward from $1,630 billion to $1,619 billion (see Table 1). That adjustment reflected all tax legislation enacted after the January 1994 projection was published.
 

Much of that legislation was contained in the Taxpayer Relief Act of 1997. CBO and JCT estimated that the provisions in that act would reduce revenues by $7 billion in fiscal year 1999.⁽⁵⁾ Similar adjustments were made for the other years in the baseline projections. The differences between those adjusted baseline projections and actual revenues represent the errors attributable to economic and technical factors (see Table 2).
 

Outlays

The estimated effects of legislation on outlays (excluding discretionary spending and interest payments) were also taken largely from CBO's reports on the budget and economic outlook. However, some adjustment to that information was necessary.

• Baseline Projections of Discretionary Spending. As noted above, differences between actual and projected levels of discretionary spending were assumed to be attributable to legislation. But the July 1981 projection did not include a separate category for discretionary spending. For that baseline only, discretionary spending was approximated by adding the projections for defense, other grants to state and local governments, and other federal operations.⁽⁶⁾
  
  
• Insufficient Details About Legislation. In some cases, the estimated effects of legislation were not published in enough detail to exclude the effects of legislation on discretionary spending. In other cases, the data were published for some but not all of the six years in the baseline budget projection. One or both of those problems applied to the following periods: August 1986 to January 1987, August 1987 to February 1988, August 1994 to January 1995, and January 1998 to August 1998. In those cases, supplemental information from CBO's files was used to estimate the needed numbers.

As with revenues, the estimated effects of legislation on outlays were used to adjust each baseline projection of outlays (after removing the projections of discretionary spending and interest payments). The differences between those adjusted projections and actual outlays are the errors attributable to economic and technical factors (see Table 3).
 

Primary Budget Surplus or Deficit

The primary budget surplus--or deficit in earlier years--excludes net interest payments. The errors for outlays were subtracted from the errors for revenues to determine CBO's errors in projecting the primary budget surplus adjusted for the effects of tax and spending legislation (see Table 4). Those errors were cumulated into errors in projecting publicly held debt, which in turn were used in stage two to estimate the uncertainty of CBO's projections of interest payments.
 

Stage Two: Constructing Probability Distributions

Drawing reliable inferences about the uncertainty of CBO's budget projections over a six-year horizon on the basis of only 14 observations requires making some important assumptions. This section describes the basic assumptions that allowed CBO to construct probability distributions of its projections of the adjusted primary surplus based on its track record, as well as the assumptions that CBO used to calculate uncertainty in its projections of the total surplus (the primary surplus plus net interest).

The Probability Model

The uncertainty calculations are based on a simple probability model for the primary surplus. For the current-year projections, errors are assumed to be normally distributed with a mean of zero. For subsequent years, projection errors are assumed to be cumulative: that is, if a projection misses early on, the error is likely to be carried into subsequent years.

Because the sample of current-year errors is too small to estimate a probability distribution with any confidence, CBO assumed that the errors were generated by a normal distribution. An appealing aspect of that assumption is that the entire distribution is determined by specifying only two parameters: the mean and the standard deviation. Although CBO's mean error in projecting the primary surplus since 1981 was 1.6 percent of actual revenues (see Table 4), CBO assumed that the mean error was zero.⁽⁷⁾ That seems like a reasonable assumption since CBO aims to produce unbiased forecasts.

The standard deviation, or dispersion, of the distribution of the current-year projection errors was assumed to equal the root mean square of the errors (RMSE) in the track record.⁽⁸⁾ A normal distribution with that standard deviation produces a very simplified version of the actual distribution of errors in CBO's current-year projections (see Figure 2). The actual errors seem very spiky compared with the assumed distribution because, in a small sample, individual events loom large.
 

Figure 2. Frequency Distributions for Errors in Projecting the Current-Year Primary Surplus

SOURCE: Congressional Budget Office.

a. Calculated by evaluating the normal probability density function with a mean of zero and a standard deviation equal to 3.6 (the root mean square error of CBO's errors in projecting the current-year primary surplus).

The probability model for subsequent years takes into account the cumulative nature of the errors. In CBO's track record, the projection errors for the adjusted primary surplus are correlated over the projection horizon, as would be expected. The correlation between the errors for the current-year and first-year-ahead projections is about 0.7. For more distant projections it is even higher, reaching almost 1 between the fourth- and fifth-year-ahead projections. Those correlations, however, do not indicate how much the influence of one error spreads to subsequent errors one or more years ahead in the same projection.

To assess the degree to which the errors grow over the projection horizon, CBO estimated regression equations in which each period's projection error was specified as some multiple of the previous period's error. To illustrate CBO's method, assume that the symbol U_t(h) denotes the projection error made for the primary surplus in year t+h as projected in year t, with h taking the value of any integer between zero and 5. (For example, the notation U₁₉₉₀(3) would be the error CBO made in projecting the 1993 primary surplus in the projection it published in January 1990. That error equaled -15.8 percent of revenues, as shown in Table 4.) The error U_t(h) is assumed to evolve through the projection horizon according to the following rule:

in which e_t(h) is a random shock and A_h is a constant propagation parameter for each value of h = 1, . . . , 5. CBO ran five regressions yielding estimates of A₁, A₂, A₃, A₄, and A₅. In each case, the estimated propagation parameter slightly exceeded 1. However, in only one case (h = 2) did the coefficient exceed 1 to a degree that was statistically significant at the 5 percent confidence level.

For simplicity, CBO assumed that each error was equal to the previous year's error plus a random shock--that is, CBO assumed A_h = 1 for each horizon h. In light of the very limited data available, CBO also assumed that changes in the projection errors at different horizons (the e_t(h), when A_h = 1) were drawn from a common distribution. Specifically, CBO assumed that the e_t(h) are normally distributed with a mean of zero and a standard deviation equal to the root mean square of the change in CBO's errors for all values of t and h. That normal distribution described the pooled e_t(h) reasonably well (see Figure 3). The spikiness of the actual data is a little less pronounced than in Figure 2 because the data for different projection horizons are grouped together, effectively increasing the sample size and reducing the influence of particular unusual events.
 

Figure 3. Frequency Distributions for Changes in Errors in Projecting the Current-Year Primary Surplus

SOURCE: Congressional Budget Office.

a. Calculated by evaluating the normal probability density function with a mean of zero and a standard deviation equal to 5.1 (the root mean square error of the changes in CBO's errors in projecting the primary surplus).

By assuming A_h = 1 when the numerical regression estimates exceed 1, CBO's estimates of the uncertainty in the primary surplus are tilted to the low side. If in fact CBO had assumed that the propagation parameters exceed 1, the estimated uncertainty of CBO's five-year-ahead projection errors would be considerably larger than reported here.

Calculating the Uncertainty of Projections

CBO's uncertainty calculations were based on simulations of the probability model for the primary surplus. CBO performed 8,000 simulations; in each one, an error was chosen at random from the distribution for current-year errors, and five additional errors were chosen at random from the distribution for the changes in errors. Each of those simulations then provided a possible path of errors for the primary surplus over a six-year period.

The total surplus shown in the fan chart (Figure 1) also requires information about how the predicted errors in the primary budget (the budget excluding net interest) will affect the government's debt-service costs. CBO ran each of the 8,000 simulations of the primary budget through a simple debt-service model. For each period, that model tracks how the errors in the surplus projection translate into errors in debt, applying an interest rate that is a weighted average of CBO's current baseline projections of rates on three-month Treasury bills and 10-year Treasury notes. That model is an approximation of the model CBO uses for its budget projections.⁽⁹⁾

The result is a collection of 8,000 simulated error paths, which were added to CBO's baseline projections to create 8,000 simulated projections of the total surplus. The percentile distribution of those projections (calculated independently for each projection horizon), which is shown in Table 5, was used to create the fan chart.
 

Conclusions

Table 5 and the chart derived from it (Figure 1) serve to illustrate the uncertainty of CBO's budget projections. The calculations for that chart, however, are based on a very limited set of data and rest on important assumptions. Thus, those calculations are themselves uncertain, and the numbers shown in Table 5 should not be considered precise. CBO would welcome suggestions about how to improve this analysis.

1. Technically, the probability density is highest near the baseline and falls off for more distant projections.

2. In previous analyses of its track record, CBO split discrepancies in discretionary spending into three components: the lion's share was attributed to legislation, but small portions were attributed to economic and technical assumptions. Attributing all discrepancies in discretionary spending to legislation, as is done here, permits the use of a larger historical record. Since 1986, the Balanced Budget and Emergency Deficit Control Act has mandated that the baseline for discretionary spending reflect assumptions about inflation. As a result, baselines for discretionary spending made before 1986 are not comparable with those made afterward. Counting all discrepancies in discretionary spending as legislative avoids that problem.

3. Specifically, CBO does not have enough information in its files to include the estimated effects of legislation enacted between February 1982 and February 1983. Much better data are available for the slightly longer period of July 1981 through February 1983.

4. The sample size could have been doubled by including the updated projections that CBO typically publishes in the summer, but those updates are closely related to the winter baselines and do not really offer additional information.

5. That $7 billion reduction was the net effect of a $29 billion decrease and a $22 billion increase ($12 billion of which reflected shifts in the timing of tax collections). See Congressional Budget Office, The Economic and Budget Outlook: An Update (September 1997), p. 36.

6. See Congressional Budget Office, Baseline Budget Projections: Fiscal Years 1982-1986 (July 1981), p. 38.

7. The sample mean was just barely statistically significant at the 5 percent confidence level.

8. The root mean square error is calculated by squaring each projection error, averaging the squares, and taking the square root of the result. When the errors are known to have a population mean of zero, the RMSE is an unbiased estimate of the population standard deviation. When the population's true mean is not zero, the RMSE would be expected to exceed the standard deviation.

9. CBO also simulated the projection errors under several alternative assumptions about the uncertainty in interest rates. Including that additional uncertainty, however, did not appreciably affect the results.