Skip Navigation
| Skip Navigation |
|
|
|
|
|
On January 27, 2004, the Congressional Budget Office (CBO) released The Budget and Economic Outlook: Fiscal Years 2005 to 2014. Appendix A of that report discusses the uncertainty of CBO's baseline projection of the total budget balance and includes a figure illustrating how the uncertainty increases as the projections extend into the future. This supplementary report describes the data and methods used to construct that figure. Ufuk Demiroglu and Frank Russek of CBO's Macroeconomic Analysis Division wrote this report under the direction of Robert Dennis. Tumi Coker provided research assistance, and Linda Lewis Harris helped in the early stages of the document's preparation. Christine Bogusz edited the report, and Christian Spoor prepared it for publication. Annette Kalicki produced the electronic versions for CBO's Web site. Douglas Holtz-Eakin
On January 27, 2004, the Congressional Budget Office (CBO) released The Budget and Economic Outlook: Fiscal Years 2005 to 2014, which presents CBO's latest projections of federal revenues and outlays for that period.(1) Appendix A of that report discusses the uncertainty in CBO's baseline projection of the total budget balance and includes a figure (reproduced here as Figure 1) illustrating how that uncertainty increases as the projections extend into the future. This supplementary report describes the data and methods used to construct the figure. In brief, CBO calculated measures of uncertainty using the inaccuracies in its past projections that arose from economic and technical factors. Uncertainty arising from legislation was not considered because baseline projections assume that current tax and spending policies remain the same.
Uncertainty of CBO's Projections of the Total Budget Deficit or Surplus Under Current Policies (Trillions of dollars)
Source: Congressional Budget Office. Note: Calculated on the basis of CBO's forecasting track record, this figure shows the estimated likelihood of alternative projections of the total budget deficit or surplus under current policies. CBO's baseline projection falls in the middle of the darkest area. Under the assumption that tax and spending policies do not change, the probability is 10 percent that the actual deficit or surplus for each year will fall in the darkest area and 90 percent that it will fall within the whole shaded area. The probability that all of the next five years of deficits or surpluses will fall within the fan is less than 90 percent; it is closer to 70 percent, according to CBO's track record. Actual deficits or surpluses will of course be affected by legislation enacted during the next five years, including decisions about discretionary spending. The effects of future legislation are not included in this figure. Figure 1 presents CBO's baseline projection of the budget balance as a fan of probabilities around the mean projection for 2004 through 2009. The fan widens as the projection period extends. The baseline projection falls in the middle of the highest-probability area--the darkest part of the figure. But the figure makes clear that nearby projections--other paths in the darkest part of the figure--have nearly the same probability of occurring as the baseline projection does. Moreover, projections that are quite different from the baseline have a significant probability of being realized.(2) The shaded area in the figure represents the 90 percent confidence range (the range within which the actual value for each year has a 90 percent chance of falling). CBO estimates that range on the basis of the uncertainty in its historical record of budget projections--a total of 22 baselines spanning the period from 1981 to 2003.(3) In other words, the estimates of uncertainty presume that, in the future, CBO will experience inaccuracies similar to those it experienced in the past, with about the same probability distribution of large and small inaccuracies. CBO's analysis of uncertainty separates inaccuracies correlated with the business cycle from those that are not. That distinction helps in estimating the probability distribution of the inaccuracies, as discussed later in this report. Cyclical inaccuracies are not expected to grow after the first few years of a projection's horizon, whereas noncyclical inaccuracies may persistently grow as the projection's horizon lengthens. According to CBO's estimates, cyclical inaccuracies historically have been small for the first two years of a baseline projection, the period for which CBO incorporates its views of the business cycle in its forecasts. Those cyclical inaccuracies rise in the later years of a projection--when CBO does not try to forecast the business cycle--but then flatten out (see Figure 2). Noncyclical inaccuracies, by contrast, continue to grow in the later years.(4) That breakdown suggests that, on average, CBO's inaccuracies in projecting the budget's bottom line have consisted, in roughly equal parts, of cyclical inaccuracies and inaccuracies in assessing economic trends and noncyclical factors that underlie the budget.
Cyclical and Noncyclical Inaccuracies in Projecting the Primary Deficit or Surplus (Percentage of total revenues)
Source: Congressional Budget Office. Notes: The lines in this figure show root-mean-square errors (RMSEs), a type of average that ignores the signs of individual errors and gives greater weight to larger errors. The RMSE of total inaccuracy is calculated by squaring the RMSEs of the cyclical and noncyclical parts, adding them together, and taking the square root of the sum. Thus, the combined RMSE is smaller than the sum of the two components' RMSEs. The primary budget deficit or surplus is the difference between federal revenues and federal outlays excluding net interest. The 1981-2003 sample period is not typical of the post-World War II period as a whole. It contains only three recessions (those of 1981-1982, 1990-1991, and 2001)--compared with seven in the earlier post-World War II years--and the two most recent recessions were milder than average. Moreover, the 1981-1982 recession is not well represented in the sample because only one of the baseline projections preceded it.(5) If CBO had been confronted over the past two decades with a less stable economy--one more representative of the cyclical experience of the whole post-World War II period--the cyclical component would have been roughly 50 percent larger than the noncyclical component, on average. However, even if CBO takes into account the greater volatility of output in that entire post-World War II period, the width of the fan chart increases by very little in the first three years of the projection period and by only one-fourth by the fifth year. Whether the next decade will more closely resemble the past two decades or the entire postwar period cannot be determined in advance. However, recent research suggests that fundamental changes in the economy that occurred in the early 1980s may have resulted in fewer and milder cyclical movements in the past two decades and may presage a relatively stable economy in the future. Analysts differ on the nature of those changes, but a return to higher volatility is not expected within the next five years.(6) Volatility in the next five years could also be lower than in the past two decades. Preparing the fan chart involved two stages. In the first stage, CBO constructed measures of its past projection inaccuracies that remove the effects of changes in legislation and other factors. In the second stage, CBO constructed probability distributions at six time horizons, beginning with the current fiscal year (the one in which the projection was made) and covering the next five years. The probability distributions were derived from a model that distinguishes between inaccuracies that appear to stem from the difficulty of forecasting the business cycle and inaccuracies that are not correlated with the business cycle and appear to stem from other causes.
Stage One: Constructing the Measures of InaccuraciesCreating measures of inaccuracies in CBO's past budget projections involved adjusting those projections for two factors: legislation (including laws that affect discretionary spending) and net interest on the federal debt. CBO added to its projections of revenues and outlays the estimated effects of laws concerning revenues or mandatory spending that were enacted after the projections were made. That adjustment was necessary because CBO's baseline projections are intended to show the future level of the budget deficit or surplus under the assumption that current tax and spending policies remain the same.(7) Without that adjustment to take into account subsequent tax and spending legislation, the measures of inaccuracies would reflect the effects of later policy changes, which would run counter to the purpose of the baseline. CBO excluded discretionary spending from both the baseline projections and actual outlays. The effect of omitting discretionary spending is to treat all discrepancies between actual discretionary spending and baseline projections of such spending in the same way as differences resulting from other budget legislation.(8) CBO decided on that treatment for several reasons: First, it permits the use of a longer historical record, which otherwise would be shorter because of insufficient data. Second, levels of discretionary spending are determined anew each year through appropriation acts, and any discrepancy between actual levels and baseline projections of discretionary spending is essentially attributable to legislation. Third, the inaccuracies in projecting discretionary outlays are small, so attributing all of those inaccuracies to legislation does not affect the measure of inaccuracy very much. Inaccuracies in projecting net interest largely result from inaccuracies in projecting the government's publicly held debt. That debt, in turn, is the cumulation of past budget deficits (minus surpluses), so inaccuracies in projecting net interest depend on the cumulation of all past inaccuracies in projecting the deficit or surplus. The final fan-chart calculations include all those effects on net interest. CBO calculated inaccuracies for each year covered by the winter baseline projections that it published from 1981 through 2003. In most years, those projections were issued in January or February, although in 1996, publication was delayed until May. For reasons involving the availability of data, CBO used its July 1981 projection in place of the one published in February 1982.(9) The resulting sample was small: only 22 current-year projections, declining to 17 five-year-ahead projections.(10) (The sample size diminishes because projections made in the past five years can be compared with actual outcomes only through 2003.) The estimated effects of legislation concerning revenues or mandatory spending were taken primarily from information published in CBO's twice-yearly reports on the budget and economic outlook. Most of those reports show the multiyear budgetary effects of legislation enacted since the previous projection was made. For cases in which estimates were not available (as will be discussed below), substitutes were constructed. The underlying worksheets used in computing the inaccuracies and a brief explanation of each one are available in the electronic version of this document at www.cbo.gov. RevenuesAs required by the Congressional Budget and Impoundment Control Act of 1974, the Joint Committee on Taxation (JCT) estimates the effects of tax legislation--bills that alter income, estate and gift, excise, or payroll taxes--at the time that the legislation is being considered by the Congress.(11) CBO produces estimates for legislation that affects customs duties and miscellaneous receipts. Those estimated effects of tax legislation were used to adjust each baseline projection of revenues. For example, the projection made in January 1994 for total revenues in fiscal year 1999 was lowered from $1,630 billion to $1,619 billion (see Table 1). That adjustment reflected all tax laws enacted after January 1994 and through fiscal year 1999. The law with the largest budgetary impact was the Taxpayer Relief Act of 1997, which JCT estimated would reduce revenues in 1999 by $7 billion.(12) Similar adjustments were made for the other years in the baseline projections. The differences between those adjusted projections and actual revenues represent the inaccuracies attributable to economic and technical factors (see Table 2).
Example: How CBO's January 1994 Revenue Projection Was Adjusted for Subsequent Legislation (Billions of dollars)
Source: Congressional Budget Office. Notes: The only major changes in tax law enacted in the five years after CBO's January 1994 baseline projection was made were in the Taxpayer Relief Act of 1997. That law's effects were incorporated in CBO's September 1997 baseline projection. Two other adjustments are notable but relatively minor. The January 1995 baseline projection reflected various reductions in tariff rates, primarily those in the Generalized System of Preferences. The downward adjustment in the August 1996 baseline projection reflected two pieces of legislation: the Health Insurance Portability and Accountability Act of 1996 and the Small Business Job Protection Act of 1996. * = between -$500 million and $500 million. Inaccuracies in CBO's Baseline Projections of Revenues That Are Attributable to Economic and Technical Factors (Percentage of actual revenues)
Source: Congressional Budget Office. Note: Forecast inaccuracies are actual revenues minus projected revenues, adjusted for the effects of legislation. CBO's and JCT's estimates of the effects of tax legislation are not revised after their initial publication, even though later economic and technical information might permit better estimates. (For instance, knowledge about an actual tax base, such as wages or corporate profits, in a given year would improve estimates of how a change in tax law would affect revenues.) Using unrevised estimates of the budgetary impacts of tax and spending legislation could affect the estimates of uncertainty in CBO's baseline budget projections, but the direction and size of that effect are unclear. OutlaysThe estimated effects of legislation on outlays (excluding net interest) were also taken largely from CBO's reports on the budget and economic outlook. However, as with revenues, some adjustment to that information was necessary. Baseline Projections of Discretionary Spending. As noted above, differences between actual and projected levels of discretionary spending were assumed to be attributable to legislation. But the July 1981 baseline projection did not include a separate category for discretionary spending. For that baseline only, discretionary spending was approximated by adding the projections for defense, other grants to state and local governments, and other federal operations.(13) Insufficient Details About Legislation. In some cases, the estimated effects of legislation were not published in enough detail to separate out the effects of legislation on mandatory spending. In other cases, the information was published for some but not all of the six years in the baseline budget projection. One or both of those problems applied to the following periods: August 1986 to January 1987, August 1987 to February 1988, August 1994 to January 1995, and January 1998 to August 1998. In those cases, supplemental information from CBO's files was used to estimate the needed numbers. As with revenues, the estimated effects of legislation on outlays were used to adjust each baseline projection of outlays. After removing interest payments and discretionary outlays, the differences between those adjusted projections and actual outlays are the inaccuracies attributable to economic and technical factors (see Table 3).
Inaccuracies in CBO's Baseline Projections of Outlays That Are Attributable to Economic and Technical Factors (Percentage of actual revenues)
Source: Congressional Budget Office. Notes: Forecast inaccuracies are actual outlays minus projected outlays, adjusted for the effects of legislation. They exclude inaccuracies in the baseline projections of discretionary spending (which are assumed to be attributable solely to legislation) and in the baseline projections of net interest (which depend on the inaccuracies in the surplus excluding interest). * = between -0.05 percent and 0.05 percent. Primary Budget Deficit or SurplusThe difference between revenues and outlays excluding net interest is known as the primary budget surplus (or deficit when negative). Correspondingly, CBO's inaccuracies in projecting revenues, minus its inaccuracies in projecting noninterest outlays, equal its inaccuracies in projecting the primary deficit or surplus (see Tables 4 and 5). As described above, that calculation excludes legislative changes. In stage two of the fan-chart preparation, the inaccuracies in projecting the primary budget deficit or surplus were cumulated into inaccuracies in projecting publicly held debt, which were used to estimate the uncertainty of CBO's projections of net interest.
Inaccuracies in CBO's Baseline Projections of the Primary Deficit or Surplus That Are Attributable to Economic and Technical Factors (Percentage of actual revenues)
Source: Congressional Budget Office. Note: Forecast inaccuracies are actual surpluses minus projected surpluses, adjusted for the effects of legislation. They exclude inaccuracies in the baseline projections of discretionary spending (which are assumed to be attributable solely to legislation) and in the baseline projections of net interest (which depend on the inaccuracies in the surplus excluding interest). The Historical Record of CBO's Baseline Budget Projections (Billions of dollars)
Source: Congressional Budget Office. Notes: Inaccuracies in projections of discretionary spending are assumed to be attributable solely to legislation and are included in the rows labeled "effect of legislation." The underlying worksheets used in computing the inaccuracies and a brief explanation of each one accompany the electronic version of this report, which is available at www.cbo.gov. * = between -$500 million and $500 million. a. Data shown here for years beyond the five-year horizon were not used in the calculation of uncertainty. Stage Two: Constructing Probability DistributionsThe historical record of inaccuracies in projecting the primary deficit or surplus (adjusted for legislation) presented in Table 4 forms the basis for the statistical calculations that CBO used to derive the probability distributions underlying the fan chart. As noted above, CBO's record of projections is both short and possibly unrepresentative (in that it is taken from a period that contains fewer and less frequent business cycles than occurred historically). In the absence of a large sample, estimates may be improved if additional information can be brought to bear. In this case, CBO used its knowledge of its forecasting procedures and of business cycles, as well as its historical record, to draw more-reliable conclusions about the probability distribution of inaccuracies in its budget projections. The Statistical Model for Inaccuracies in Forecasts of the Primary Deficit or SurplusWith the effects of legislation removed, CBO's past inaccuracies are closely related to errors in the projection of the business cycle. Forecasting the course of a business cycle more than two years ahead is virtually impossible, so CBO has traditionally tried to incorporate the business cycle in its economic projections explicitly for only the current year and the budget year.(14) In its projections for longer horizons, CBO simply assumes that gross domestic product (GDP) will, on average, adhere to its trend (or "potential") path.(15) That assumption recognizes that, in fact, GDP will sometimes be above and sometimes below its potential level, but CBO does not attempt to fore-cast those boom or recession periods more than a couple of years ahead. As long as CBO continues to do a reasonably good job of forecasting the business cycle for the current year, that cycle should not contribute much to the inaccuracy of current-year budget projections. For the budget year, its contribution should be larger (because errors in forecasting increase as the horizon lengthens) but still modest. For later years, however, cyclical factors should loom larger. In the last two years of the five-year projection period, CBO assumes that GDP is the same as or close to its potential level. Thus, any actual difference between GDP and its potential will not be reflected in those budget projections. Consequently, as the projection horizon extends, the budget misestimates that result from miscalculating the business cycle should grow in importance, until they reach their maximum level in the last two years of the five-year period. The portion of budget inaccuracies attributable to the business cycle may be estimated by using the correlation between those inaccuracies and the GDP gap (the percentage difference between actual GDP and its potential value). According to the above analysis, for projections several years ahead, the level of the GDP gap is a good indicator of unexpected cyclical conditions. For projections only one or two years ahead, by contrast, the change in the GDP gap may be a better indicator of cyclical surprises than the level, because the approaching levels of the gap can be quite similar to the recent level. Using the GDP gap and its change to measure unforeseen changes in cyclical conditions, CBO estimated by means of a linear regression what portion of its past inaccuracies was attributable to business cycles (see Box 1). Restrictions on the regression incorporate the exogenous information that, of the two variables, the change in the GDP gap is the main source of uncertainty over shorter horizons and the level of the gap over longer ones. For the intermediate year (the first year after the two-year forecast), both the level of the GDP gap and its change are taken to be important indicators of unexpected cyclical changes.
The portion of the overall inaccuracies explained by the two business-cycle variables in the regression is called the cyclical part. The rest, the noncyclical part, represents the inaccuracies that result from such factors as noncyclical changes in average tax rates, capital gains realizations, the share of GDP that goes to taxpayers in high tax brackets, and federal spending for Medicare and Medicaid.(16) CBO does not expect its projection inaccuracies to display a negative or positive bias--otherwise it would change its projections. Accordingly, CBO assumed that the probability distribution of its projection inaccuracies was centered around an average of zero. The data do not contradict that assumption. Calculating the Distribution of Inaccuracies from the ModelThe regression model produces coefficients that relate misestimates of the primary deficit or surplus (shown in Table 4) to the business-cycle variables. Given the historical pattern of the business cycle, those coefficients can be used to describe the distribution of inaccuracies that might be expected to occur simply because of the business cycle. One way to describe that distribution is through the root-mean-square error (RMSE), a kind of average error that ignores the signs of individual errors and gives extra weight to large errors.(17) The model assumes that the RMSE of the cyclical part of misestimates will rise to a plateau (see Figure 2). That model does not account for all of a given projection inaccuracy, however. What is left, the noncyclical part, also has a distribution that can be summarized by its RMSE. Like the cyclical component, that part of a misestimate has an RMSE that rises as the projection horizon lengthens, but it does not plateau (see Figure 2). For simplicity, CBO assumes that the noncyclical influences captured in the residual from the regression are independent of the cyclical component at each horizon.(18) That assumption is not contradicted by the data, and using the sample correlations makes little difference to the results.(19) The estimated RMSEs for the cyclical and noncyclical parts can be combined to form an estimate of the RMSE for overall budget misestimates. Two RMSEs are combined by squaring each of them, adding those squares together, and taking the square root of the sum. That calculation yields a combined RMSE that is less than the sum of the two component RMSEs (see Figure 2). The estimated RMSEs for a given year were formulated as a percentage of that year's actual revenues. For the projection of the total deficit or surplus, those RMSEs can be converted into dollars or expressed as a percentage of GDP using CBO's current baseline projection of total revenues and GDP. The model's estimate of the distribution of budget misestimates appears generally consistent with CBO's past record. Out of 117 past projection inaccuracies for the primary deficit or surplus in 1981 through 2003, only 8 percent fall outside the calculated 90 percent confidence range--a range that ought, in a large enough sample, to encompass 90 percent of the observations (see Figure 3).
CBO's Past Inaccuracies in Projecting the Primary Deficit or Surplus, Compared with the Constructed 90 Percent Confidence Range (Percentage of GDP)
 Source: Congressional Budget Office. Note: Each thin line represents the actual inaccuracies of the winter baseline projection made in a given year. The thick lines represent the 90 percent confidence range constructed from CBO's statistical model for inaccuracies. That range encompasses most of CBO's past record. Figure 4 compares the 90 percent confidence band for primary surplus projections with the inaccuracy of individual baselines from 1981 through 1998 (the only baselines for which the full record is available). The figure shows that the five-year projections made between 1993 and 1998 tended to be too pessimistic, and those made earlier (especially before 1991) tended to be too optimistic. The primary source of inaccuracy for the baselines between 1993 and 1998 was the unforeseen productivity acceleration of the 1990s and the associated rapid rise in revenues. For the earlier baseline projections, the primary sources of inaccuracy were the unexpected continuation of the productivity slowdown that started in the 1970s and the recessions of 1980, 1981-1982, and 1990-1991.
Misestimates in CBO's Projections of Primary Budget Deficits or Surpluses Made from 1981 to 1998, and the Estimated 90 Percent Confidence Intervals (Percentage of GDP)
 Source: Congressional Budget Office. Notes: CY = current year; BY = budget year. This figure shows misestimates in CBO's projections of the primary deficit or surplus--the total deficit or surplus excluding net interest--made at different times. In each panel, plotted points that lie below the center line reflect instances in which CBO underestimated the primary deficit or overestimated the primary surplus, whereas points above the center line reflect the opposite. The shaded cones indicate the estimated 90 percent confidence band; that is, there was a 90 percent chance that CBO's projection would be within the shaded area. CBO estimated that confidence band on the basis of its track record since 1981 (excluding 1982, because of insufficient data). The figure excludes the effects of legislation enacted after the projections were made. Uncertainty in Projections of the Total Deficit or SurplusDetermining the uncertainty range for CBO's current baseline projection of the total deficit or surplus (shown in Figure 1) requires information about how the predicted inaccuracies in the primary budget (the budget excluding net interest) will affect the government's debt-service costs. Those inaccuracies are run through a simple debt-service model that tracks how inaccuracies in projecting deficits or surpluses translate into inaccuracies in projecting debt; the model applies an interest rate that is a weighted average of CBO's current baseline projections of rates on three-month Treasury bills and 10-year Treasury notes. That model is an approximation of the model CBO uses for its budget projections. The inaccuracy in interest rate projections is not considered because its contribution to the overall inaccuracy is not expected to be substantial. The extent to which projection inaccuracies for the primary surplus are correlated across horizons is important for the computation of debt-service costs. When those inaccuracies are highly correlated, they have a large accumulated effect on outstanding debt, and the associated change in the government's interest burden is large. In calculating the probability distribution of projection inaccuracies for the total surplus (including net interest), CBO assumed that the cyclical and noncyclical parts would continue to have the same correlation structure as in the past.(20) The percentiles for the total surplus that are used to draw the fan chart are computed by multiplying the values associated with the various percentiles for the standard normal distribution by the calculated RMSE of the probability distribution of the total surplus at different horizons. Those percentiles are shown in Table 6. CBO will continue its efforts to refine these calculations. It welcomes suggestions for improving the methodology.
Estimated Probability Distribution of Total Budget Deficits or Surpluses (Billions of dollars)
Source: Congressional Budget Office. Notes: These numbers--constructed using the percentiles of the standard normal distribution and a simple probability model based on CBO's track record--underlie the fan chart presented as Figure 1. The row in the table corresponding to the 50th percentile is CBO's current baseline projection of the surplus. These estimates permit the construction of probability statements about CBO's baseline projection of the total budget surplus. For example, the table indicates that there is a 90 percent chance that the budget's balance in 2005 (the budget year) will be a deficit somewhere between $95 billion and $629 billion, and a 50 percent chance that the deficit or surplus in 2009 (the budget year + 4) will be within $316 billion of the baseline projection. (That last calculation takes the range from the 25th to the 75th percentiles and halves it.)
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
1 wh dt+h +