Since publishing its first macroeconomic forecast in 1976, the Congressional Budget Office (CBO) has compiled a forecasting track record comparable in quality with that of the Blue Chip consensus (an average of private-sector forecasters) as well as that of the Administrations during that time. In particular, CBO's accuracy for two-year forecasts made between 1982 and 2001 did not differ markedly either from those of the Administration or from those of the Blue Chip survey over the years (see Table 1a). CBO's errors in five-year projections of growth rates for real (inflation-adjusted) and nominal (current-dollar) output also were similar to those of the various Administrations and the Blue Chip consensus (see Table 1b). Comparing CBO's forecasts with those of the Blue Chip consensus suggests that when CBO's economic predictions missed by the largest margin, those inaccuracies probably reflected problems all forecasters had in predicting turning points in the business cycle.
 

Sources of Data for the Evaluation

The data employed in this evaluation were compiled using forecasts published early in the years from 1976 through 2001. (Two-year average forecasts published in 2002 and in early 2003 could not be included in this evaluation because the latest full-year historical data is for 2002.) For all years except 1981, CBO's forecasts were based on the calendar year forecasts published early each year (or from related files of unpublished forecasts for some variables in some years), whereas the Administration's forecasts were taken from the budget documents for all years except 1981. Where possible, Blue Chip consensus forecasts were taken from those published in the same month as CBO's forecasts. The appendix to this report gives further details on the choice of historical time-series data as well as on the sources of forecast data for the comparisons.
 

Measuring the Quality of Forecasts

Like earlier studies of economic forecasts, this evaluation focused on two aspects of the quality of CBO's forecasts: statistical bias and accuracy. Other desirable characteristics--such as the efficiency of a forecast, which is discussed later--are harder to assess definitively and would require a larger sample than is available for CBO's forecasts.

Bias

The statistical bias of a forecast is the extent to which the forecast can be expected to differ from what actually occurs, that is, its tendency to be overly pessimistic or optimistic. CBO's evaluation used the mean error (the arithmetic average of the forecast errors) to measure statistical bias. The mean error is the simplest and most widely used measure of forecast bias. Because it is a simple average, however, underestimates and overestimates offset each other in calculating it. As a result, the mean error imperfectly measures the quality of a forecast--a small mean error would result either if all the errors were small or if all the errors were large but the overestimates and underestimates happened to balance each other out.

Accuracy

The accuracy of a series of forecasts is the degree to which their values are narrowly dispersed around actual outcomes. Measures of accuracy more clearly reflect the usual meaning of forecast quality than does the mean error because overestimates and underestimates do not offset each other in these measures. CBO's evaluation used two measures of accuracy. The mean absolute error (the average of the forecasts' errors without regard to arithmetic sign) indicates the average distance between forecasts and actual values without regard to whether individual forecasts are overestimates or underestimates. The root mean square error (calculated by first squaring the errors, then taking the square root of the arithmetic average of the squared errors) also shows the size of the error without regard to sign, but it gives greater weight to larger errors.

Alternative Measures of Forecast Quality

Studies by analysts outside CBO have used measures that are somewhat more elaborate than the mean error to test for statistical bias in CBO's forecasts. Those studies have generally concluded, as does this evaluation, that CBO's short-term economic forecasts do not contain a statistically significant bias.⁽¹⁾

A number of other methods have been developed to evaluate a forecast's "efficiency," or the extent to which a particular forecast could have been improved by using additional information that was available when the forecast was made.⁽²⁾ the Blue Chip consensus forecasts represent a wide variety of economic forecasters and thus reflect a broader blend of sources and methods than can be expected from any single forecaster. In this evaluation, the Blue Chip predictions can therefore serve as a proxy for an efficient forecast. The fact that CBO's forecasts are about as accurate as the Blue Chip's is a rough indication of their efficiency.

Some researchers contend that the economic forecasting profession, in general, does not use all the information available to it to forecast downturns in the business cycle. As a result, it makes avoidable systematic errors in forecasting those business cycles.⁽³⁾ Such critics point to the ability of certain leading indicators to predict recessions. Nevertheless, it is unclear how the information in such leading indicators can reliably be translated into economic forecasts.

Limitations of Forecast Evaluations

Elaborate measures and methods are not necessarily reliable indicators of a forecast's quality when the sample of observations is small, as is the case with CBO's sample of only 26 two-year observations. Small samples present three main problems in evaluating forecasts. First, they reduce the reliability of statistical tests that are based on the assumption that the underlying population of errors in the forecast follows a normal distribution. The more elaborate measures of forecast quality all make such an assumption about the hypothetical ideal forecast with which the actual forecasts are being compared. Second, in small samples, individual errors in a forecast can have an unduly large influence on the measures. Third, the small sample means that CBO's track record indicates only weakly the possible direction or size of forecasting inaccuracies in the future.

Apart from the general caveat that should attend any statistical conclusions, there are several reasons for viewing any evaluation of CBO's forecasts with particular caution. First, the procedures and purposes of CBO's and the Administration's economic forecasts have changed over the past two decades and may change again. For example, in the late 1970s, CBO characterized its long-term projections as a goal for the economy; it now considers them to be a projection of what will prevail, on average, if the economy continues to reflect historical trends under current fiscal policies. Unlike CBO's projections, the Administration's normally included the projected economic effects of its own policy proposals. Second, an institution's track record in forecasting may not indicate its future abilities because of changes in personnel or methods. Third, inaccuracies in a forecast increase when the economy is more volatile and when economic trends change. All three groups of forecasters--CBO, the Administration, and the Blue Chip survey--made relatively large errors when forecasting for periods that included turning points in the business cycle and for the late 1990s, when the sustainable growth of the economy apparently increased.
 

The Effects of Business Cycles and Changes in the Trend Rate of Productivity Growth

As the track record shows, forecasters collectively tend to err during periods that include either turning points in the business cycle or significant shifts in the trend rate of productivity growth. For example, most forecasters overestimated the economy's growth rate and underestimated inflation in forecasts they made just before and during the back-to-back recessions of the early 1980s. That pattern was repeated, albeit to a lesser degree, in the forecasts they made just before the more moderate recession of the early 1990s. In addition, during the middle to late 1970s, forecasters continued to assume that the productivity trend of the previous two decades would prevail. In retrospect, however, the productivity trend of the 1970s and 1980s was significantly lower than that of the 1950s and 1960s. Because forecasters in the 1970s felt that the previous trend would recur, their forecasts of real output in the middle to late 1970s turned out to be too optimistic.

The years from 1995 to 2000 were a mirror image of the forecasting experience of the late 1970s. Partly because forecasters underestimated the trend rate of productivity growth since 1996, they underpredicted the economy's growth rate and overpredicted inflation. To be sure, as the economy continued to outperform expectations, analysts put more effort into investigating the possible causes of the increase in productivity growth. Those investigations focused on the possible contribution of the so-called new economy--especially the better flow of information among producers and between producers and consumers, which has permitted improved productivity, lower inventories, and greater customer satisfaction. Productivity growth has held up very well through the 2001 recession and early recovery, suggesting that the strong performance of U.S. productivity in the late 1990s was not simply a transient feature of the economic boom. Nevertheless, great uncertainty about the trend rate of productivity growth remains. That uncertainty will continue to cause forecasting errors, particularly for projections made five years' ahead.
 

CBO's Forecasting Record

This analysis evaluates the Congressional Budget Office's macroeconomic forecasts over two-year and five-year periods. Because the budget reports that the Administration and CBO publish each winter focus on budget projections for the fiscal year that begins in the following October, an economic forecast that is accurate not only for the months leading up to that budget year but also for the budget year itself will provide the basis for a more accurate forecast of the budget's bottom line--hence the interest in the two-year period. The five-year period is used to examine the accuracy of longer-term projections of growth in real and nominal output.

This analysis does not consider how assumptions about key economic factors can affect federal budget projections. "Rules of thumb" for estimating the effects of changes in various macroeconomic variables appear in Appendix C of CBO's The Budget and Economic Outlook: Fiscal Years 2004-2013 (January 2003).

Two-Year Forecasts

Historically, the accuracy of the Congressional Budget Office's two-year forecasts has been similar to the accuracy of the forecasts of the Administration and the Blue Chip consensus.

Growth in Real Output. There has been little difference in forecast accuracy between CBO and the Administration for the two-year forecasts made between 1976 and 2001 (see Table 2). CBO was closer to the actual value in 12 of the 26 forecasts made during those years, the Administration was closer in 10 periods, and the two forecasters had identical errors in four periods. In addition, CBO's predictions of real growth made between 1982 and 2001 were, on average, as accurate as those of the Blue Chip consensus.

As noted earlier, forecast errors tend to be larger at turning points in the business cycle and when there are shifts in major economic trends. That tendency can be clearly seen in the forecasts of real output growth by comparing the large errors for 1979 through 1983--when the economy went through its most turbulent recessionary period of the postwar era--with the smaller errors recorded for later years. Similarly, the business cycle accounts for the large errors in the predictions made in the 1989-1991 period. During that time, the Congressional Budget Office's errors were only slightly larger than those of the Blue Chip consensus. Most recently, the recession of 2001 and slow subsequent recovery account for the overpredictions made by all three forecasters in 2000 and 2001.

All three forecasters underpredicted two-year real gross domestic product (GDP) growth in every year between 1992 and 1999. Much of that apparent pessimism, however, resulted from revisions in the national income and product accounts (NIPAs). The Bureau of Economic Analysis's (BEA's) benchmark revisions published in November 2000 increased the two-year growth rates for real GDP over most of the historical period; in particular, rates went up by about 0.4 percentage points, on average, for the 1992-1998 period. The upward revision in growth rates stemmed largely from including software spending as investment in the accounts as well as adopting new price series for various categories of consumption. In addition to making the mean forecast error less informative, those revisions distort the reliability of the statistical measures of accuracy. Nevertheless, even after accounting for the latest revisions, the underpredictions of economic growth made between 1996 and 1999 are significant.

Growth in Nominal Output. The records of CBO and the Administration in forecasting two-year growth in nominal output are also quite similar (see Table 3). Of the 26 forecasts made between 1976 and 2001, the Administration recorded the smaller error 13 times, CBO had the smaller error 11 times, and the two forecasters recorded identical errors twice. CBO's mean error and root mean square error for that period are similar to the Administration's. BEA's benchmark revisions of November 2000 (noted above) also had a significant effect on the measures of accuracy for nominal output growth.

Over the shorter interval between 1982 and 2001, the bias and accuracy of CBO's forecasts of two-year growth in nominal output are nearly identical to those of the Blue Chip consensus.

CPI Inflation. CBO and the Administration had very similar records in forecasting the two-year average growth in the consumer price index (CPI) as well (see Table 4). Both underestimated future inflation in their forecasts for 1977 through 1980 and overestimated it for 1981 through 1986. Their average measures of bias and accuracy were virtually identical. CBO was closer to the actual value in eight of the 26 periods, the Administration was closer in 12 periods, and the two forecasters had matching errors in six periods. For the 1982-2001 period, CBO's forecasts of inflation were about as accurate as those of the Administration and slightly more accurate than those of the Blue Chip consensus.

Nominal Interest Rates. For the 1976-2001 forecasts, CBO's record was about as accurate as the Administration's for nominal short-term interest rates over a two-year period (see Table 5). On average, the Administration tended to underestimate those rates, whereas CBO overestimated them slightly (the forecasters' mean errors over that period were -0.4 and 0.1 percentage points, respectively). The Administration was closer to the actual value in 12 of the 26 periods, and the two forecasters had identical errors twice.

Between 1982 and 2001, the root mean square error of CBO's forecasts was slightly above that of the Blue Chip consensus, meaning that CBO made a few relatively large errors (such as those in 1982, 1983, and 1991). By contrast, in 2001, although CBO's error in forecasting the nominal short-term interest rate was quite large, the errors for the Blue Chip and the Administration were appreciably larger.

For the 1984-2001 forecasts of long-term interest rates, CBO did somewhat better than the Administration (see Table 6). Although the Administration's forecasts have tended to underestimate rates, its forecast for 2001 overestimated rates by a full percentage point. CBO was closer to the true value in 11 of the 18 periods, the Administration was closer in six periods, and the two forecasters had identical errors in one period.

The Congressional Budget Office's forecasts of long-term interest rates were about as accurate as those of the Blue Chip consensus. Both CBO and the Blue Chip tended to overestimate long-term rates for 1984-2001. CBO had a mean error of 0.2 percentage points over that period, compared with 0.3 percentage points for the Blue Chip.

Real Short-Term Interest Rates. For the forecasts made in 1976 through 2001, CBO had an edge over the Administration in estimating short-term interest rates adjusted for inflation (see Table 7). Again, the Administration was more likely than CBO to underestimate interest rates. Both forecasters recorded similar mean absolute and root mean square errors. CBO's forecasts were closer to the actual value in 14 of the 26 periods, the Administration's were closer in 10, and the two registered identical errors in two periods. For forecasts made between 1982 and 2001, CBO's errors were generally similar in both direction and magnitude to those of the Blue Chip consensus, although CBO's error in the 2001 forecast was appreciably smaller than those of both the Administration and the Blue Chip.

The Difference Between the Growth of the CPI and the GDP Price Index. The difference in the forecasted growth rates of the two major price indexes, the CPI and the GDP price index, is important for budget projections. The growth of the GDP price index is a major determinant in forecasting the growth of nominal GDP and, therefore, the growth of income subject to federal tax. All else being equal, the faster the projected growth of the GDP price index, the faster the projected growth of revenues. The growth of the consumer price index affects forecasts of outlays because a number of programs are indexed to the CPI. The CPI projection, however, also affects projections of revenues because parts of the personal tax code, such as tax brackets, are indexed to the CPI. All else being equal, the faster the growth of the CPI, the faster the growth of outlays and the slower the growth of revenues. Therefore, if the growth of the GDP price index is forecast to be large relative to that of the CPI, the projection of the surplus will be larger than if the GDP price index is assumed to grow much less rapidly than the CPI.

CBO's forecast record of the difference two years' ahead was slightly better than that of the Administration and the Blue Chip consensus (see Table 8). CBO was more accurate than the Administration in 14 of the 26 periods, and the two forecasters had identical errors in five periods; CBO was more accurate than the Blue Chip in nine of 20 forecasts, with seven ties.

Taxable Income. One of the greatest sources of error in budget projections is in forecasting taxable income. The errors in the first step of forecasting taxable income, the forecast of nominal GDP growth, were discussed above. The errors in the second step--the forecast of the relationship of major components of taxable income to nominal GDP, that is, the "high-tax" income share of GDP--are discussed here.

The errors in CBO's and the Administration's forecasts of the share of GDP of the two major components of taxable income--wages and salaries and corporate profits--were similar, on average (see Table 9). Forecasts of the taxable income share were consistently too low from 1994 to 1999, but far too high for 2001. (the Blue Chip consensus does not forecast wages and salaries.)

Two specific factors contributed to the string of underpredictions for taxable income in the mid- to late-1990s. The first was the degree to which total income has exceeded total product in the national income and product accounts. In principle, those two aggregate measures of economic activity should be equal, but in practice they are not, because the Bureau of Economic Analysis, which publishes the NIPAs, must use different primary sources to estimate total income, on the one hand, and total product, on the other. The statistical discrepancy in the NIPAs measures the difference between total product and total income; between 1997 and 2000, the excess of total income over total product grew--that is, the statistical discrepancy became more negative (see Figure 1). At this point, it is impossible to tell how much the change in the discrepancy has caused forecasters to err in their forecasts of income.
 

Figure 1.

Statistical Discrepancy in the National Income and Product Accounts

SOURCES: Congressional Budget Office; Department of Commerce, Bureau of Economic Analysis.

NOTES: The shaded vertical bars in this figure indicate periods of recession. The bars extend from the peak to the trough of each recession.

The statistical discrepancy is negative when the measure of national income exceeds the measure of national product.

A second source of difficulty in forecasting taxable income is the recent variability in the part of labor income that is not subject to tax. Throughout most of the post-World War II period, the nontaxable part of labor income rose as a share of total labor compensation because employers and employees preferred to substitute fringe benefits (such as employer-paid insurance premiums and pension contributions) for wages and salaries. But between 1994 and 2000, that trend reversed (see Figure 2). The decline in the share of total labor compensation that is not taxed (namely, fringe benefits) increased the share of compensation that is taxed (namely, wages and salaries). That turnaround stemmed from changes in the way health care is provided and from the rise in the stock market (which reduced the need for employers to contribute to defined-benefit pension plans). Since 2000, however, the trend appears to have reversed again, with the nontaxable share of labor income rising.
 

Figure 2.

Nonwage Labor Income

SOURCES: Congressional Budget Office; Department of Commerce, Bureau of Economic Analysis.

NOTE: The shaded vertical bars in this figure indicate periods of recession. The bars extend from the peak to the trough of each recession.

The large forecast error for the high-tax share for 2001-2002 (shown in Table 9) resulted in part from legislation enacted after that forecast was made. The legislation, passed in March 2002, allowed firms to deduct more from their profits for depreciation charges ("bonus depreciation"). That change depressed the measure of profits that CBO forecasts as part of the high-tax share of GDP. More than 1 percentage point of the 2.6 percentage point error for 2001-2002 stemmed from that bonus depreciation.

Five-Year Projections

In projecting real growth for the more distant future, measured here as five years ahead, the Administration's errors were larger than CBO's (see Table 10), but the record of various Administrations' projections of the relationship between the tax base and GDP was better than CBO's.

Accuracy in the longer term is obviously important for budgetary planning over several years. Neither the Administration nor CBO, however, considers its projections to be its best guess about the year-to-year course of the economy. The Administration's annual projections are based on the adoption of the President's budget as submitted, and for most years, CBO has considered its projections an indication of the average future performance of the economy if major historical trends continue. Neither institution attempts to anticipate cyclical fluctuations within the projection period.

CBO's projections of longer-term growth in real output were closer to the actual values than the Administration's were in 13 of the 23 forecasts, while they were the same on four occasions. The average errors for 1976-1998 indicate that the Administration's projections had an upward bias of 0.4 percentage points, compared with a bias of zero for CBO. The Administration's larger bias occurred mainly because of the projections it made in early 1976 through 1979. During that period, the projections of both CBO and the Administration were too optimistic, in part because both agencies presented their projections at that time as target rates of growth and because real growth was extraordinarily low during the back-to-back recessions of 1980 and 1982. Through the subsequent years of expansion until the 1990-1991 recession, the upward bias was much smaller in CBO's projections. In the five-year-ahead projections made between 1992 and 1998, both CBO and the Administration underpredicted long-term growth. The reasons were the surprisingly strong economy of the late 1990s and, to a lesser extent, the upward revisions to BEA's estimates of the rate of growth.

Those underpredictions of real growth also resulted in underestimates of long-run growth in nominal output, but the errors for nominal growth in the 1990s were smaller than those for real growth (see Table 11). That is because most forecasters have overestimated inflation in recent years. Between 1976 and 1998, CBO and the Administration appear to have done equally well in projecting five-year growth in nominal output. Moreover, as the record since 1982 shows, both forecasters compared well with the Blue Chip consensus.

As noted in the discussion of two-year forecasts, the difference between the two major inflation measures is also important for five-year budget projections. The error statistics indicate a downward bias, meaning that the projections of the difference in the growth of the CPI and the GDP price index over five-year horizons were too low in virtually all of the periods (see Table 12). However, at least 0.2 percentage points of the 0.5 percentage-point bias for CBO's projections stems from revisions in the methods used for calculating the GDP price index. During the period examined here, BEA shifted from using a measure based on 1972 dollars to one based on 1982 dollars, then one based on 1987 dollars, and finally to one based on chained prices (see the appendix). Those successive rebenchmarkings caused previously published estimates of the growth of the GDP price index to be revised downward. Therefore, forecasters at the time were projecting a GDP price index that had grown more quickly than the historical data now indicate. Similarly, on the basis of current historical data, the CPI now appears to have grown more, relative to the GDP price index, than the growth indicated by the data at the time the projections were made.

Even after the errors are adjusted for the effect of the revisions, a slight downward bias remains for all three forecasters. That bias indicates that projections of the relationship between those two price measures tend to contribute to optimistic budget projections. CBO's forecasting record is slightly better than those of the various Administrations and about the same as that of the Blue Chip.

The final five-year projection record examined here is that of the change in the high-tax share of GDP. The mean absolute error of the projections of the change in the high-tax share is large, although 0.3 percentage points of CBO's 1.8 percentage-point error stemmed from the first projection shown, that of 1980-1984 (see Table 13). As with some other variables, the errors show alternating periods of optimism and pessimism. The projections made early in 1991 through early 1997 indicated less growth in the tax base relative to GDP than actually occurred; but the last five-year-ahead projection shown here, that of early 1998, turned out to overstate the tax share. Difficulties in forecasting the statistical discrepancy and the nontaxable component of labor income are clearly major causes of errors in five-year projections as well as in two-year forecasts.

The Administration's projections were generally better than CBO's for this variable, with the Administration closer in 12 of the 19 periods. (As noted previously, the Blue Chip does not forecast wages and salaries.)
 

APPENDIX: Choosing Sources of Historical and Forecast Data

Evaluating CBO's forecasting record requires compiling the basic historical and forecast data for growth in real and nominal output, inflation in the consumer price index, interest rates, and taxable income.

Selection of Historical Data

Which historical data to use for the evaluation was limited by the availability of actual data and the nature of the forecast variables examined. Although CBO, the Administration, and the Blue Chip consensus all published the same measure for real output growth, selecting a historical series was difficult because of periodic benchmark revisions to the actual data.⁽⁴⁾ By comparison, not all of the forecasters published the same measures for CPI inflation and interest rates, but the selection of historical data for those series was clear-cut.

Growth in Real and Nominal Output. Historical two-year averages of growth in real output were developed from calendar year averages of the quarterly chain-type annual-weighted indexes of real gross national product and real gross domestic product published by the Bureau of Economic Analysis. The fact that several real GNP and GDP series were discontinued because of periodic benchmark revisions meant that they were unsuitable historical series. For example, during the 1976-1985 period, the three forecasters published estimates for a measure of growth in real GNP that was based on 1972 prices, which was the measure published by BEA at that time. In late 1985, however, BEA discontinued the 1972-dollar series and began to publish GNP on a 1982-dollar basis. As a result, an official series of values for GNP growth in 1972 dollars is not available for the years after 1984, and actual two-year average growth rates are not available to compare with the forecasts made in early 1984 and 1985.

From 1986 to 1991, forecasters published estimates of growth in real GNP based on 1982 prices. BEA revised the benchmark again in the second half of 1991: it discontinued the 1982-dollar GNP and began to publish GNP on a 1987-dollar basis. Consequently, the historical annual series for 1982-dollar GNP is available only through 1990, and actual two-year average growth rates are not available for the forecasts made in early 1990 and 1991. The forecasters then published estimates of growth in real GDP on a 1987-dollar basis until 1995, when BEA made another switch, late in the year, to a chain-weighted measure of GDP. Therefore, the historical annual series for 1987-dollar GDP ends with the 1994 annual value, and actual two-year average growth rates are not available for the forecasts made in early 1994 and 1995.

By periodically updating the series to reflect more recent prices, BEA's benchmark revisions yield a measure of real output that is more relevant for analyzing contemporary movements in real growth. But the process makes it difficult to evaluate forecasts of real growth produced over a period of years in series that are later discontinued. This comparison avoids the difficulties presented by periodic revisions of the data by using BEA's chain-type annual-weighted index of real GNP or GDP throughout the data series.⁽⁵⁾

Historical two-year averages for growth of nominal GNP and GDP were developed from calendar year averages of the quarterly values published by BEA.

CPI Inflation. CBO calculated two-year averages of inflation in the consumer price index from calendar year averages of monthly data published by the Bureau of Labor Statistics. Before 1978, the bureau published only one consumer price index series, now known as the CPI-W (the price index for urban wage earners and clerical workers). In January 1978, however, the bureau began to publish a second, broader consumer price index series, the CPI-U (the price index for all urban consumers). CBO's comparison of forecasts uses both series.

Until 1992, the Administration published its forecasts for the CPI-W, the measure used to index most of the federal government's spending for entitlement programs. In contrast, for all but four of its forecasts since 1979 (1986 through 1989), CBO based its inflation forecast on the CPI-U, a more widely cited measure of inflation and the one now used to index federal income tax brackets. the Blue Chip consensus has always published its forecast for the CPI-U. Although annual fluctuations in the CPI-U and CPI-W are virtually indistinguishable, the indexes differ in some years. For that reason, CBO used historical data for both series to evaluate the alternative forecasting records.

Interest Rates. CBO used monthly data published by the Board of Governors of the Federal Reserve System to calculate two-year averages of nominal short- and long-term interest rates.

The forecasts of short-term interest rates were compared using historical values for two measures of the interest rate on three-month Treasury bills: the new-issue rate and the secondary-market rate. The Administration forecasts the new-issue rate, which corresponds to the price of three-month bills auctioned by the Treasury Department--that is, it reflects the interest actually paid on that debt. CBO forecasts the secondary-market rate, which corresponds to the price of the three-month bills traded outside the Treasury auctions. Such transactions occur continually in markets that involve many more traders than do Treasury auctions. As a result, the secondary-market rate provides an updated evaluation of short-term federal debt by the wider financial community. Blue Chip has alternated between those two rates; it published the new-issue rate from 1982 to 1985, switched to the secondary-market rate during the 1986-1991 period, and then returned to the new-issue rate beginning in 1992. Clearly, there is no reason to expect the two rates to differ persistently; indeed, the differences between their calendar year averages are minuscule.

CBO likewise compared the various forecasts of long-term interest rates using historical values for two measures of long-term rates: the 10-year Treasury note rate and Moody's Aaa corporate bond rate. A comparison of forecasts is not possible before 1984 because not all of the forecasters published projections of long-term interest rates before then. For forecasts made in early 1984 and 1985, CBO projected the Aaa corporate bond rate. Beginning with its early 1986 forecast, however, CBO switched to the 10-year Treasury note rate. The Administration has always published its projection for the 10-year Treasury note rate, but Blue Chip published the Aaa corporate bond rate until January 1996, when it switched to the 10-year Treasury rate.

CBO calculated separate historical values for real short-term interest rates using the nominal short-term interest rate and the inflation rate appropriate for each forecaster. In each case, the two-year average nominal short-term interest rate was discounted by the two-year average rate of inflation. The resulting real short-term interest rates were similar.

Taxable Income. Through its direct influence on projections of federal revenues, the forecast for taxable income plays a critical role in determining the accuracy of budget projections. The income measure examined here--wage and salary distributions plus the book value of corporate profits--combines the two sources of income to which tax receipts are most sensitive. Considering wages and profits together is appropriate because the effective rates of taxation on wages (including payroll and income taxes) and on corporate profits are nearly the same. In addition, those tax rates exceed the rate at which other income sources (such as interest income) are taxed.

Although the level of taxable income is the factor that most directly affects federal revenues, historical estimates of that level are subject to substantial statistical revision. As a result, using the level of taxable income would distort the comparison of forecasts. Instead, the forecasts are presented here as changes in taxable income as a share of total output; the historical revisions, carried forward consistently to projections, should not affect projections of revenues. Moreover, the change in taxable income as a share of total output is closer to the concept that macroeconomists consider when they construct their forecasts.

Sources of Forecast Data

For everything except taxable income, this evaluation used the calendar year forecasts and projections that CBO has published early each year since 1976, coinciding with the publication of the Administration's annual budget proposals. The Administration's forecasts were taken from its budget in all but one case: the forecast made in early 1981 came from the Reagan Administration's revisions of President Carter's last budget. The corresponding CBO forecast was taken from CBO's published analysis of President Reagan's budget proposals. That forecast did not include the economic effects of the new Administration's fiscal policy proposals.⁽⁶⁾

The average two-year forecasts in the Blue Chip consensus survey, which are published monthly, were taken from those published in the same month as CBO's forecasts. Because the Blue Chip consensus did not begin publishing its two-year forecasts until the middle of 1981, the first such forecast available for this comparison was published in early 1982. Average five-year projections, however, are published by Blue Chip only two or three times a year. All but one of its five-year projections used in this evaluation were published in March; the 1980-1984 projection of real output was published in May.

Some of the CBO forecasts for wages, salaries, and corporate profits that are used here were not published in CBO's annual reports. Instead, they were taken from CBO's files of unpublished forecasts. CBO has published forecasts for wages and salaries regularly since 1985 but has published forecasts for book profits only in recent reports.

Table 1a.
Summary Measures of Forecast Performance for Two-Year Averages
(In percentage points)
		CBO		Administration		Blue Chipa
Growth Rate for Real Output (1982-2001)
	Mean error	-0.5		-0.3		-0.5
	Mean absolute error	1.1		1.2		1.1
	Root mean square error	1.3		1.4		1.3
Growth Rate for Nominal Output (1982-2001)
	Mean error	0.2		0.4		0.2
	Mean absolute error	1.1		1.2		1.1
	Root mean square error	1.4		1.5		1.3
Inflation in the Consumer Price Index (1982-2001)
	Mean error	0.5		0.4		0.5
	Mean absolute error	0.7		0.8		0.8
	Root mean square error	0.9		0.9		0.9
Nominal Interest Rate on Three-Month Treasury Bills (1982-2001)
	Mean error	0.5		0.1		0.5
	Mean absolute error	1.0		1.0		1.0
	Root mean square error	1.3		1.3		1.2
Nominal Long-Term Interest Rate (1984-2001)
	Mean error	0.2		-0.2		0.3
	Mean absolute error	0.6		0.9		0.6
	Root mean square error	0.7		1.0		0.7
Real Interest Rate on Three-Month Treasury Bills (1982-2001)
	Mean error	0		-0.3		0
	Mean absolute error	0.9		0.9		0.8
	Root mean square error	1.1		1.2		1.1
Difference Between Inflation in the CPI and GDP Price Index (1982-2001)
	Mean error	-0.2		-0.3		-0.2
	Mean absolute error	0.3		0.4		0.3
	Root mean square error	0.3		0.5		0.4
Change in Wage and Salary Disbursements Plus Corporate Book Profits as a Share of Output (1980-2001)
	Mean error	0.2		0.3		*
	Mean absolute error	1.0		0.9		*
	Root mean square error	1.2		1.2		*
SOURCES: Congressional Budget Office; Office of Management and Budget; Aspen Publishers, Inc., Blue Chip Economic Indicators; Department of Commerce, Bureau of Economic Analysis.
NOTES: The values reported here are derived from Tables 2 through 9. Errors are forecast values minus actual values; thus, a positive error is an overestimate.
* = not applicable.
a. Blue Chip is the average of approximately 50 private-sector forecasters.

Table 1b.
Summary Measures of Forecast Performance for Five-Year Averages
(In percentage points)
		CBO		Administration		Blue Chipa
Growth Rate for Real Output (1979-1998)
	Mean error	-0.3		0.1		-0.4
	Mean absolute error	0.6		0.9		0.6
	Root mean square error	0.9		1.0		0.8
Growth Rate for Nominal Output (1982-1998)
	Mean error	0.5		0.7		0.7
	Mean absolute error	0.9		0.9		0.9
	Root mean square error	1.0		1.1		1.1
Difference Between Inflation in the CPI and GDP Price Index (1983-1998)
	Mean error	-0.3		-0.5		-0.4
	Mean absolute error	0.3		0.5		0.4
	Root mean square error	0.4		0.5		0.4
Change in Wage and Salary Disbursements Plus Corporate Book Profits as a Share of Output (1980-1998)
	Mean error	-0.1		0.2		*
	Mean absolute error	1.8		1.4		*
	Root mean square error	2.2		1.8		*
SOURCES: Congressional Budget Office; Office of Management and Budget; Aspen Publishers, Inc., Blue Chip Economic Indicators; Department of Commerce, Bureau of Economic Analysis.
NOTES: The values reported here are derived from Tables 10 through 13. Errors are projected values minus actual values; thus, a positive error is an overestimate.
* = not applicable.
a. Blue Chip is the average of approximately 50 private-sector forecasters.