As the data plots show, the flux of relativistic electrons at geo-synchronous orbit undergoes significant diurnal variation. The cumulative daily fluence can also show a day to day variation of many orders of magnitude. Yet even with this wide variation, scientists have found a statistical link between the daily electron fluence and average solar wind speed. Baker et al [1990] showed that the daily >2 MeV electron fluence measured at geo-synchronous orbit could be predicted one day in advance using a linear filter technique with solar wind speed as the input. SEC has made further refinements to this technique, improving the forecast skill and extending the lead-time, resulting in the Relativistic Electron Forecast Model (REFM).

REFM forecasts are intended to supplement current SEC alerts by providing guidance as to the future likelihood of damaging relativistic electron events. Since the REFM output is a daily fluence forecast, not flux, it applies most directly to the high fluence threshold. In fact the graphical model output displays a warning message during periods when the 72-hour fluence exceeds or is forecast to exceed 10⁹ pfu. High 24-hour fluence values can also be used as a proxy for high flux levels. A study of data measured between 1995 and 2000 showed that when the 24-hour fluence exceeds 3 × 10⁷ pfu the SEC flux alert criteria is nearly always met (false alarm ratio = 0.07, miss frequency = 0.06). With this threshold, REFM fluence forecasts can be used to predict SEC flux alerts.

Return to Top

Opening the main REFM web page loads the most recent model run. A plot of the output dominates the page. In addition to the plot, the page contains links to text files of model output and recent performance statistics for both the ACE and Wang-Sheeley (WS) modes. There is also a link to a verification plot.

Standard Plot

The standard REFM plot displays roughly 30 days of observed and forecast data. Output from both the ACE and WS modes are shown on the same plot; previous forecast values are kept on screen for comparison with observed data. An example from a period of high electron fluence in 1996 is given below, along with a description of the plot's features.

Plot Features:

1. Plot symbols correspond to the 24-hour >2 MeV electron fluence at geo-synchronous orbit, either observed or forecast. For example, in the plot shown above, the most recent observed value corresponds to data from 24/00 to 25/00 UTC; the +1 Day Forecast is valid 25/00 to 26/00 UTC.
2. A legend in the lower left corner indicates the symbol and color-coding used for the Observed, Forecast (ACE), and Forecast (WS) values.
3. Two vertical dashed lines indicate the most recently observed 24-hour period; in the example above this corresponds to 24/00 to 25/00 UTC.
4. The lower right corner displays the latest observed and 1-4 day forecast values in a tabular format. The values are also color-coded in the same manner as the plot symbols. The date shown is valid at the beginning of the 24-hour period.
5. Missing values, observed or forecast, are indicated on the bottom of the plot with a color-coded dashed line; the word "Missing ---" also appears in the legend.
6. When the 72-hour fluence exceeds 10⁹ pfu, a warning message is displayed. Red lines (solid for observed and dashed for forecast) appear at the top of the plot, corresponding to the applicable days. A warning message also appears in the legend.

Verification Plot

Return to Top

The Relativistic Electron Forecast Model (REFM) predicts the >2 MeV 24-hour electron fluence at geo-synchronous orbit. It is based on a linear prediction filter (Baker, 1990) that uses average solar wind speed as its input. An offset is employed to help account for additional physical processes that can dramatically affect the electron fluence (the fluence can drop 2-3 decades in 24 hours without a corresponding change in solar wind speed).

The linear prediction filter is a relatively simple statistical technique in which correlation coefficients are created from historical data by solving a set of linear equations (Levinson, 1947). These coefficients are then applied to real-time data. The REFM uses 30 days of solar wind speed data (currently from the ACE spacecraft) to predict the 24-hour fluence with +1 to +8 days of lead-time. In the ACE mode the +1, +2, and +3 day forecasts each use a different set of coefficients. In the Wang-Sheeley mode, forecast solar wind speed values are substituted for observed data to extend the electron fluence forecast out to +8 days.

As mentioned earlier, the electron fluence can exhibit dramatic day-to-day variability, typically associated with strong shocks and/or geomagnetic storming. This is believed to be due to both magnetic field stretching, in which the electron population is temporarily shifted into or out of geo-synchronous orbit, and sudden electron loss to either the magnetopause or upper atmosphere. The basic linear filter is unable to account for these processes; given high solar wind speeds it will forecast high electron fluences, even if the electrons actually experience a sudden decrease or increase.

To help account for this short-term variability as well as longer-term drifts, REFM adjusts the output of the basic linear prediction filter with a flux offset. REFM compares previous forecasts to the observed values, computes an offset, and applies it as a correction to the current forecast. The number of days used to compute the offset varies; for example, the +1 Day ACE mode uses only the most recent forecast/observed pair, while the +2 and +3 Day ACE mode and WS mode all use the past 20 days of forecast/observed pairs. These numbers were chosen to maximize the performance statistics of each mode.

Although the offset improves the statistics, at best the flux offset's response lags a day behind the environment's variability. This causes an associated rise in model errors during solar maximum when coronal mass ejections (CMEs) and subsequent strong geomagnetic storms are prevalent. For this reason, the REFM performs best during solar minimum. In addition to the problems associated with electron variability, the WS mode is limited by the capability of the Wang-Sheeley technique. The WS forecasts of solar wind speed assume a "quiet sun", i.e. no CMEs, further increasing the problems for the WS mode during solar max. Fortunately the largest electron events occur during solar minimum when both the ACE and WS modes perform at their best.

Return to Top

Verification was done using standard measures of skill. The Skill Score (SS) is a relative measure in which performance of the target forecast is compared to a reference forecast. The SS is defined as

The most commonly used reference forecast is simply the average observation (i.e. sample mean); when this is used, the SS is referred to as the Prediction Efficiency (PE). Other common reference forecasts include Persistence (using the most recent observation as a forecast) and Recurrence (using the observation from 27 days prior). Perfect predictions result in an SS = 1.0, positive and negative values respectively indicate performance better or worse than the reference forecast.

The Skill Score given above is appropriate for gauging model performance as a whole. To measure the model's ability to forecast rare events, such as 72-hour fluence exceeding 10⁹ pfu, we used the True Skill Statistic (TSS) which is an absolute measure based on contingency tables (Doswell, 1990). The TSS ranges from –1.0 (no correct forecasts) to 1.0 (perfectly correct), with positive values indicating a forecast skill exceeding that expected from pure chance.

As explained in Model Specifics, the REFM performs best during solar minimum conditions, when large geomagnetic storms are less frequent. Therefore, in order to test the model under optimal conditions, scores were tabulated using data from the fall of Cycle 22 into the rise of Cycle 23, 1996 through 1999. Model coefficients for this test were generated using data from 1995, guaranteeing independent results. Due to the large day-to-day variations in electron fluence, the Skill Scores were calculated using the logarithm (base 10) of the observed and forecast values.

The following table lists the Skill Scores for each forecast period, +1 to +8 days, and each forecast mode, ACE or WS. Three standard reference forecasts were used: sample mean (Prediction Efficiency), persistence, and recurrence.

The results show that the ACE mode has good forecast skill, especially on day one, but that in the short-term the results are only slightly better than using persistence. The WS mode shows lower skill (PE) that continues to drop as the lead-time increases. This is not too surprising since the WS mode relies on other model output to create its forecast. In fact, for the +2 and +3 day forecasts, the ACE mode out-performs the WS mode in all three categories. In both modes the Prediction Efficiencies drop with increasing lead-time, while the skill relative to Persistence increases.

The next table shows the True Skill Statistics for the model's ability to predict 72-hour fluences that exceed 10⁹ pfu. For comparison, scores are also shown when persistence is used to forecast these events.

Return to Top

Baker, D.N., et al, "Deep Dielectric Charging Effects Due to High Energy Electrons in the Earth’s Outer Magnetosphere", J. Electrost., 20, 3, 1987.

Baker, D.N., et al, "Linear Prediction Filter Analysis of Relativistic Electron Properties at 6.6 Re", J. Geophysical Res., 95, A9, 1990.

Doswell, et al, "On Summary Measures of Skill in Rare Event Forecasting Based on Contingency Tables", Weather and Forecasting, 5, December 1990.

Reagan, J.B., et al, "Space Charging Currents and their Effects on Spacecraft Systems", IEEE Trans. Electr. Insul., EI-18, 345, 1983.