On March 1-2, 2000, the Residual Risk Subcommittee conducted a peer review of an Agency draft case study of the residual risk assessment methodology for the secondary lead smelter source category (USEPA, 2000). The SAB understands that the Agency plans another iteration, including additional data collection and analysis before the results are considered for use in a regulatory context. The review of the seven-volume set of material focused on eight specific questions that are addressed in detail in the accompanying SAB report.
In short, the Subcommittee concludes that the Agency developed a useful, self-described "work-in progress." The methodology used in this interim work product, as far as it currently goes, is consistent with the methodology described in the Report to Congress. Further, many of the assumptions used are consistent with current methods and practice. The case study provides an example of how the approach presented in the Report might be implemented. However, it also raises a number of concerns that we have provided in our report on this document (EPA-SAB-EC-ADV-00-005 "An SAB Advisory on the USEPA's Draft Case Study Analysis of the Residual Risk of Secondary Lead Smelters"). The major concerns will be highlighted here.
Because the Subcommittee has not yet seen a full residual risk analysis and, thus, is unable to comment on the complete process, a number of important concerns were identified that should be addressed. Specifically, this interim analysis does not include the following important elements: 1. an ecosystem risk assessment 2. a health risk assessment that includes population risks 3. a full analysis of uncertainty and variability 4. a computer model for assessing multimedia transport and fate that has been adequately evaluated 5. a clear description of the process and how the assessments will be linked to the eventual risk management decisions. With respect to the specific approaches taken in the interim analysis, a number of questions are discussed in detail in the Subcommittee's report.
Ecosystem Risk Assessment
One of the greatest shortcomings of the case study in its incomplete state is that only the first stage screening analysis has been done for the ecological risk assessment. Even in this screening the top carnivore species were not included. This is the group of organisms at greatest risk from persistent and accumulated toxic chemicals. While the Office of Air Quality Planning and Standards (OAQPS) acknowledges that a full ecological risk assessment is needed, the Subcommittee is disappointed at the pace at which the assessment is being developed and implemented for ecology and natural resources. It would appear that a more concerted and scientifically complete analysis will be needed in order to meet the mandate of the Clean Air Act Amendments (CAAA) with respect to ecological risk.
Health Risk Assessment That Includes Population Risks
Regarding the health risk assessment portion of the case study, the Subcommittee finds that, within the limitations of data and resources, the approaches employed by the Agency were able to qualitatively identify potentially high human health risk situations. However, the Subcommittee also concluded that the currently available science presented in the working document is insufficient to be comfortable with the quantitative values estimated by the models currently used. In particular, the analysis calls into question the ability of the model to reliably quantify the amount of the deposited contaminant transferred in the food chain. In addition, the current risk assessment will have to be further developed in order to include population risks if it is to meet the needs of the Agency.
Computer Model for Assessing Multimedia Transport and Fate That Has Been Adequately Evaluated
The case of multimedia computer models is one of the other major areas with which the Subcommittee has concerns. It seems that the models were applied without due consideration of the plausibility of the assumptions and the physical meaning of the results. In several cases, results presented in the draft report were implausible in that the predicted concentrations would have produced immediately observable results on the affected human and ecological populations. For example, ambient lead concentrations measured because of the National Ambient Air Quality Standard for Particulate Lead could be used to test the concentrations at site boundaries of these facilities. Thus, it suggests that overly conservative estimates were likely to have been used. Such results could be eliminated if an iterative process were used in which implausible results are flagged so that the Agency can make appropriate revisions in the model and/or its inputs, and the model run again. A number of plausibility checks were described by the Subcommittee, and in public comments, that would provide checkpoints in the analysis and, thereby, indicate the need for alternative assumptions and recalculation. Inclusion of these checkpoints would be helpful to both the Agency and the reader.
In addition the models being used need to undergo rigorous peer review. In this test case, a model, IEM-2M, originally used for mercury movement in the environment was modified for lead. However, this model was never rigorously reviewed even for its utility in the mercury modeling. The Agency has been developing a new multimedia exposure model, Total Risk Integrated Methodology (TRIM), that has undergone an initial review by the SAB that was encouraging. However, it appears now that the completion of this model and its use in risk assessment has been slowed so that it may not be available in the near term. This delay produces serious doubts in any of the assessments that have to be based on a temporary model that has not been subjected to careful external scrutiny.
Full Analysis of Uncertainty and Variability
The lack of a more rigorous treatment of uncertainty and variability may lend an aura of precision to the risk estimates in the case study that is not warranted and could, thereby, be misleading for Agency decision makers. In particular, the uncertainty analysis omits some important aspects of uncertainty and does not clearly distinguish between uncertainty and variability.
Clear Description of the Process and How the Assessments Will Be Linked to the Eventual Risk Management Decisions
Moving beyond the strictly technical aspects of the document on which the SAB has been asked to provide advice, I would like to share with you my comments on what the subcommittee understood to be the Agency's intention to make decisions based on these results. Specifically, the Agency is mandated under Section 112(f) of the Clean Air Act to conduct the residual risk assessment and to make a decision about whether or not further regulation is necessary in order to protect public health and the environment. In particular, as stated in the Agency's response to the previous SAB review of the Report To Congress (SAB-EC-98-013), "the decision made with the results of the screening analysis is [either] no further action or refine the analysis, while the decision made with the results of the more refined analysis is [either] no further action or consider additional emissions control."
As discussed above, as currently presented, the results of the refined analysis will provide essentially the same answer as the initial screening analysis; that is, an even more refined analysis is needed. Therefore, the case study has not achieved its decision objective, and another level of analysis or iteration is needed. A better-informed decision will be possible if the results of the case study more fully reflect both the best estimate of the risk combined with an adequate uncertainty/variability analysis that will more clearly define the range of risks.
An important policy question arises as to how good do such residual risk assessments need to be. The understanding the RRS came to during its discussion is that when the form of controls specified in Title III was being considered by Congress a decade ago, the expectation for the level of these residual risk analyses was quite low. The scientific basis of risk assessment has grown considerably over the past ten years and thus, the level of expectation from the scientific community such as those who have served on the SAB Subcommittee has risen considerably. Thus, the Subcommittee has expressed its concerns regarding future assessments.
The present source class, secondary lead smelters, is a relatively data-rich category. Because of the existence of the lead National Ambient Air Quality Standard (NAAQS) and the concern for blood lead levels in children, there are more data in the vicinity of these source types than are likely to be available for other HAPs from other source types. The basic Congressional approach of imposing controls and assessing residual risk is a sensible response to the problem of HAPs emissions. However, the number of HAPs and the number of source types, coupled with the limited data on speciated emissions and quantitative dose-response information, makes the residual risk task into a substantial one.
At this time, it appears that there have not been sufficient resources provided to EPA to allow their Office of Air Quality Planning and Standards (OAQPS) to collect and assess all of the pertinent data from EPA, state/local air quality, and public health agencies that could be fruitfully brought to bear on this problem. For example, the Subcommittee was told that it had not been possible to get the lead NAAQS monitoring data from AIRS to provide checks on the fugitive emissions estimates because of resource limitations.
There are certainly not sufficient resources to permit the testing of specific HAPs for their toxicity if those dose-response data are not already available. Such testing would be expensive and may not be the best use of limited resources. In the case of secondary lead smelters, only seven of the 50 identified HAPs were excluded from the residual risk assessment due to the lack of dose-response data. This lack of data will likely pose much greater problems when other source categories are addressed in the future. Such data gaps could lead to the omission of compounds from the assessment, resulting in a subsequent underprediction of the residual risk. It may be possible to utilize computational chemical methods to provide at least an estimation of the possible risk. However, that would require some limited additional effort.
Accordingly, I wish to use this opportunity to express the RRS's concern regarding the level of analysis that can and should be done to assess the residual risk as part of the control of hazardous air pollutant emissions. The RRS believes it is possible to provide more quantitative and useful human health and ecological risk assessments than is currently envisioned for a reasonable investment of additional resources for data collection and some additional outside expertise as appropriate. The resulting assessments will be much more credible.
As we all know well, science alone does not a decision make. Science can inform but policy decides when making regulatory judgments. I say this, because many non-scientific considerations are taken into account when making decisions (e.g. legal precedent, policies, values, economics, technical feasibility, etc.). Each must be considered carefully and applied wisely if the decisions are to be effective and widely accepted by the public. However, while the decision inputs based on values, politics and other social considerations are often debatable, we expect the science to be based on facts determined by measurable, repeatable observations of nature. More explicit recognition of this problem by members of Congress could help the Agency carry out its duties more effectively and could help provide the public with a clear understanding of how Congress interprets National priorities.
I want to express my gratitude to the Members of the Committee for inviting me and giving me the opportunity to discuss the SAB Residual Risk review message with you. I look forward to your questions.