CONCLUSIONS
The Chief Administrative Officer (CAO) has taken a number of significant
actions to ensure the successful implementation of the House's
new financial management system. For example, the CAO: (1) established
an Executive Steering Committee, which meets on a weekly basis,
to oversee the management of the project; (2) executed a cross-servicing
agreement with U.S. Geological Service (USGS) to implement the
Federal Financial System (FFS) within the House; (3) appointed
a Project Manager and Project Director to manage the project on
a day-to-day basis; (4) established an implementation team to
execute the implementation of the project; (5) completed
a preliminary implementation strategy and project plan; (6) operated
FFS in parallel with the old financial management system for the
months of October and November 1995 and reconciled the data in
each system for this time period; (7) completed a significant
amount of analysis for determining how the House will use FFS;
(8) worked with USGS on a daily basis providing information about
the House and making system set-up decisions; (9) reviewed
all work products of USGS to verify completeness and accuracy;
and (10) developed new and revised financial management reports
for the House.
Despite these accomplishments, a formal System Development Life
Cycle (SDLC) methodology, which provides a structured approach
to managing and conducting a system implementation project, was
not being used for the FFS Implementation Project. As a result,
critical FFS implementation activities were not always carried
out in the most effective manner to ensure project success. In
addition, the FFS project has not followed all of the steps outlined
in the August 3, 1995 Committee on House Oversight Resolution.
In particular, the functional requirements for the set-up
of FFS, custom interfaces to FFS, and FFS custom reports have
not been developed. Also, the current project work plan does not
identify the level of effort required for the individual tasks.
Thus, the managers of the project cannot determine the extent
of resources necessary to complete the project within the established
timeframes. Consequently, the implementation of the FFS core system
may be delayed beyond the current target date of March 4, 1996.
Furthermore, should the system be implemented prematurely, the
House may be implementing a system which does not meet its needs.
Additionally, a premature implementation could prompt system development
and implementation shortcuts and errors, which could prove more
costly and time consuming to correct later. In order to ensure
the timely and successful implementation of all four phases of
FFS, the following actions are critical: (1) adoption of
a formal SDLC methodology; (2) completion of functional requirements;
(3) increased top management attention and direction; (4)
development of a more realistic work plan which includes the level
of effort for each individual task; and (5) assignment of a sufficient
number of qualified staff necessary to meet the level of effort
required by each work plan task.
In addition, comprehensive system testing, certification, and
accreditation are essential for ensuring FFS data integrity (i.e.,
reliability, accuracy, completeness, and timeliness). Although
we found no indication that the House intends to curtail or shortcut
testing functions, these activities are essential to the operational
success of any system development and implementation project.
Curtailing or shortcutting such important system development activities
could result in the implementation of a system that produces erroneous
data and introduces other operational problems. Often these problems
ultimately result in a system which does not meet user needs and
requires fixes that significantly escalate costs and resource
requirements. Therefore, the FFS Implementation Team must thoroughly
complete their planned system testing, certification, and accreditation
activities before FFS implementation. We cannot over-emphasize
the importance of these activities.
So far, the FFS Implementation Team has focused most of its efforts
on getting the FFS system ready for the March 4, 1996 implementation
date and has completed relatively few steps to make the necessary
work flow, organizational, and procedural changes to support using
the new system. Without having a number of needed changes in place
by March 4, 1996, the House may not fully achieve the benefits
of FFS or make substantial improvements in the House's financial
management practices. This is particularly important for the House
because the changes from the old to the new financial management
system will be so dramatic.
A contributing factor to these problems has been the complexity
of the FFS implementation at the House. In comparison to many
financial management system implementations, this FFS Implementation
Project is very complex and difficult. Many times when FFS is
implemented in a Federal organization, it is replacing a system
and procedures that comply with routine financial management standards
followed by private organizations and Federal government agencies.
However, the implementation of FFS at the House is challenging
because before FFS the House did not follow standard accounting
policies and methods and its Office of Finance was not organized
or staffed to produce financial information routinely produced
by organizations of similar size. This meant that no framework
was in place that would have: (1) aided the House in addressing
the requirements of system users; or (2) allowed the House
to more easily develop or update financial and operational policies
and procedures. In effect, by implementing FFS, the House is initiating
a new way of doing business that it is simply not accustomed to.
Another dynamic factor of the FFS Implementation Project is its
aggressive schedule. Because the House needed to quickly establish
a standard general ledger to accumulate and summarize transactions
for decision making, an aggressive implementation schedule was
set. The amount of work scheduled for completion with the House's
FFS Implementation Project typically takes twice as long in similar
financial management system implementations.
A third factor that contributes to the complexity of the FFS Implementation
Project is the fact that the House has limited experienced resources
available to dedicate to the project. Because little time was
available between the decision to implement FFS and the start-up
of the project, the House could neither quickly obtain sufficient
internal or external resources experienced in implementing financial
management systems, or quickly familiarize external personnel
with the House's operations. Ideally, these resources would have
been immediately available to assist the House in completing many
of the implementation tasks.
RECOMMENDATIONS
We recommend that the Chief Administrative Officer: (1) identify
and follow a formal SDLC methodology, such as the National Institute
of Standards and Technologies' Special Publication 500-153, for
the implementation of FFS; (2) identify, analyze, and document
functional requirements for the implementation of FFS; (3) reorganize
and staff the FFS Implementation Team project management to more
effectively manage and support FFS implementation; (4) monitor
the progress of the project through tracking actual versus planned
activities on the project work plan, maintain an issues log that
includes key issues raised and their resolution, and present the
status of the work plan and issues to the Executive Steering Committee
on at least a bi-weekly basis; (5) develop a level of effort work
plan for the implementation of FFS; (6) review the work plan
and prioritize tasks to focus on tasks critical to Phases I and
II and the March 4, 1996 implementation date and review and prioritize
those tasks that are critical to Phases I and II, but do not require
completion until the end of March 1996; (7) add additional resources
to critical tasks.
We also recommend that the Chief Administrative: (1) continue
to thoroughly perform all activities related to system testing,
certification, and accreditation; (2) document and maintain the
results of verifying reference tables and testing activities;
(3) ensure the formal certification of FFS verification and testing
activities by FFS Implementation Team members, who are responsible
for overseeing the execution of FFS testing tasks; (4) ensure
the formal accreditation of FFS, prior to placing it into production,
by an appropriate senior House official (e.g., Associate
Administrator, Office of Finance); (5) report the results of certification
and accreditation to the Executive Steering Committee prior to
FFS implementation; (6) develop work flows to support the use
of FFS in the Office of Finance and other CAO offices prior to
March 4, 1996; (7) identify roles and responsibilities for staff
in the Office of Finance and other CAO offices to support using
FFS prior to March 4, 1996, and determine which positions will
be responsible for new activities associated with using FFS; (8)
align present staff to satisfy new roles and responsibilities
(e.g., reassigning present staff to new positions because
they have the experience and skills necessary to perform the required
duties) and hire new staff, if necessary; and (9) establish financial
management policies and procedures to complement the use of FFS
prior to March 4, 1996 and to support the implementation of additional
functionality in future phases.
MANAGEMENT RESPONSE
In the February 14, 1996 response to our draft report, the Office
of the CAO generally agreed to the findings and recommendations
in this report, and indicated that corrective actions have been
initiated for some areas and planned for the remaining areas.
OFFICE OF INSPECTOR GENERAL COMMENTS
We fully concur with the initial and planned actions of the CAO
and commend them on their prompt action with respect to the issues
raised in this report. Furthermore, since the issuance of our
draft report, we have had extensive discussions with CAO officials
regarding the FFS implementation process. Subsequently, the CAO
requested the Committee on House Oversight to extend the implementation
date giving the FFS Implementation Team more time to complete
the tasks required for system implementation.
I. INTRODUCTION
Background
On August 3, 1995, the Committee on House Oversight passed a resolution--Financial
Management System--directing that the Chief Administrative
Officer (CAO) "take such steps as might be necessary, in
consultation with the Inspector General, and subject to the review
and approval of the Chairman, in consultation with the Ranking
Minority Member, to implement a new Financial Management System
for the House no later than January 3, 1996." Such steps
included, but were not limited to:
House Audit dated July 18, 1995.
The Committee subsequently approved the CAO's December 13, 1995
request for extending this target date to March 4, 1996.
On August 25, 1995, the CAO established an Executive Steering
Committee to oversee the implementation of a new financial management
system for the U.S. House of Representatives (House). Accordingly,
in September 1995, the CAO formally entered into a cross-servicing
agreement with the U.S. Geological Survey (USGS), Washington Administrative
Service Center (WASC), to implement USGS's Federal Financial System
(FFS) for the House. (Informal discussions had been ongoing since
August 1995.) FFS is an off-the-shelf software package that is
owned and maintained by American Management Systems Inc. (AMS).
The FFS license that USGS has with AMS allows USGS to provide
cross-servicing to external Federal government agencies.
Full implementation of FFS was originally scheduled to be completed
in four phases by December 31, 1996. Phase I involved identifying
and setting up the functionality of FFS to support parallel processing
of payment transactions and identifying the needs for custom interfaces
and reports. This phase was completed on September 30, 1995. Phase
II includes establishing the "Core" FFS system, custom
interfaces, and custom reports at the House and is now expected
to be completed by March 4, 1996. Phases III and IV will encompass
the reengineering of accounting procedures and the determination
and implementation of additional functionalities to be provided
by FFS. Target dates for Phases III and IV have not yet been adjusted
to reflect the impact resulting from the extension of the original
date for the "Core" FFS system. See Exhibit A for details
on the four phases of the FFS implementation.
The FFS software package complies with the Joint Financial Management
Improvement Program (JFMIP) requirements, which stipulate the
functional system requirements necessary to effectively manage
and report on Federal government financial transactions. Implementation
of FFS will enable the House to apply generally accepted accounting
principles and practices. In addition, it will provide decision
makers more relevant, understandable, and useable financial information,
consistent with information used by public and private sector
organizations.
On September 29, 1995, the Office of Inspector General (OIG) engaged
Price Waterhouse LLP to review, analyze, evaluate, and provide
advice on various aspects of the House's implementation of FFS.
To assist in ensuring project progress, the OIG and Price Waterhouse
(OIG/PW) team serves as advisors to the House's FFS Implementation
Project Executive Steering Committee and FFS Implementation Team
by proactively providing advice on various implementation efforts.
Objective, Scope, And Methodology
The objective of this audit was to assess the adequacy of implementation
activities carried out during Phases I and II of the FFS Implementation
Project at the House. We focused our review on the implementation
tasks outlined in the resolution passed by the Committee on House
Oversight on August 3, 1995. We also evaluated the effectiveness
of project management over the House FFS Implementation Team.
As advisors on the FFS Implementation Project, the OIG/PW team
proactively provided advice throughout Phases I and II to facilitate
project progress. That advice included key issues dealing with
the management of the project, the importance of fully defining
FFS functional requirements, and the importance of a detailed
work plan for managing the project's aggressive schedule. We also
emphasized the importance of ensuring that the system is tested,
certified, and accredited for accuracy and completeness prior
to implementation. These issues are discussed in detail in this
report.
We conducted our review during the period of October 1, 1995 through
January 12, 1996. However, our review encompassed financial management
system development and implementation activities during the period
of August 1, 1995 through January 12, 1996. Audit fieldwork was
conducted at the House's Washington, D.C. offices, primarily in
the Office of Finance, and at the USCG/WASC in Reston, Virginia.
We conducted our review in accordance with Government Auditing
Standards, issued by the Comptroller General of the United
States. In conducting this review, we attended FFS Implementation
Team meetings, reviewed FFS Implementation Project-related documents,
reviewed data entered into FFS, and interviewed FFS Implementation
Team members. Our review included a comparison of FFS implementation
activities to the following:
-- National Institute of Standards and Technology's (NIST) Special Publication 500-153 - Guide to Auditing for Controls and Security: A System Development Life Cycle Approach.
-- Price Waterhouse LLP - System Management Methodology: Package Software Implementation.
Internal Controls
Within the scope of this audit, we evaluated internal controls related to the implementation of the FFS. The audit disclosed serious internal control weaknesses in the FFS system development life cycle management and process which could adversely affect the successful implementation of the FFS. The internal control weaknesses we identified are described in Findings A and C of this report.
Prior Audit Coverage
Proposed New Financial Management System Will Not Meet the House's Needs And Should Be Terminated (Report No. 95-CAO-02, dated May 12, 1995): This review evaluated the functional adequacy of the proposed Financial Management System (FMS) and the system development life cycle procedures that were utilized in the development of the system. This report recommended that the system be terminated and also made four recommendations to improve the House's systems development practices as well as management oversight. The CAO agreed with the recommendations and is taking actions to correct the deficiencies identified.
Problems Plagued The House's Financial Operations (95-CAO-16, dated July 18, 1995): This review assessed opportunities to increase efficiency and effectiveness of operations within the Office of Finance. This report identified opportunities for savings and made 26 recommendations with respect to operations in the Office of Finance, primarily through improved internal controls and operational efficiencies. The CAO agreed with the recommendations and is taking actions to correct the deficiencies identified.
Audit of Financial Statements for the 15-Month Period Ended December 31, 1994 (Report No. 95-HOC-22, dated July 18, 1995): This review presented the results of Price Waterhouse's audit of the House of Representatives' financial statements for the 15-month period ended December 31, 1994. This report disclosed that the House lacked the organization and structure to periodically prepare financial statements that are accurate and reliable. The report also identified serious internal control deficiencies related to the House's financial operations. The report made 59 recommendations to correct 14 material weaknesses in the House's internal controls structure. The CAO agreed with the recommendations and is taking actions to correct the deficiencies identified.
I. FINDINGS AND RECOMMENDATIONS
Finding A: The Implementation Of FFS May Not Meet Its Currently Scheduled Target Date
A formal SDLC methodology, which provides a structured approach to managing and conducting a system implementation project, was not being used for the FFS Implementation Project. As a result, critical FFS implementation activities were not always carried out in the most effective manner to ensure project success. In addition, the FFS project has not followed all of the steps outlined in the August 3, 1995 Committee on House Oversight Resolution. In particular, the functional requirements for the set-up of FFS, custom interfaces to FFS, and FFS custom reports have not been developed. Also, the current project work plan does not identify the level of effort required for the individual tasks. Thus, the managers of the project cannot determine the extent of resources necessary to complete the project within the established timeframes. Consequently, the implementation of the FFS core system may be delayed beyond the current target date of March 4, 1996. Furthermore, should the system be implemented prematurely, the House may be implementing a system which does not meet its needs. Additionally, a premature implementation could prompt system development and implementation shortcuts and errors, which could prove more costly and time consuming to correct later. In order to ensure the timely and successful implementation of all four phases of FFS the following actions are critically needed: (1) adoption of a formal SDLC methodology; (2) completion of functional requirements prior to the implementation of FFS; (3) increased top management attention and direction; (4) development of a more realistic work plan which includes the level of effort for each individual task; and (5) assignment of a sufficient number of qualified staff necessary to meet the level of effort required by each task set forth in the work plan.
Generally accepted project management practices
Well-run implementations of financial management systems generally are managed using a formal SDLC methodology. The SDLC methodology provides a structured approach to managing and conducting a system implementation project. It includes guidelines for planning the project and identifying requirements through the implementation of the system. One of the key factors in the successful execution of a SDLC methodology includes a project manager who is: (1) a full-time employee of the organization, having a detailed knowledge of how the organization operates and the strengths and weaknesses of its existing accounting and financial systems, and (2) experienced in using a formal SDLC methodology.
One important responsibility of the project manager is to maintain and manage the project's work plan. The work plan lists all tasks and supporting sub-tasks that are required to complete the project. Dates and resources are assigned to tasks on the work plan based on the level of effort and the project's scope. The project manager ensures that the resources assigned to the work plan are experienced and can meet their task commitments. The project manager also ensures that staff critical to the implementation project are sufficiently dedicated to the project and do not have conflicting responsibilities. By using standard project management practices and an adequately staffed implementation team, the project manager helps ensure the completion of the project in a timely and efficient manner. To the extent an implementation project is more challenging, either because deadlines are tight or because the project is technically or logistically complex, it is common to infuse the project team with more senior leadership and more experienced personnel.
Lack of functional requirements definition hinders a timely implementation of FFS
To date, functional requirements for the set-up of FFS, custom interfaces to FFS, and FFS custom reports have not been fully identified and documented. Most implementations of financial management systems begin with identifying, analyzing, and documenting the organization's requirements for the functions to be met by the new system. Documented requirements provide the foundation for the system's implementation. All subsequent tasks are driven by the requirements. For example, a custom report is develop based on the requirements for the purpose and content of the report. In effect, the requirements are the blueprints for implementing the system and developing custom pieces. Without requirements as a guide, an implementation project could prove more costly and time consuming because unnecessary or avoidable steps are carried out.
Because the FFS implementation did not identify, analyze, and document functional requirements for the set-up of FFS, custom interfaces to FFS, and FFS custom reports, the project has not progressed as efficiently as possible. The FFS Implementation Project has experienced delays because requirements were not identified and documented for the set-up of FFS and the development of its custom pieces. Completion of tasks were sometimes delayed until the FFS Implementation Team identified and communicated requirements to USGS. Often, requirements were identified and communicated in a piece-meal manner. This caused additional work or rework for USGS and the House. Without the full involvement of users in implementing FFS and its custom pieces, there is not a level of assurance that the use of FFS and the data produced by it will meet the House's needs.
The original implementation schedule did not include the time or resources needed to complete a thorough, well thought out requirements definition. Although the Committee on House Oversight's Resolution for the implementation of a new financial management system included the requirement that functional requirements be developed, the FFS Implementation Team did not plan for the completion of this task. This strategy was primarily based on the fact that the FFS implementation was subject to an extremely aggressive schedule that did not allow for the thorough documentation of requirements. In addition, the FFS Implementation Team believed that FFS complies with JFMIP requirements and that the House's main financial management requirements would be met through the implementation of FFS. However, to avoid delays and inefficiencies as discussed above, requirements for additional functionality in future phases need to be identified, analyzed, and documented at the beginning of these phases. Also, this will help ensure that financial management requirements specific to the House will be identified.
Increased top management attention and direction are needed to ensure the success of the FFS implementation
More day-to-day involvement, direction, and leadership from senior staff of the CAO is needed for the House's FFS Implementation Project. This is particularly important given the aggressive implementation schedule for FFS. While senior management officials have provided strategic direction, day-to-day project management is largely conducted by contracted personnel who lack authority over other House offices involved with the project, and detailed knowledge of the House's financial management practices and past problems.
To date, the FFS Implementation Project has been managed on a day-to-day basis by a contractor, appointed as the Project Director, who had prior experience managing large system development and implementation projects, but had no previous work experience with the House before being hired for the FFS Implementation Project. While a project director with this background may be effective for many implementation projects, the House's problems are unique and its target implementation date is very aggressive. In addition, the FFS Implementation Team was understaffed and it was necessary for the Project Director to spend much of her time completing implementation tasks. Thus, she could not devote all her time to performing project management activities.
Because the House is implementing a new system with new accounting practices under an aggressive schedule and with limited resources, the FFS Implementation Project needs to be managed more aggressively than the typical system implementation. The House's day-to-day management of the FFS Implementation Project requires a senior House officer with authority over all House officers and project team members involved with the project, and detailed knowledge of the House and its past problems.
In order to correct this deficiency, on January 16, 1996, the CAO assigned overall coordination of the FFS Implementation Project to the Director of Internal Controls and Continuous Improvement on a full-time basis. We believe this should increase the level of involvement, direction, and leadership from top management needed to provide a more aggressive management approach to the FFS Implementation Project.
The work plan and scheduled completion dates may not be realistic
A work plan for managing the FFS Implementation Project was not developed until mid-December. Furthermore, the work plan was not used for managing the completion of tasks or used to track actual versus planned activities until mid-January. While preparing a work plan is difficult and time-consuming in the midst of a demanding implementation schedule, it is still an essential part of determining whether enough staff time is available and whether resources are efficiently deployed.
Furthermore, the FFS implementation work plan has slipped and needs to be modified. Many tasks on the work plan that were scheduled to be completed by now have not been completed. Examples of these tasks are as follows:
In addition, two tasks on the work plan that were scheduled to be started by now have not been started. These tasks are as follows:
Some tasks that could have been postponed without loss of function were on the critical path of implementation. For example, in support of the March 4 implementation date, some of the interface testing was scheduled for completion in late December and early January, when the completion of testing these interfaces could have been postponed until the end of February or early March.
The dates presented in the work plan were not based on the level of effort (i.e., hours) required to complete the tasks. Instead, the work plan presented tasks with beginning and ending dates only. Thus, the required resources for each of the tasks on the work plan were not based on the level of effort required to complete the project tasks. As a result, the prioritization and timing of tasks were not well coordinated. In addition, because the work plan did not present the hours required to complete the tasks and instead presented tasks with beginning and ending dates only, we could not analyze the work plan to determine its reasonableness. Without identifying the level of effort, it is not feasible to determine, with a sufficient level of confidence, whether or not the March 4, 1996 implementation date is achievable.
Key resources are assigned to multiple tasks simultaneously and may be over committed
The FFS Implementation Project is not currently supported by adequate resources. The original team structure was composed of mainly full-time individuals and a few part-time individuals. (See Exhibit B, page 20.) The current FFS Implementation Team is mainly composed of part-time individuals. (See Exhibit B, page 21.) In addition, many of the key team members are assigned to multiple tasks simultaneously. Examples of this are as follows:
To validate that team members could meet the commitments in the work plan, the Project Director met with each staff member to review the assigned tasks and the time frames for completing them. This is not an adequate measure to ensure that resources are not over committed because most of the team members do not have experience implementing financial management systems. Therefore, they may not be familiar with the level of effort required to complete their assigned tasks.
Because the FFS Implementation Team members were multi-tasked and sometimes over committed, team members have not been able to complete their assigned tasks in accordance with the work plan. In the absence of a more precise means of assessing available resources, the House should identify additional resources (including additional contractor support) which can be assigned to critical areas in order to avoid any additional slippage in the schedule.
In addition, the House had limited experienced resources available to dedicate to this project. Because little time was allowed between the decision to implement FFS and the start-up of the project, the House could neither quickly obtain sufficient internal or external resources experienced in implementing financial management systems, nor quickly familiarize external personnel with the House's operations. Ideally, these resources should have been immediately available to assist the House in completing many of the implementation tasks.
Recommendations
We recommend that the Chief Administrative Officer:
1. Identify and follow a formal SDLC methodology, such as NIST's Special Publication 500-153, for the implementation of FFS.
2. Identify, analyze, and document functional requirements for the implementation of FFS.
3. Reorganize and staff the FFS Implementation Team to more effectively manage and support FFS implementation.
4. Monitor the progress of the project through tracking actual versus planned activities on the project work plan, maintain an issues log that includes key issues raised and their resolution, and present the status of the work plan and issues to the Executive Steering Committee on at least a bi-weekly basis.
5. Develop a level of effort work plan for the implementation of FFS.
6. Review the work plan and prioritize tasks to focus on those tasks critical to implementing Phases I and II and meeting the March 4, 1996 implementation date. For example, the following critical tasks should be assigned higher priorities for completion by March 4, 1996:
Review the work plan and prioritize other tasks that are critical to implementing Phases I and II, but do not require completion until the end of March 1996, such as:
7. Add additional resources to critical tasks. We have identified the following critical tasks which appear to be behind schedule and are likely to require additional resources to ensure completion by March 4, 1996:
Management Response
On February 14, 1996, the Office of the CAO fully concurred with this finding and all seven recommendations including subparts (see Appendix). According to the response, several initiatives are either underway or planned to ensure the successful implementation of the House's new financial management system. Key actions taken and planned include: (1) adopting and applying a formal SDLC methodology in the evaluation and testing of the Core FFS system; (2) preparing and documenting functional requirements for the Procurement and Check writing modules, and subsequent system requirements; (3) reorganizing and staffing of the FFS Implementation Team to more effectively manage and support FFS implementation; (4) instituting a detailed project tracking system and issues log to present to the Executive Steering Committee at regular meetings; (5) incorporating the level of effort for tasks in work plans for work subsequent to the implementation of the core system; (6) reviewing the work plan and prioritizing tasks to focus on those tasks critical to Phases I and II and the March 4, 1996 implementation date versus those critical to Phases I and II but can be completed by the end of March 1996; and (7) assigning additional House and WASC resources to critical implementation tasks.
Office of Inspector General Comments
The CAO's actions are responsive to the issues we identified and, when fully implemented, should satisfy the intent of our recommendations. Furthermore, since the issuance of our draft report, we have had extensive discussions with CAO officials regarding the FFS implementation process. Subsequently, the CAO requested the Committee on House Oversight to extend the implementation date giving the FFS Implementation Team more time to complete the tasks required for system implementation.
Finding B: Comprehensive System Testing, Certification, And Accreditation Activities Are Essential For Ensuring Data Integrity Within FFS
Comprehensive system testing, certification, and accreditation are essential for ensuring FFS data integrity (i.e., reliability, accuracy, completeness, and timeliness). Although we found no indication that the House intends to curtail or shortcut testing functions, these activities are essential to the operational success of any system development and implementation project. Curtailing or shortcutting such important system development activities could result in the implementation of a system that produces erroneous data and introduces other operational problems. Often these problems ultimately result in a system which does not meet user needs and requires fixes that significantly escalate costs and resource requirements. Therefore, the FFS Implementation Team must thoroughly complete their planned system testing, certification, and accreditation activities before FFS implementation. We cannot over-emphasize the importance of these activities.
Most system implementation projects follow a formal SDLC methodology to plan, manage, and perform testing activities, including certification and accreditation. From the start of the project, a strong focus and emphasis on testing the system is necessary to ensure that the system meets users needs and produces reliable, accurate, complete, and timely data. A formal testing plan provides a framework that, when combined with trained and experienced project personnel, assists in ensuring a successful and effective approach to the conduct of system testing activities. The overall result of using a formal testing plan is the implementation of an effective, quality controlled, accurate system.
The FFS Implementation Project work plan includes system testing, certification, and accreditation tasks. The tasks that are critical to ensuring the accuracy of FFS and their order of importance are as follows:
For each reference table reviewed, the data should be compared to the source documents used originally in determining the data to be included in the reference tables. Furthermore, the verification process must be properly documented, including any errors or omissions identified in the reference tables. Procedures should also be developed to establish control over changes to the reference tables and ensure timely corrections.
For each testing task, the FFS Implementation Team should develop test plans to guide the execution of the testing. The test plan should include any assumptions concerning the execution of the test, what condition will be tested, how the condition will be tested, and the expected results. Both valid and invalid conditions should be thoroughly tested. After testing the condition, the person responsible for the test plan should update the test plan with the actual results and compare the actual results to the expected results. In instances where expected results are not achieved, necessary steps should be taken to resolve the discrepancy thoroughly and in a timely manner. The action taken to resolve the discrepancy should also be recorded in the test documentation. This documentation should be maintained for future system testing needs.
Designated FFS Implementation Team officials, responsible for overseeing the execution of FFS testing tasks, should review the test plans, test results, and validation of data in the reference tables to certify that all necessary steps were taken to thoroughly validate the correct processing of FFS. For example, the certification should state that the interface (or system) was tested in accordance with the test plan and met the documented and approved system specifications. In addition, the certification should include a recommendation on whether or not to authorize operation of the interface (or system) based on risks associated with any known exceptions. This certification process provides the framework for system accreditation. The FFS accreditation should be performed by a senior House official (e.g., the Associate Administrator, Office of Finance) assuring that the data it will produce will be reliable, accurate, complete, and timely, and will meet user needs, and the system will operate as planned.
In mid-December, the FFS Implementation Project work plan was developed with thorough, well planned system testing activities. The completion and approval of system testing, certification, and accreditation activities should assure the successful implementation and operation of FFS. However, if these activities can not be performed thoroughly by March 4, 1996, these activities must be completed prior to FFS implementation; thereby, requiring the postponement of the implementation date in order to ensure the accuracy of the FFS.
Recommendations
We recommend that the Chief Administrative Officer:
Management Response
On February 14, 1996, the Office of the CAO fully concurred with this finding and all five recommendations (see Appendix). According to the response, the CAO will continue to test, certify, and accredit the system in accordance with the agreed upon plans. Also, all tests and verifications are being documented and the documentation is being maintained. Upon completion of the testing, each module will be formally certified and tables will also be certified at completion of verification. After all modules and tables are certified, individually and collectively, and the system has completed a monthly cycle, FFS will be formally accredited by the Associate Administrator for Finance. In addition, results of tests, verifications, certifications, and accreditation will be promptly shared with the Steering Committee.
Office Of Inspector General Comments
The CAO's actions are responsive to the issues we identified and, when fully implemented, should satisfy the intent of our recommendations.
Finding C: Work Flow, Organizational, And Procedural Changes Are Critical To Successfully Implementing FFS
So far, the FFS Implementation Team has focused most of its efforts on getting the FFS system ready for the March 4, 1996 implementation date and has completed relatively few steps to make the necessary work flow, organizational, and procedural changes to support using the new system. Without having a number of needed changes in place by March 4, 1996, the House may not fully achieve the benefits of FFS or make substantial improvements in the House's financial management practices. This is particularly important for the House because the changes from the old to the new financial management system will be so dramatic.
All formal SDLC methodologies require or imply the need to realign the work flows, organization, and procedures to support a new system. This is a generally accepted practice in almost all organizations in the private sector and the Federal government. Changing the existing work flows, organizational, and procedures to support the new system allows for the organization to achieve the full benefits from implementing a new system. All financial management needs can then be better met and the organization can operate more efficiently and effectively.
The House has and plans to continue expending efforts on getting FFS ready for processing by March 4, 1996. However, additional steps are necessary to ensure that the CAO staff are ready for the changes that will accompany implementing FFS. Because FFS is so dramatically different from FMS, the House will be forced to initiate a new way of doing business. Consequently, the CAO needs to make changes to its work flow, organization, and procedures to support the new way of doing business. Specific changes that are needed to support these new processes on and after March 4, 1996 include the following:
The original strategy for implementing FFS did not include providing the time or resources needed to make work flow, organizational, and procedural changes to support using FFS by March 4, 1996. House management did not realize the magnitude of the changes needed to support a financial management system that will provide the House the capability to follow standard accounting policies and methods. Thus, steps have not been taken to prepare House office staff to take advantage of the improved capabilities and use of the new financial management system. Because the House is initiating a new way of doing business for which it is not accustomed, extensive changes are needed. Without these changes, the House may not recognize the full benefits of its new financial management system.
Recommendations
We recommend that the Chief Administrative Officer:
1. Develop work flows to support the use of FFS in the Office of Finance and other CAO offices prior to March 4, 1996. This includes the following tasks:
2. Identify roles and responsibilities for staff in the Office of Finance and other CAO offices to support using FFS prior to March 4, 1996. Determine which positions will be responsible for new activities associated with using FFS. New roles and responsibilities should include, but not be limited to, the following:
3. Align present staff to satisfy new roles and responsibilities (e.g., reassigning present staff to new positions because they have the experience and skills necessary to perform the required duties) and hire new staff, if necessary.
4. Establish financial management policies and procedures to complement the use of FFS prior to March 4, 1996 and to support the implementation of additional functionality in future phases. Procedural changes needed as of March 4, 1996 include the following:
Procedural changes needed for future phases include the following:
Management Response
On February 14, 1996, the Office of the CAO generally concurred with this finding and all four recommendations, including subparts (see Appendix). According to the response, initiatives are either underway or planned to improve the work flow, organization, and procedures to ensure the successful implementation of the House's new financial management system. Examples of key actions planned include: (1) developing work flow tasks to support the use of FFS in the Office of Finance and other CAO offices prior to March 4, 1996; (2) defining roles and responsibilities as workflows and procedures are determined; (3) reviewing data entry workflows with emphasis on assuring accuracy; (4) developing a reorganization proposal for approval by the Committee on House Oversight by February 29, 1996; and (5) establishing and documenting financial policies and procedures by March 4, 1996. In addition, the CAO plans to discuss with the Office of Inspector General the results of their review of data entry workflows in an effort to determine if additional actions are necessary during the implementation period.
Office of Inspector General Comments
The CAO's actions are generally responsive to the issues we identified and, when fully implemented, should satisfy the intent of our recommendations. However, we do not necessarily agree with the CAO's response concerning the impracticality of reviewing daily FFS transaction reports. In our view, daily FFS transaction reports can be compared to source documents on a sampling basis. Nevertheless, considering that the CAO plans to discuss the results of their review of data entry workflows with us, we will work closely with the CAO's staff to develop and implement sound and effective processes and procedures to ensure data integrity.
Exhibit A
Page 1 of 1
August 1, 1995 - September 30, 1995 |
· Budget Execution Subsystem (appropriations) · General Ledger Subsystem (using Standard General Ledger) · Accounts Payable Subsystem (payment vouchers) | |
Original Schedule:
October 1, 1995 - January 3, 1996 Revised Schedule: October 1, 1995 - March 4, 1996 | · Interfaces: ·· Office Telephone Support ·· Office of Stationary and Supplies ·· Office of Equipment Services ·· Photography Office ·· House Recording Studio ·· General Services Administration ·· Payroll · Project Cost Accounting Subsystem (allowances) · Planning Subsystem (CAO office budget) · Purchasing Subsystem (obligations) · Accounts Payable Subsystem (recurring expense setup) · Automated Disbursements Subsystem · Accounts Receivable Subsystem (cash receipts) · Monthly Close Subsystem · User training/feedback | |
Original Schedule:
January 4, 1996 - March 31, 1996 Revised Schedule: Not yet determined | · Reengineer/document procedures · Procurement Desktop · To be determined: ·· Inventory Subsystem ·· Fixed Assets Subsystem ·· External Reports Subsystem ·· Travel Subsystem ·· Planning Subsystem ·· Document Tracking Subsystem ·· Cost Allocation Subsystem ·· Accounts Receivable Subsystem (receivables) | |
Original Schedule: April 1, 1996 - December 31, 1996 Revised Schedule: Not yet determined | · Not yet determined |
Exhibit B
Page 1 of 2
Exhibit B
Page 2 of 2